An Experimental Analysis of Steady-State Response Rate Components onVariable Ratio and Variable Interval Schedules of Reinforcement

Phil ReedSwansea University

Three experiments explored a novel approach to analyzing the different components of response rate thatare produced by exposure to free-operant schedules of reinforcement. It has been suggested that overallresponse rate comprises a tendency to initiate responding, and to continue to respond once the bout isinitiated. Previous post hoc analyses of interresponse times (IRT) data have suggested several features ofthese different aspects of responding that the current experimental procedure broadly confirmed.Increasing the size of a variable interval (VI) schedule decreases the number of “burst-initiation”responses, but has less effect on responding once the burst has been initiated (Experiment 1); that themajor difference between a variable ratio (VR) schedule and a VI schedule, is not in the number of“burst-initiation” responses, but in the number of “within-burst” responses, with shorter interresponsetimes that are emitted, with greater numbers of such “within-burst” responses being emitted on a VRschedule (Experiments 2 and 3).

Keywords: response components, response initiation, burst responses, variable ration, variable interval,rats

Response rates maintained by variable ratio (VR) schedules ofreinforcement are typically higher than those maintained by vari-able interval (VI) schedules of reinforcement (Ferster & Skinner,1957; Peele, Casey, & Silberberg, 1984; Zuriff, 1970). This find-ing is one of some generality across species, having been noted inrats (e.g., Dickinson, Peters, & Schechter, 1984; Reed, 2007),pigeons (e.g., Peele et al., 1984; Ferster & Skinner, 1957), andhumans (e.g., Dack, McHugh, & Reed, 2009; Raia, Shillingford,Miller, & Baier, 2000). Apart from the strength of the empiricalfinding, this effect has been the source of a number of therapeuticapplications (see Lattal & Neef, 1996, for a review) and hasformed the basis of a number of investigations into the factors thatcontrol free-operant responding (e.g., Baum, 1973, 1993; Morse,1966; Reed, 2007).

Theoretical interpretations of this effect tend to focus eitheron the molecular factors that control behavior, such as thereinforcement of interresponse times (IRT; e.g., Morse, 1966),or the molar features of the contingencies, such as the feedbackfunction relating the rate of response to the obtained rate ofreinforcement (e.g., Baum, 1973). In such experiments, theoverall rate of response is related to some feature of the con-tingency, such as the length of the reinforced IRT, or the natureof the feedback function, in order to determine the degree ofcontrol which that aspect of the environment has over behavior.

Typically, such studies have shown that nonhuman rates ofresponding on free-operant schedules of reinforcement arestrongly related to the molecular features of the environment(e.g., Cole, 1999; Morse, 1966; Peele et al., 1984). However,there also is some evidence, obtained from contingencies otherthan VR and VI schedules, that the molar feedback function cancontrol rates of responding not only in humans (McDowell &Wixted, 1986; Reed, 2007), but also in nonhumans (Bowers,Hill, & Palya, 2008; Reed, 2007). That different aspects of theenvironment can control behavior suggests that a strict molarversus molecular dichotomy may be a false one, and that bothaspects of the environment will control behavior under differentconditions.

One feature of these theoretical debates is the assumption thatrate of response can be treated as a unitary dependent variable, andthat the fluctuation in the rate of response can be unambiguouslyattributed to a single process, or to a set of processes. However,there is an extensive literature that questions whether response rateper se is a strong measure of the degree of learning about a singleaspect of the environment (Blough, 1963; Nevin, 1979; Pear &Rector, 1979; Reed, Schachtman, & Hill, 1988). Some have sug-gested that overall response rate may be the product of the devel-opment of “units” of behavior, which consist of several integratedresponses (see Bacha-Mendez, Reid, & Mendoza-Soylovna, 2007;Reed & Morgan, 2006). In the case of a VR schedule, such a unitmay comprise a group of responses, whereas, on a VI schedule thisunit may comprise a response pattern that is characterized by along IRT (e.g., see Morse, 1966). It may be that such units areemitted with roughly equal probability on a VR and a VI schedule,but as units on the former schedule contain more individual re-sponses, the rate of response is higher on this schedule than on theVI schedule (see Shull & Grimes, 2003). If this were the case, thenthe overall rate of response may not be a completely ideal measure

This article was published Online First August 16, 2010.Phil Reed, Department of Psychology, Swansea University.These data were first presented at the First Conference of the European

Association for Behavior Analysis, Parma, Italy, 2003. Thanks are due toLisa A. Osborne for her support.

Correspondence concerning this article should be addressed to PhilReed, Department of Psychology, Swansea University, Singleton Park,Swansea, SA2 8PP, United Kingdom. E-mail: p.reed@swansea.ac.uk

Journal of Experimental Psychology: © 2010 American Psychological AssociationAnimal Behavior Processes2011, Vol. 37, No. 1, 1–9

0097-7403/10/$12.00 DOI: 10.1037/a0019387

1

of learning on such schedules, although it should be acknowledgedthat its use certainly has brought a number of advances in theo-retical understanding.

More generally, such a view may be taken to suggest that overallresponse rate may be a product of periods of responding that areseparated by periods of disengagement from responding (Blough,1963; Pear & Rector, 1979; Shull, Gaynor, & Grimes, 2001). Anumber of recent articles have attempted to analyze free-operantresponding in such a manner and to explore the consequences ofadopting such a view for the understanding of schedule perfor-mance (e.g., Shull, 2004; Shull et al., 2001; 2002; Shull & Grimes,2003). These studies have used an approach in which the respond-ing of organisms on various schedules of reinforcement is ana-lyzed in terms of their patterns of IRTs. The IRTs are displayed asa log survivor plots; that is, the proportion of IRTs emitted (rela-tive to the opportunity to emit that IRT) that are longer than anygiven interval, are plotted against that interval. This method ofdisplaying the IRTs has the potential to produce a number ofdifferent patterns of data. For example, if responding is randomlyemitted at a constant rate across a session, then the log survivorplot would be a decreasing straight line with a simple exponentialdecay (see Shull et al., 2001, p. 250, for discussion). However,Shull et al. (2001, p. 250) noted that a feature of the data whichemerged when such plots are produced for rats’ nose-poke re-sponses, was that the slope of the line was not uniform, but rathercomprised a sharply decreasing initial portion, followed by aportion with a shallower negative gradient. This finding has beentaken to suggest that there are two different types of responding inoperation: a set of shorter IRTs, which are taken to reflect “within-burst” responding; and a set of longer IRTs, which are taken toreflect “burst-initiation” responses.

In a series of studies, it has been suggested that the two types ofresponding (“burst-initiation” and “within-burst” responses) areoperated on by different aspects of the contingencies. For example,the rate of “burst-initiation” responses has been noted to increasewith increasing rates of reinforcement, but “within-burst” re-sponses are not so clearly affected by this aspect of the contin-gency (Shull et al., 2001; Shull, 2004). That is, increasing the sizeof a VI schedule (i.e., reducing the rate of reinforcement) decreasesthe number of “burst-initiation” responses, but has less effect onresponding once the burst has been initiated. Moreover, it has beensuggested that the major difference between a VR schedule and aVI schedule, is not in the number of “burst-initiation” responses,but in the number of shorter “within-burst” responses that areemitted (see Bowers et al., 2008; Shull et al., 2001; Shull &Grimes, 2003); with greater numbers of “within-burst” responsesbeing emitted on a VR schedule. These results have lead to theclaim that “burst-initiation” responses are controlled by motiva-tional factors (e.g., rate of reinforcement), and the “within-burst”responses are controlled by schedule factors (e.g., reinforcement ofparticular IRTs). This suggestion mirrors the distinction betweenthe strengthening and shaping properties of reinforcement made byMorse (1966; see Shull et al., 2001).

Obviously, such a finding has many implications of the under-standing of performance on free-operant schedules of reinforce-ment, and is certainly worthy of further study. The form of anal-yses suggested by Shull et al. (2001, 2002; see also Shull &Grimes, 2003; Shull, 2004) is clearly ingenious, but there are anumber of problems that remain with these demonstrations. First,

a problem with the log survivor plot analyses is that they will notprovide a good estimate of the numbers of “burst-initiation” and“within-burst” responses if the assumed exponential model doesnot fit the data well, and/or if the distribution of “burst-initiation”responses is similar to the distribution of “within-burst” responses.Although these assumptions may fit data derived from rats’ nose-poking, they may well not fit other response systems (see Kessel& Lucke, 2008; Kulubekova & McDowell, 2008). Second, suchpotential problems with the log survivor analyses, as outlinedabove, have been documented, to some extent, in practice. Forexample, Bowers et al. (2008) have suggested that identification ofsuch break points is not always easy for VI schedules, and,moreover, that VR schedules may produce several such breakpoints. Furthermore, it has been suggested that, for some re-sponses, such as the nose-poke of the rat, the break point may beeasily identifiable (Shull et al., 2001), but that for other responses,such as the more commonly studied lever-pressing in rats, theremay be less clear points of inflection (see Shull & Grimes, 2003);and that key-pecking in pigeons may be similarly resistant to astraightforward analyses (see Bennett, Hughes, & Pitts, 2007;Bowers et al., 2008).

Thus, a number of potentially important findings emerge fromthe literature on the constituents of response rate; that is, “burst-initiation” is dependent on rate of reinforcement, and effect that isespecially noticeable in VI schedules; increasing the VR schedulevalue increases to number of within-burst responses; and VR andVI schedules produce similar numbers of “burst-initiation” re-sponses, but differ in the number of “within-burst” responses.However, these suggestions may substantiated by a complimentaryexperimental approach in order to corroborate and explore thesuggestions made on the basis of the analytic approach outlinedabove. One potential approach to achieve this goal may be pro-vided by the development of an experimental approach based onthat suggested by Mechner (1992), in terms of the “revealedoperant.” In this approach, two discrete manipulanda are providedto the subject, a response to one manipulandum marks the start ofa response, which is then conducted on another manipulandum.This clearly demarks the “burst-initiation,” from “within-burst,”responses. A similar procedure has been conducted, on a smallscale, by Pear and Rector (1979) using pigeons as subjects. In thisstudy, pigeons were required to make one response (mounting asmall platform), which would gain access to an illuminated key,which they could then peck to produce grain, according to a seriesof interval schedules.

In the current series of experiments, rat subjects will be pre-sented with two levers. At the start of a session, one of the levers(the “burst-initiation” lever) will have a light, located above thelever, illuminated. A response to this lever will be taken to markthe start of a response, and will extinguish the light over that lever.A light located above another, spatially distinct, lever will then beilluminated, and that lever will be operative for the schedule to becompleted. If the rats cease responding for a period of time priorto the schedule being completed, this is taken as the burst beingterminated (Mellgren & Elsmore, 1991). The light above theresponse-burst lever will be extinguished, the lever will cease to beoperative, and the light above the burst-initiation lever will beilluminated again. This way the numbers of “burst-initiating,” and“within-burst” responses, can be clearly demarked from one an-other. Of course, the length of time without a response that

2 REED

determines the end of a burst is an arbitrary (albeit preexperimen-tally defined) criterion. A value of 5s was chosen for all of theseexperiments based on previous data from rats lever-pressing thatsuggested the point of inflection occurred around 5s for this typeof response (Bowers et al., 2008; Shull & Grimes, 2003).

Using this procedure it was hoped to established whether theobservations made from the post hoc analysis of IRT data by Shullet al. (2001; see also Shull & Grimes, 2003), and noted above,would be corroborated.

Experiment 1

In terms of responding on VI schedules of reinforcement, pre-vious analyses of IRT patterns has suggested that within-burstresponding is not largely (or at least not clearly) altered by anincrease in the rate of reinforcement, but that the numbers ofburst-initiations are increased by this manipulation (Pear & Rector,1979; Shull et al., 2001; although later work has noted a modestpositive relation between bout length and rate of reinforcement onVI schedules; see Shull, Grimes, & Bennett, 2004). This is areasonably well established finding and, thus, would seem a rea-sonable place to start to investigate if the current procedure wouldproduce a similar pattern of results to those obtained from a posthoc analysis of IRTs (Shull et al., 2001). This would serve tovalidate the current procedure for the examination of less wellestablished findings, but would also serve to bolster the findings ofShull et al. (2001; Shull, 2004) in the light of the difficulties ininterpreting the IRT log survivor plots mentioned by Bowers et al.(2008).

To this end, in this experiment, three groups of rats respondedon the current contingency, each group with a different randominterval (RI) schedule of reinforcement: RI 30s, RI 60s, or RI 180s.If previous results were to be replicated, it would be expected thatoverall response rates would increase as the mean interval valuedecreased (i.e., as the rate of reinforcement increased). However,this would be reflected in an increase in the number of “burst-initiation” responses made during the session, rather than in thenumber of “within-burst” responses being made.

Method

Subjects. Twenty-four male, experimentally naıve ListerHooded rats served in the present experiment. The subjects wereapproximately 3–4 months old at the start of training and had afree-feeding body-weight range of 355–435g. The subjects weremaintained at 85% of their free-feeding body weight throughoutthe study. The subjects were housed in groups of four, with waterconstantly available in the home cage.

Apparatus. Four identical operant conditioning chambers(Campden Instruments Ltd.) were used. Each chamber measured23.5 cm � 23.5 cm � 20.cm (length � width � height). Eachchamber was housed in a light and sound-attenuating case, venti-lated by a fan that provided background masking noise (65db[A]).Each chamber had two levers, positioned either side of a centrallylocated food hopper. The hopper was covered by a hinged clearPerspex flap. There were jeweled houselights above each of thelevers. Reinforcement consisted of one 45-mg food pellet and wasdelivered to the food hopper.

Procedure

All of the subjects received two 30-min sessions of magazinetraining, during which the levers were retracted from the cham-ber, and food was delivered according to a variable time (VT)60s schedule. Following this training, all the rats were giventwo 15-min sessions of lever-press training with a continuousreinforcement schedule in operation, one session on each lever(the other lever being retracted from the chamber). After lever-press training, both of the levers were inserted into the chamber,and the rats were given four 30-min sessions of training on aconcurrent RI 15s RI 15s schedule (i.e., the probability ofreinforcement becoming available for a response after eachsecond was 1/15). There was no change over delay in operationto encourage alternation responding that would form part of theresponse sequence to be studied at a later point in the experi-ment. Following these four sessions, the rats experienced afurther six 60-min sessions of a concurrent RI 15s RI 15sschedule, but this time the lights above the levers would both beilluminated for 45s, during which time the schedule was inoperation, or would both not be illuminated for 75s, duringwhich no reinforcement was available. At the end of these sixsessions, the rats were making the majority of their response inthe presence of the light, and very few in its absence.

Following this training, the rats were then all exposed to the targetcontingency, whose parameters were systematically altered over thecourse of 10 further training sessions. In these sessions, initially thelight above the left lever was illuminated, and the light above the rightlever was not illuminated. A response to the left lever extinguishedthe light above the left lever, and illuminated the light above the rightlever. While the light above the left lever was illuminated, a responseto the right lever had no programmed consequences. After illumina-tion of the light above the right lever, responses to the right levercould be reinforced according to a particular schedule. Failure torespond to the right lever for a period of time while the light abovethat lever was illuminated resulted in the extinction of the light abovethe right lever, and responses to that lever having no programmedconsequences. The light above the left lever would then be illumi-nated, and the sequence started again. The schedule continued tooperate irrespective of which light was illuminated. During thesesessions, an RI 15s schedule was in operation for all of the rats(which could be satisfied by a response to the right lever onlywhen the right light was illuminated). During the first two ses-sions, failure to make a response to the right lever for 10s while theright light was illuminated resulted in the extinction of the rightlever. This value was systematically reduced by 1s every twosessions, so that by the ninth and tenth sessions of this phase of thetraining, the value was 6s.

Following this training regime, the rats were divided into threegroups of eight rats. One group of rats responded on an RI 30sschedule, one group responded on an RI 60s schedule, and the finalgroup responded on an RI 180s schedule. The criterion value for allgroups for nonresponding extinguishing the light above the right leverwas 5s. All sessions lasted until the subjects had earned 30 reinforcers.The subjects were exposed to these contingencies for 90 sessions (andby the end of training around 70% of all responses emitted were toilluminated levers; only these responses were analyzed).

3RESPONSE RATE COMPONENTS

Results and Discussion

The data from the last six sessions of training, which were takento reflect steady-state performance, were analyzed. The mean(standard deviation) obtained numbers of reinforcers per minutefor the three groups were: RI 30s, 1.6 (0.1); RI 60s, 0.8 (0.1); RI180s, 0.4 (0.1). An analysis of variance (ANOVA) conducted onthese data revealed a statistically significant effect of group, F(2,21) � 324.19, p � .001, and Tukey’s Honestly Significant Dif-ference (HSD) tests revealed that all pairwise comparisons be-tween the groups were statistically significant, all ps � .001. Thesedata show that the programming of the three schedules was oper-ating broadly as planned.

The mean (standard deviation) overall responses per minute (allresponses to illuminated levers for the whole session period) forthe three groups were: RI 30s, 52.2 (16.0); RI 60s, 31.5 (6.1); RI180s, 13.8 (5.4). These data were analyzed by an ANOVA thatrevealed a statistically significant effect of group, F(2, 21) �27.44, p � .001. Tukey’s HSD tests revealed that all pairwisecomparisons between the groups were statistically significant, allps � .01. This difference corroborates what has long been known;that is, richer interval schedules of reinforcement produce higherrates of response than leaner interval schedules of reinforcement.In fact, the response rates obtained in this study were in approxi-mate proportion to the obtained rates of reinforcement in accor-dance with predictions based on the Matching Law (Herrnstein,1970; although it might be noted that simple proportionality is notthe only prediction that could be derived from Herrnstein’s equa-tion when applied to a single manipulandum). This suggests thatthe programming was producing the expected rates of responseoverall for the rats.

The mean (standard deviation) number of “burst-initiation” re-sponses made by the three groups per minute were: RI 30s, 11.1(2.3); RI 60s, 9.1 (1.3); RI 180s, 6.3 (1.5). An ANOVA revealeda statistically significant effect of group, F(2, 21) � 15.27, p �.001, and Tukey’s HSD tests revealed that the difference betweenthe RI 30s and the RI 180s groups, and that between the RI 60s andthe RI 180s groups, were statistically significant, ps � .05. Ofcourse, these response rates include periods of time in which itwould be impossible to make a burst-initiating response. Recalcu-lating the rate of “burst-initiation” responses after the time spentresponding in the bursts (including the 5-s periods without aresponse) revealed a similar numerical pattern to that above, butwith much inflated rates of response (due to relatively largeportions, around 50%, of the session being taken up in 5-s periodsspent not responding). These recalculated mean (standard devia-tion) responses per minute were: RI 30s, 107.1 (147.0); RI 60s,35.2 (18.2); RI 180s, 14.0 (7.1). An ANOVA revealed only amarginally statistically significant effect of group, F(2, 21) � 2.60,p � .08, most likely due to the large variance in the groups. Giventhis suggestion, an additional analysis, using an ANOVA con-ducted on the burst initiation rate following a square root trans-formation (as the variance will increase with the mean), revealeda statistically significant difference between the groups, F(2, 21) �5.51, p � .05. Tukey’s HSD tests conducted on these transformeddata revealed a difference between the RI 30s group and each ofthe other two groups. These differences partly corroborate whathas been apparent in the log survivor plots of IRT distributionsfrom single manipulandum situations, in that the number of “burst-

initiating” responses is determined by the rate of reinforcement(see Shull et al., 2001; Shull, 2004).

The group mean (standard deviation) number of “responses perburst” were: RI 30s, 3.9 (1.9); RI 60s, 2.5 (0.6); RI 180s, 1.2 (0.6).An ANOVA revealed a statistically significant effect of group,F(2, 21) � 9.95, p � .01, and Tukey’s HSD tests revealed thatonly the difference between the RI 30s and the RI 180s groups wasstatistically significant. It should be noted that not all “burst-initiation” responses were followed by responses to the “within-burst” lever; approximately half of the “burst-initiation” responsesfor each group were not followed by responses to the “within-burst” lever (RI 30s, 53%; RI 60s, 45%; RI 180s, 47%). Calculat-ing the numbers of responses per burst, in those bursts that pro-duced a response, gives the following group mean (standarddeviation) results: RI 30s, 21.2 (15.9); RI 60s, 12.5 (11.7); RI 180s,14.0 (14.0). An ANOVA conducted on these data revealed nostatistically significant effect of group, F � 1. While there werenumerical differences between the groups, and some of thesedifferences were significant, on the whole, these data relating tothe number of responses per burst show little clear differencebetween the groups. Such data corroborate what is apparent fromthe post hoc inspection of IRT distributions in previous reports(see Shull et al., 2001).

In addition, the group mean (standard deviation) rate of within-burst responding for the three groups (from the first response to the“burst” lever to the last, and excluding the 5s time of not respond-ing at the end of the bursts) was calculated, and was: RI 30s, 247.9(83.3); RI 60s, 142.3 (26.8); RI 180s, 61.8 (23.8). An ANOVArevealed a statistically significant effect of group, F(2, 21) �25.35, p � .001, and Tukey’s HSD tests revealed that all pairwisedifferences between the groups were statistically significant.Taken together with the nonsignificant trends toward greater num-bers of “within-burst” response noted above, these “within-burst”response rate data suggest that there may be a modest relationshipbetween this “within-burst” responding variable and the rate of RIreinforcement, as suggested in data presented by Shull et al.,2004).

In summary, the present experiment replicated almost all fea-tures of the log survivor plots produced previously, and the resultsare consistent with the analysis of the effects on “burst-initiating”responses, and “within-burst” responses produced by those reports.This suggests that the current experimental procedure for investi-gating these phenomena has criterion validity with other types ofpost hoc analysis, and the results from additional studies using thispresent procedure may well have validity in illuminating theseprocesses.

Experiment 2

The first experiment corroborated one prediction from the logsurvivor analysis of IRT patterns, that on RI schedules, numbers of“burst-initiation” responses vary directly with the rate of reinforce-ment, but that “within-burst” responses are largely unaffected bythis manipulation (e.g., Shull et al., 2001; Shull, 2004). The secondstudy turns to examine the finding that, on random ratio (RR)schedules, the value of the ratio does not impact the number of“burst-initiation” responses, but that, as the ratio increase in size,the number of “within-burst” responses increases (Bowers et al.,2008; Shull et al., 2001; Shull & Grimes, 2003).

4 REED

To this end, three groups of rats were studied, each groupresponding on a different RR schedule of reinforcement: RR-10,RR-30, and RR-60. If previous results were to be replicated, itwould be expected that overall response rates would increase asthe ratio value increased (at least for this range of RR values, seeReed & Hall, 1988). However, this would be reflected in anincrease in the number of “within-burst” responses, rather than inthe number of “burst-initiation” responses being made.

Method

Subjects and apparatus. Twenty-four male, experimentallynaıve Lister Hooded rats served in the present experiment. Thesubjects were approximately 3–4 months old at the start of train-ing, and had a free-feeding body-weight range of 380–445g. Thesubjects were housed and maintained as described in Experiment1. The apparatus was that described in Experiment 1.

Procedure

The subjects were magazine and lever-press trained as describedin Experiment 1. After lever-press training, both of the levers wereinserted into the chamber, and the rats were given four 30-minsessions of training on a concurrent RI 15s VI 15s schedule, asdescribed in Experiment 1. Following these four sessions, the ratsexperienced a further six 60-min sessions of a concurrent RI 15sRI 15s schedule, with both lights above the levers illuminated for45s, during which time the schedule was in operation, or both notilluminated for 75s, during which no reinforcement was available,as described in Experiment 1. Following this exposure, the ratswere then all exposed to the training contingency, also as describedin Experiment 1, over the course of 10 further training sessions.

Following this training regime, the rats were divided into threegroups of eight rats. One group of rats then responded on a RR 10schedule, one group responded on a RR 30 schedule, and the finalgroup responded on a RR 60 schedule (in all cases, the probabilityof each response being reinforced was set at 1/RR value). Thecriterion value for all groups for nonresponding extinguishingthe light above the right lever was 5s. All sessions lasted until thesubjects had earned 30 reinforcers. The subjects were exposed tothese contingencies for 90 sessions (as in Experiment 1, around70% of all responses emitted were to illuminated levers, and thesewere the only responses analyzed for this study).

Results and Discussion

The data from the last six sessions of training, which were takento reflect steady-state performance, were analyzed. The mean(standard deviation) obtained numbers of reinforcers per minutefor the three groups were: RR-10, 6.5 (2.3); RR-30, 3.4 (1.3);RR-60, 2.1 (0.8). An ANOVA conducted on these data revealed astatistically significant effect of group, F(2, 21) � 15.86, p � .001,and Tukey’s Honestly Significant Difference (HSD) tests revealedthat the pairwise comparisons between the RR-10 and RR-30groups, and between the RR-10 and RR-60 groups, were statisti-cally significant, ps � .01.

The mean (standard deviation) overall responses per minute (allresponses to illuminated levers for the total session) for the threegroups were: RR-10, 69.9 (27.6); RR-30, 101.7 (38.7); RR-60,

125.4 (43.5). An ANOVA that revealed a statistically significanteffect of group, F(2, 21) � 4.52, p � .05. Tukey’s HSD testsrevealed that only the difference between the RR-10 and RR-60groups was statistically significant, p � .05.

The mean (standard deviation) number of “burst-initiation” re-sponses made by the three groups per minute were: RR-10, 12.0(4.9); RR-30, 6.1 (3.9); RR-60, 1.9 (0.8). An ANOVA revealed astatistically significant effect of group, F(2, 21) � 15.87, p � .001.Tukey’s HSD tests revealed that the differences between theRR-10 and RR-30 groups, and RR-10 and RR-60 groups, werestatistically significant, both ps � .05. Of course, these responserates include periods of time in which it would be impossible tomake a burst-initiating response. Recalculating the rate of “burst-initiation” responses after the time spent responding in the bursts(including the 5s periods without a response) revealed a similarnumerical pattern to that above. These recalculated mean (standarddeviation) responses per minute were: RR-10, 26.5 (18.4); RR-30,11.9 (13.8); RR-60, 2.1 (0.9). An ANOVA revealed a statisticallysignificant effect of group, F(2, 21) � 6.80, p � .01. Tukey’s HSDtests revealed that the difference between the RR-10 and RR-60groups was statistically significant, p � .05.

The above data do not conform precisely to the findings fromthe log survivor plots of IRT distributions from single manipulan-dum situations, which suggest that the number of “burst-initiating”responses on RR schedules is not greatly affected by the ratiovalue (see Bowers et al., 2008; Shull et al., 2001; Shull & Grimes,2003). However, it should be noted that these data would appear tocorroborate what was apparent from the analysis of the RI perfor-mance in Experiment 1; that is, the rate of “burst-initiation”responses varies with the rate of reinforcement (see also Shull etal., 2001; Shull, 2004).

The group mean (standard deviation) number of “responses perburst” were: RR-10, 5.0 (0.9); RR-30, 18.2 (5.6); RR-60, 68.6(10.3). An ANOVA revealed a statistically significant effect ofgroup, F(2, 21) � 196.02, p � .001, and Tukey’s HSD testsrevealed that all pairwise differences between the groups werestatistically significant, all ps � .01. While it should be noted thatnot all “burst-initiation” responses were followed by responses tothe “within-burst” lever; approximately a quarter of the “burst-initiation” responses for each group were not followed by re-sponses to the “within-burst” lever (RR-10, 24%; RR-30, 18%;RR-60, 27%), correcting for this did not alter the pattern of results.Calculating the numbers of responses per burst, in those bursts thatproduced a response, gives the following group mean (standarddeviation) results: RR-10, 6.5 (1.1); RR-30, 22.2 (6.7); RR-60,93.8 (6.5). An ANOVA revealed a statistically significant effect ofgroup, F(2, 21) � 582.23, p � .001, and Tukey’s HSD testsrevealed that all pairwise differences between the groups werestatistically significant, all ps � .01. It might be noted that in bothof these analyses, the RR-60 group appeared to have to emitgreater numbers of responses than might be expected to obtainreinforcement according to that schedule, this can only be due torandom variations in the programming requirements. However,what is apparent from both of these analyses, is that as the ratiovalue increased, greater numbers of responses were emitted‘within-burst,’ as has been suggested from IRT analyses (Bowerset al., 2008; Shull et al., 2001).

The group mean (standard deviation) rate of within-burst re-sponding for the three groups (from the first response to the

5RESPONSE RATE COMPONENTS

“burst” lever to the last, and excluding the 5s time of not respond-ing at the end of the bursts) reflected the above values, and was:RR-10, 63.0 (5.5); RR-30, 121.1 (7.6); RR-60, 263.4 (25.8). AnANOVA revealed a statistically significant effect of group, F(2,21) � 339.24, p � .001, and Tukey’s HSD tests revealed that allpairwise differences between the groups were statistically signif-icant. These “within-burst” rates of responding conform, generally,to the rates of responding that might be expected in typical single-manipulandum schedules for these ratio values (see Reed & Hall,1988), with one exception; namely, that the rates of responseobserved are numerically greater than in such procedures, a trendthat was especially noticeable in the RR-60 group (i.e., fourresponses per second), and, in fact, this within-burst’ rate suggestssome unitization of responses had occurred (see Reed, Schacht-man, & Hall, 1991).

In summary, the present experiment replicated several featuresof the log survivor plots produced previously, and the results areconsistent with the analysis of the effects on ‘burst-initiating’responses, and “within-burst” responses produced by those reports.

Experiment 3

The final study aimed to directly compare the impact of RI andRR schedules on “burst-initiation” and “within-burst” respondingwhen the rate of reinforcement between the schedules is equated.Thus far, only one study has attempted to make this comparison(Bowers et al., 2008) using an IRT analysis. The results from thislatter study were made somewhat difficult to interpret by the lackof clear points of inflection in the log survivor plots of the IRTs,but in general, it was noted that RR schedules produced greaternumbers of “within-burst” responses, and RI schedules tended toproduce greater numbers of “burst-initiation” responses.

Should such a finding be replicated using the current procedure,it would suggest a number of issues that need to be consideredwhen interpreting standard RR versus RI response rate differences(e.g., Ferster & Skinner, 1957; Peele et al., 1984). Typically, thehigher response rate seen on a RR schedule has been the startingpoint for an analysis of why the RR schedule maintains strongerresponding than a RI schedule (e.g., Baum, 1973; Morse, 1966;Peele et al., 1984). Certainly, if there are greater numbers of“within burst” responses on a RR schedule, compared to a RIschedule, then these analyses would be supported for these “withinburst” responses. However, if there is a greater tendency to initiateresponding on a RI schedule than on a RR schedule, it may suggestthat strength of responding maintained by the schedules, may notbe so easily established by simple analysis of overall responserates. Some evidence that seems to corroborate this suggestioncomes from a cross-experimental comparison of the RR-60 groupfrom Experiment 2, and the RI 30s group from Experiment 1.These two groups had similar rates of reinforcement to one an-other, which would allow some control of this variable in thiscomparison. The comparison of these groups reveals that the RRgroup had a higher rate of “within-burst” responding (as predictedby previous studies, see Ferster & Skinner, 1957; Peele et al.,1984), but the RI group had a higher rate of “burst-initiation.”However, such cross-experimental comparisons are not ideal, anda direct comparison needs to be made. To these ends, two groupsof rats were studied, one group responding on a RR schedule of

reinforcement, and the other on a yoked RI schedule of reinforce-ment.

If previous results, and the above cross-experimental compari-son, were to be replicated, it would be expected that overallresponse rates would be higher on the RR schedule (e.g., Peele etal., 1984), there would be greater numbers of “within burst”responses on the RR schedule (Bowers et al., 2008), but greaternumbers of “burst initiating” responses on the RI schedule (cf.Bowers et al., 2008; Nevin et al., 2001; Shull et al., 2001).

Method

Subjects and apparatus. Sixteen male, experimentally naıveLister Hooded rats served in the present experiment. The subjectswere approximately 4–5 months old at the start of training and hada free-feeding body-weight range of 405–480g. The subjects werehoused and maintained as described in Experiment 1. The appa-ratus was that described in Experiment 1.

Procedure. The subjects were magazine and lever-presstrained as described in Experiment 1. After lever-press training,both of the levers were inserted into the chamber, and the rats weregiven four 30-min sessions of training on a concurrent RI 15s RI15s schedule, as described in Experiment 1. Following these foursessions, the rats experienced a further six 60-min sessions of aconcurrent RI 15s RI 15s schedule, with both lights above thelevers illuminated for 45s, during which time the schedule was inoperation, or both not illuminated for 75s, during which no rein-forcement was available, as described in Experiment 1. The ratswere then all exposed to the contingency as described in Experi-ment 1 for 10 further training sessions.

Following this training regime, the rats were divided into twogroups of eight rats. One group of rats then responded on a RR 30schedule, and the other group responded on a yoked RI schedule.The RR and RI schedules were programmed as described above inExperiments 1 and 2. The rats were placed in master-yoked animalpairs, which remained constant throughout the study. The timesbetween the successive reinforcers obtained by the master RR rat,became the successive criterion interval times for the yoked RI rat.The criterion value for all groups for nonresponding extinguishingthe light above the right lever was 5s. All sessions lasted until thesubjects had earned 30 reinforcers. The subjects were exposed tothese contingencies for 90 sessions.

Results and Discussion

The data from the last six sessions of training, which were takento reflect steady-state performance, were analyzed. The mean(standard deviation) obtained numbers of reinforcers per minutefor the groups was: RR-30, 3.7 (0.7); RI-y, 3.4 (0.6). A t testrevealed no statistically significant effect of group, t � 1, suggest-ing that the yoking procedure had worked as expected.

The mean (standard deviation) overall responses per minute (allresponses to both levers for the total session) for the groups were:RR-30, 118.2 (23.5); RI-y, 65.5 (21.3). A t test revealed a statis-tically significant effect of group, t(14) � 4.74, p � .001. Thiseffect replicates the standard RR versus RI response rate differencefound in multiple studies, previously (e.g., Ferster & Skinner,1957; Peele et al., 1984; Zuriff, 1970).

However, the mean (standard deviation) number of “burst-initiation” responses made by the groups per minute showed the

6 REED

reverse pattern of results, with the group means being: RR-30, 5.8(0.9); RI-y, 11.3 (1.6), t(14) � 8.77, p � .001. Of course, theseresponse rates include periods of time in which it would beimpossible to make a burst-initiating response. Recalculating therate of “burst-initiation” responses after the time spent respondingin the bursts (including the 5-s periods without a response) re-vealed a similar numerical pattern to that above, but with muchinflated rates of response and variability. These recalculated mean(standard deviation) responses per minute were: RR-30, 76.9(81.7); RI-y, 230.9 (194.4), t(14) � 2.07, p � .05. These datacorroborate cross-experimental comparisons from the study of IRTlog survivor plots (cf. Bowers et al., 2008; Shull et al., 2001), andalso suggestions from previous reports that, when the rate ofresponding is equated across RR and RI schedules, that subjectswill differentially respond to access the RI rather than the RRschedule (cf. Nevin et al., 2001).

The group mean (standard deviation) number of “responses perburst” were: RR-30, 19.2 (3.6); RI-y, 4.8 (1.9). A t test revealed astatistically significant effect of group, t(14) � 10.11, p � .001. Asnot all “burst-initiation” responses were followed by responses tothe “within-burst” lever (RR-30, 21%; RI-y, 28%), correcting forthis did not alter the pattern of results. Calculating the numbers ofresponses per burst, in those bursts that produced a response, givesthe following group mean (standard deviation) results: RR-30,24.5 (5.3); RI-y, 6.6 (2.3), t(14) � 8.70, p � .001. In both of theseanalyses, the RR-30 group emitted greater numbers of “within-burst” responses than a yoked RI group. The group mean (standarddeviation) rate of within-burst responding for the three groups(from the first response to the “burst” lever to the last, andexcluding the 5-s time of not responding at the end of the bursts)reflected the above values, and was: RR-30, 151.6 (40.7); RI-y,100.2 (45.1), t(14) � 2.39, p � .05.

In summary, the present experiment confirmed what was pre-dicted on the basis of the previous literature from the log survivorplots: there was a standard RR versus RI difference in overallresponse rate (see Ferster & Skinner, 1957), which appeared to bedue to a greater number of responses being emitted once respond-ing was initiated (see Bowers et al., 2008; Shull et al., 2001).However, there was a greater tendency to initiate responding onthe RI relative to the RR schedule, despite an equation of theoverall rates of reinforcement. These findings were more pro-nounced than those obtained from the cross-experimental compar-ison of Experiment 1 and 2, noted above, but this may reflectsomewhat different rates of reinforcement obtained, or even adifference between the effects of single and multiple schedules.Overall, these findings suggests that previous indications of greaterresponse strength for RI than RR schedules: for example, subjectswill chose RI schedules in preference to RR schedules, at leastunder some conditions (e.g., Williams, 1985), and show greaterresistance to change following RI than RR schedules (Nevin et al.,2001), may well be indexing the same aspect of behavior as thecurrent “burst initiation” measure.

General Discussion

The current series of studies were designed to explore a novelapproach to analyzing the different components of response ratethat are produced by exposure to free-operant schedules of rein-forcement. It has been suggested that overall response rate com-

prises a tendency to initiate responding and to continue to respondonce the bout is initiated. Previous post hoc analyses of IRT datahave suggested several features of these different aspects of re-sponding that the current experimental procedure broadly con-firmed. That is, increasing the size of a RI schedule decreases thenumber of “burst-initiation” responses, but has less effect onresponding once the burst has been initiated (Experiment 1); thatthe major difference between a RR schedule and a RI schedule, isnot in the number of “burst-initiation” responses, but in the numberof shorter “within-burst” responses that are emitted, with greaternumbers of “within-burst” responses being emitted on a RR sched-ule (Experiments 2 and 3).

These results have lead to the claim that “burst-initiation”responses are controlled by motivational factors (e.g., rate ofreinforcement), and the “within-burst” responses are controlledby schedule factors (e.g., reinforcement of particular IRTs).This suggestion mirrors the distinction between the strengthen-ing and shaping properties of reinforcement made by Morse(1966; see Shull et al., 2001). The modulation of “within burst”responding seen in the current studies (notably Experiments 2and 3), supports this type of analysis for this component ofresponding. Alternatively, Nevin (1979; Grace & Nevin, 1997)have suggested that resistance to change is a stronger index ofstrength of learning than response rate. When directly comparedon this measure, RI schedules demonstrate a greater resistanceto change than RR schedules, suggesting stronger learning onthe RI, than the RR, schedule (Nevin et al., 2001). That subjectsin Experiment 3 were more likely to initiate responding on a RIschedule than a RR schedule is in line with this suggestion, andalso in line with some results from choice studies (see Williams,1985; but note that this may only occur under some parameters,and further exploration of this effect would be wise before firmconclusions are drawn).

Although the primary focus of the current series of studies wasempirical, and an attempt to develop an experimental procedurethat could corroborate, or otherwise, the conclusions based on amore analytic approach to schedule behavior, the results do havesome implications for theories of schedule performance. In partic-ular, they suggest that the traditional approach of attempting toaccommodate all responses with a single molecular or molar viewmay be misguided. Rather, it may well be that, not only do thesetwo aspects of the environment control behavior, more or lessstrongly, under different conditions (see Reed, 2007), but thatdifferent aspects of behavior may come under different aspects ofenvironmental control (burst-initiations being more sensitive tomolar contingencies, and within-burst responses to molecular pro-cesses). Such a set of functions will, of course, make analysis ofschedule performance more complex, but may offer opportunitiesto unify the various competing views.

It should be noted that, in the current series of experiments, anarbitrary feature of the design was the selection of a 5-s pause fromresponding as the criterion for assuming that a response-burst hadbeen terminated. The use of this particular length of time wasindicated by an analysis of prior data on rats’ responding patternson schedules of reinforcement, which had suggested that this valuewas appropriate (e.g., Bowers et al., 2008; Shull & Grimes, 2003).Nevertheless, it is wise to speculate, briefly, about the implicationsof choosing a value shorter, or longer, than the current 5s, as it ispossible that different results would have been generated with the

7RESPONSE RATE COMPONENTS

use of a different criterion cut off point. The selection of a longercut-off criterion would have tended to produce fewer responsebursts containing greater numbers of responses, and this couldhave, potentially, resulted in fewer burst-initiation responses.Presumably, the greater the rats’ tendency to respond within-burst, the more pronounced this predicted pattern of data wouldhave been (i.e., the less likely it would be that the longer cut-offcriterion would be reached). In turn, this tendency might haveinteracted with the above general truism concerning longer crite-rion cut-off times producing fewer bursts. Hence, given the resultsfrom Experiments 1 and 2, longer ratios and shorter intervals mayhave lead to fewer bursts, and fewer burst-initiations, which wouldhave made the pattern of differences between the schedules thatwere noted in the current studies even more pronounced. Theopposite is true, of course, if a shorter cut-off criterion had beenused. In the absence of further data (or the ability to analyze thecurrent data more precisely), this issue of whether the overallpattern of data remained the same with different cuts is one forfuture empirical investigation.

In addition to establishing some of these effects, the currentreport also aimed to document an experimental procedure thatmight be useful in this area. In this way it was hoped to offer amore experimental, than analytic, approach to this area, based onthe notion of a revealed operant (Mechner, 1992; Pear & Rector,1979). Such a tool could be a useful additional approach tocompliment the post hoc analysis of IRT distributions discussedabove. A number of problems with the latter approach wereoutlined in the General Introduction, including the method ofidentifying “within-burst” and “between-burst” responses that itdepends upon post hoc inspection of the IRTs produced by theorganism. That the present experiments replicated almost all fea-tures of the log survivor plots produced previously, and the resultsare consistent with the analysis of the effects on ‘”burst-initiating”responses, and “within-burst” responses produced by those reports,suggests that the current experimental procedure for investigatingthese phenomena has criterion validity with other types of post hocanalysis, and the results from additional studies using this presentprocedure may well have validity in illuminating these processes,without incurring the problems of interpretation inherent in the pothoc analyses.

References

Bacha-Mendez, G., Reid, A. K., & Mendoza-Soylovna, A. (2007). Resur-gence of integrated behavioral units. Journal of the Experimental Anal-ysis of Behavior, 87, 5–24.

Baum, W. M. (1973). The correlation-based law of effect. Journal of theExperimental Analysis of Behavior, 20, 137–153.

Baum, W. M. (1993). Performances on ratio and interval schedules ofreinforcement: Data and theory. Journal of the Experimental Analysis ofBehavior, 59, 245–264.

Bennett, J. A., Hughes, C. E., & Pitts, R. C. (2007). Effects of metham-phetamine on response rate: A microstructural analysis. BehaviouralProcesses, 75, 199–205.

Blough, D. S. (1963). Interresponse time as a function of continuousvariables: A new method and some data. Journal of the ExperimentalAnalysis of Behavior, 6, 237–246.

Bowers, M. T., Hill, J., & Palya, W. L. (2008). Interresponse time struc-tures in variable-ratio and variable-interval schedules. Journal of theExperimental Analysis of Behavior, 90, 345–362.

Cole, M. R. (1999). Molar and molecular control in variable-interval andvariable-ratio schedules. Journal of the Experimental Analysis of behav-ior, 71, 319–328.

Dack, C., McHugh, L., & Reed, P. (2009). Generalization of causalefficacy judgments after evaluative learning. Learning and Behavior.

Dickinson, A., Peters, R. C., & Shechter, S. (1984). Overshadowing ofresponding on ratio and interval schedules by an independent predictorof reinforcement. Behavioural Processes, 9, 421–429.

Ferster, C. B., & Skinner, B. F. (1953). Schedules of reinforcement.Appleton-Century-Crofts: New York.

Grace, R. C., & Nevin, J. A. (1997). On the relation between preferenceand resistance to change. Journal of the Experimental Analysis of Be-havior, 67, 43–65.

Herrnstein, R. J. (1970). On the law of effect. Journal of the ExperimentalAnalysis of Behavior, 13, 243–266.

Kessel, R., & Lucke, R. (2008). An analytic form for the response rateanalysis of Shull, Gaynor, and Grimes with applications and extensions.Journal of the Experimental Analysis of Behavior, 90, 363–386.

Kulubekova, S., & McDowell, J. J. (2008). A computational model ofselection by consequences: Log survivor plots. Behavioural Processes,78, 291–296.

Lattal, K. A., & Neef, N. A. (1996). Recent reinforcement-schedule re-search and applied behavior analysis. Journal of Applied BehaviorAnalysis, 29, 213–230.

McDowell, J. J., & Wixted, J. T. (1986). Variable-ratio schedules asvariable-interval schedules with linear feedback loops. Journal of theExperimental Analysis of Behavior, 46, 315–329.

Mechner, F. (1992). The revealed operant: A way to study the character-istics of individual occurrences of operant responses. Cambridge, MA:Cambridge Center for Behavioral Studies.

Mellgren, R. L., & Elsmore, T. F. (1991). Extinction of operant behavior:An analysis based of foraging considerations. Animal Learning & Be-havior, 19, 317–325.

Morse, W. H. (1966). Intermittent reinforcement. In W. K. Honig (Ed.),Operant behavior: Areas of research and application (pp. 52–109). NewYork: Appleton Century-Crofts.

Nevin, J. A. (1979). Reinforcement schedules and response strength. InM. D. Zeiler & P. Harzem (Eds.), Reinforcement and the organization ofbehavior. Chichester, England: Wiley.

Nevin, J. A., Grace, R. C., Holland, S., & McLean, A. P. (2001). Variable-ratio versus variable-interval schedules: Response rate, resistance tochange, and preference. Journal of the Experimental Analysis of Behav-ior, 76, 43–74.

Pear, J. J., & Rector, B. L. (1979). Constituents of response rate. Journalof the Experimental Analysis of Behavior, 32, 341–362.

Peele, D. B., Casey, J., & Silberburg, A. (1984). primacy ofinterresponse-time reinforcement in accounting for rate differencesunder variable-ratio and variable-interval schedules. Journal of Ex-perimental Psychology: Animal Behaviour Processes, 10, 149 –167.

Raia, C. P., Shillingford, S. W., Miller, H. L., Jr., & Baier, P. S. (2000).Interaction of procedural factors in human performance on yokedschedules. Journal of the Experimental Analysis of Behavior, 74,265–281.

Reed, P. (2007). The role of response rate in determining sensitivity tothe molar feedback function relating response rate and reinforcementrate on VI� schedules of reinforcement. Journal of ExperimentalPsychology: Animal Behavior Processes, 33, 428 – 439.

Reed, P., & Hall, G. (1988). The schedule dependency of the signalledreinforcement effect. Learning and Motivation, 19, 387–407.

Reed, P., & Morgan, T. A. (2006). Resurgence of response-sequenceduring extinction in rats shows a primacy effects. Journal of the Exper-imental Analysis of Behavior, 86, 307–315.

Reed, P., Schachtman, T. R., & Hall, G. (1988). Overshadowing and

8 REED

potentiation of instrumental responding in rats as a function of theschedule of reinforcement. Learning and Motivation, 19, 13–30.

Reed, P., Schachtman, T. R., & Hall, G. (1991). The effect of signalledreinforcement on the formation of behavioral units. Journal of Experi-mental Psychology: Animal Behavior Processes, 17, 475–485.

Shull, R. L. (2004). Bouts of responding on variable interval schedules:Effects of deprivation level. Journal of the Experimental Analysis ofBehavior, 81, 155–167.

Shull, R. L., Gaynor, S. T., & Grimes, J. A. (2001). Response rateviewed as engagement bouts: Effects of relative reinforcement andschedule type. Journal of the Experimental Analysis of Behavior, 75,247–274.

Shull, R. L., Gaynor, S. T., & Grimes, J. A. (2002). Response rate viewedas engagement bouts: Resistance to extinction. Journal of the Experi-mental Analysis of Behavior, 77, 211–231.

Shull, R. L., & Grimes, J. A. (2003). Bouts of responding from variable-

interval reinforcement of lever pressing by rats. Journal of the Experi-mental Analysis of Behavior, 80, 159–171.

Shull, R. L., Grimes, J. A., & Bennett, J. A. (2004). Bouts of responding:The relation between bout rate and the rate of variable-interval rein-forcement. Journal of the Experimental Analysis of Behavior, 81, 65–83.

Williams, B. A. (1985). Choice behavior in discrete-trial concurrent VI-VR:A test of maximizing theories of matching. Learning & Motivation, 16,423–443.

Zuriff, G. E. (1970). A comparison of variable-ratio and variable-intervalschedules of reinforcement. Journal of the Experimental Analysis ofBehavior, 13, 369–374.

Received April 11, 2009Revision received February 17, 2010

Accepted February 19, 2010 �

Online First Publication

APA-published journal articles are now available Online First in the PsycARTICLES database.Electronic versions of journal articles will be accessible prior to the print publication, expeditingaccess to the latest peer-reviewed research.

All PsycARTICLES institutional customers, individual APA PsycNET� database package sub-scribers, and individual journal subscribers may now search these records as an added benefit.Online First Publication (OFP) records can be released within as little as 30 days of acceptance andtransfer into production, and are marked to indicate the posting status, allowing researchers toquickly and easily discover the latest literature. OFP articles will be the version of record; thearticles have gone through the full production cycle except for assignment to an issue andpagination. After a journal issue’s print publication, OFP records will be replaced with the finalpublished article to reflect the final status and bibliographic information.

9RESPONSE RATE COMPONENTS