ebook img

ERIC EJ872845: Delayed Reinforcement of Operant Behavior PDF

11 Pages·0.15 MB·English
by  ERIC
Save to my drive
Quick download
Download
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.

Preview ERIC EJ872845: Delayed Reinforcement of Operant Behavior

JOURNALOFTHEEXPERIMENTALANALYSISOFBEHAVIOR 2010,93,129–139 NUMBER1(JANUARY) DELAYED REINFORCEMENT OF OPERANT BEHAVIOR KENNON A. LATTAL WESTVIRGINIAUNIVERSITY The experimental analysis of delay of reinforcement is considered from the perspective of three questionsthatseembasicnotonlytounderstandingdelayofreinforcement,but,also,byimplication, the contributions of temporal relations between events to operant behavior. The first question is whethereffectsofthetemporalrelationbetweenresponsesandreinforcerscanbeisolatedfromother featuresoftheenvironmentthatoftenaccompanydelays,suchasstimuliorchangesinthetemporal distributionorrateofreinforcement.Thesecondquestionisthatoftheeffectsofdelaysonoperant behavior. Beyond the common denominator of a temporal separation between reinforcers and the responsesthatproducethem,delayofreinforcementproceduresdifferfromoneanotheralongseveral dimensions, making delay effects circumstance dependent. The final question is one of interpreting delayofreinforcementeffects.Itcentersontheroleoftheresponse–reinforcertemporalrelationinthe contextofother,concurrentlyoperatingbehavioralprocesses. Keywords: delay,reinforcement,gradient,temporalcontiguity,chainedschedule,tandemschedule, primaryprinciple,derivedprinciple _______________________________________________________________________________ Along with rate, quality, and magnitude, neither of these reviews was on free-operant delay has been considered a primary determi- behavior. nant of the effectiveness of a reinforcer (e.g., The experimental analysis of delay of re- Catania, 1979; Kimble, 1961). The study of inforcementhasresultednotonlyinanexten- delay of reinforcement in the experimental sive empirical literature, but the results of analysis of behavior is a contemporary mani- these analyses have been incorporated into a festation of the long-standing question in the number of integrative theories of reinforce- history of ideas, from Aristotle to Hume and ment, such as delay-reduction theory (e.g., on to James, of how the temporal relations Fantino,1969), the correlation-based law of betweeneventsinfluencetheactionsoforgan- effect (Baum, 1973; cf. Williams, 1983), and isms. Early in the history of experimental behavioral economics (e.g., Mazur, 1987). psychology, Thorndike (1911) noted that Practically, delay of reinforcement is part and response acquisition is negatively related to parcel of many human endeavors, as well as the interval between a response and its effect the applications of behavioral research (e.g., or consequence, and Watson (1917) studied Hayes & Hayes, 1993; Stromer, McComas, & delay of reinforcement experimentally by Rehfeldt, 2000). Perhaps because of its long imposing a period of time between rats’ historyofstudy,ratherthaninspiteofit,delay responses of digging through sawdust and ofreinforcementcontinuesasafruitfulareaof subsequent access to food. Thereafter, a research, application, and theory. plethora of experiments have examined delay Delayofreinforcementisconsideredherein of reinforcement using a variety of methods from the perspective of three questions. The and from an equal variety of theoretical first is one of separating the time period perspectives. Earlier work was placed in per- between a reinforcer and the response that spective in previous reviews (Renner, 1964; produces from other features of the environ- Tarpy & Sawabini, 1974), though the focus of ment that can, and often do, accompany the introduction of a delay, such as stimuli or changes in the temporal distribution and rate of reinforcement. The second question is that Thanks to Alicia Roca, Rogelio Escobar, Carlos Can- c¸ado, Andre´s Garcia Penagos, David Jarmolowicz, Toshi- of the effects of delays on operant behavior. kazu Kuroda, and Allison Tetrault for thoughtful discus- Beyond the common denominator of tempo- sionsandreviewsofearlierversionsofthisarticle. ral separation, delay of reinforcement proce- Address correspondence to the author at Department dures differ from one another along several ofPsychology,WestVirginiaUniversity,Morgantown,WV dimensions, making delay effects circum- 26056-6040(e-mail:[email protected]). doi:10.1901/jeab.2010.93-129 stance-dependent. The final question is one 129 130 KENNON A. LATTAL of interpreting delay of reinforcement effects. successive reinforcers. Resetting delays typical- It revolves around the contribution of the lyarefixed,thoughinprincipletheyneednot response–reinforcer temporal relation in the be. A variable delay can be arranged such that context of other, concurrently operating be- the values are selected and then set for the havioral processes. Before considering these duration of each specific delay by using a three questions, however, answering a more variable-time (VT) schedule (e.g., Cicerone, basic question is in order. 1976). With a nonresetting unsignaled delay, theactualdelayprecedingreinforcersalsowill vary as a function of when responding occurs WHAT IS DELAY OF REINFORCEMENT? during the delay. In the latter case, the Reinforcement is delayed whenever there is nominal delay value defines the upper limit period of time between the response produc- of the delay, with actual or obtained delays ing the reinforcer and its subsequent delivery. typicallybeinglessthantheprogrammeddelay This time period has been arranged in value(cf.Dews,1981;Sizemore&Lattal,1977, different ways. Skinner (1938) programmed 1978). the delay in the presence of the same stimuli thatwereineffectduringthenondelayperiod, ISOLATING DELAYS FROM OTHER as did Dews (1960) and Azzi, Fix, Keller, & CONCURRENTLY OPERATING VARIABLES RochaeSilva(1965).Ferster(1953),andmany subsequent researchers (e.g., Chung, 1965; Thefirstquestionposedintheintroduction Chung&Herrnstein,1967;Pierce,Hanford,& is, in essence, whether there is a delay of Zimmerman, 1972; Richards, 1981) correlated reinforcementeffect.Imposingeithersignaled the delay interval with a stimulus change. or unsignaled delays of reinforcement can These two procedures define, respectively, simultaneously alter other features of the unsignaled and signaled delays of reinforce- environment, which in turn can contribute to ment. With both, there are two components: the behavioral changes nominally attributable one before the reinforced response, and the to imposing the delay. This is most obvious in other between it and the reinforcer. Thus, in signaled delays, where there is an immediate, the conventional schedule nomenclature of response-produced stimulus change that can Ferster and Skinner (1957), an unsignaled have conditioned reinforcing, eliciting, or delayofreinforcementmaybecategorizedasa overshadowingeffects on operantresponding, type of tandem schedule and a signaled delay independently of the effects of the delay. ofreinforcementasatypeofchainedschedule Breaking the chains of conditioned rein- of reinforcement. forcement, or these other stimulus functions, Otherfeaturesofthedelayareimportantin is easily accomplished by eliminating the its discussion. When delays have been accom- signal. When unsignaled delay value is varied, panied by a distinct stimulus, the stimulus a delay of reinforcement gradient is observed most often is a blackout, but changes in both (Sizemore & Lattal, 1978) that qualitatively visual stimuli (e.g., key color lights, lever resembles the gradient obtained with signaled retraction)andauditorystimulialsohavebeen delays (Richards, 1981). The unsignaled delay investigated (Ferster, 1953; Pierce et al., 1972; gradient, based on obtained delay values, Lattal, 1987). Delays also can be nonresetting however, is characterized by lower response or resetting. In the former, once the delay rates and a steeper slope than the gradient interval is initiated by a response, further obtained with otherwise equivalent signaled responses have no effect. A resetting delay is delays (Richards; Sizemore, 1976). one in which each response during the delay Evenwithunsignaleddelays,otherpotential returns the delay to its initial value. Resetting problems remain. Consider a typical manipu- and nonresetting delays are most often com- lation for imposing an unsignaled delay of bined with unsignaled delays, because the reinforcementonrespondingmaintainedbya stimuli present during asignaleddelayusually variable-interval (VI) 60-s schedule of immedi- control the behavior during the delay (Lattal, ate reinforcement. When a delay of, for 1987). The delay duration also can be either example, 20 s is imposed, the schedule is fixed,thatis,thesameeachtimeitiseffected, converted to a tandem VI 60-s fixed-time FT or variable, such that it changes before 20-s schedule. One problem is that the DELAY OF REINFORCEMENT 131 structure of the schedule changes such that Differentapproachestoassessingtheroleof reinforcers that previously could occur within delay relative to changes in rate of reinforce- less than a second of one another now are ment were used by Shull, Spear, and Bryson alwaysseparatedby20additionalseconds.This (1981, Experiment 2; cf. also Moore, 1979) lengthens postreinforcement pausing, which and Weil (1984). Shull et al. arranged for in turn can be reflected as lower overall pigeons’ key pecks during the initial link of a response rates, the same effect that might be chained schedule to produce a terminal link expectedwitha20-sdelay.Anotherproblemis composed of a fixed-duration reinforcement thattherateofreinforcementisreducedfrom cycle, accompanied by a key color change. anaverageofoneper60 stooneper80 s,also During this cycle three food presentations potentially reducing response rate indepen- occurred. The first occurred at a fixed time dently of the delay effects. afterthechoiceresponsewasmade,asdidthe Different solutions to these two problems third,whichwasneartheendofthereinforce- have been employed. Chung (1965; see also ment cycle. Because the time of the second Lattal, 1984), arranged a concurrent VI 1-min reinforcer was varied across different condi- VI1-minscheduleinwhichrespondingonone tions of the experiment, it was possible to of two keys produced scheduled reinforcers examinetheeffectsofdifferentdelaysbetween after a delay signaled by a blackout. Pecks on second-reinforcer onset and the initial-link the second response key yielded reinforcers response while holding overall reinforcement immediately after they were programmed. At rateconstant.Respondingvariedasafunction the same time, according to a VI 1-min of these delays. schedule,pecksonthissamekeyalsoproduced Weil (1984) employed a related technique blackouts of identical duration to those of the to eliminate the potential confound between signaled delay on the other key. These black- reinforcement rate changes and delay dura- outswerenotfollowedbyreinforcement.Thus, tions. He used a schedule derived from the reinforcement rate during the immediate and work described by Schoenfeld and Cole delayed reinforcement conditions were the (1972). Under this arrangement, a repeating same. At delay values greater than 1 s, relative timecycle,T,iscomposedoftwotimeperiods. responserateswereloweronthekeycorrelated During tD the first response results in rein- with the delay of reinforcement. The proce- forcementattheendoftheperiod.DuringtD, dure, however, converts the schedule on the responses have no consequence. By holding T nondelay key to a multiple VI Extinction constantandvaryingtheplacement(beforeor schedule with effects on responding on both after tD) and duration of tD, Weil generated thatkeyaswellasonthekeyassociatedwiththe different obtained delays to reinforcement delayed reinforcement. Lattal (1984) found while holding reinforcement rate constant. that adding such a blackout to a VI schedule For 3 of 4 pigeons, response rates were a often increased response rates, instances of decreasingmonotonicfunctionofincreasesin behavioral contrast (Catania, 1961; Reynolds, the obtained (as opposed to programmed, 1961). Sizemore and Lattal (1977, 1978) used which was unrelated to response rate) delay as the immediate reinforcement baseline a value. This led Weil to conclude that ‘‘both tandem VT FI schedule in which the values of obtained delay and reinforcer frequency ap- theVTandFIscheduleswereequivalenttothe pear to contribute to response rate, but values of the subsequent unsignaled delay obtained delay does so to a much greater condition (i.e., a tandem VI FT schedule). degree’’ (p. 154). Thus, the distribution and rate of reinforce- Another potential confounding variable mentremainsunchanged from the immediate when interpreting delay of reinforcement reinforcement baseline condition. This proce- effects occurs when delays are unsignaled. dure works well for assessing the effects of a The response is free to occur during un- single delayvalue;however,whenconstructing signaled nonresetting delays, making the a delay gradient by comparing different delay obtained delays less than the nominal delays. values, reinforcement rate still varies as the Dews (1981, p. 216) suggested that results delays are varied. As a result, variations in from such a procedure were difficult to reinforcement rate seem an inevitable con- interpret because they involve variable delays foundofintroducingdelaysinthismanner. of reinforcement and the obtained delays are 132 KENNON A. LATTAL likely to be of brief duration. By implication, responding than does a similar schedule of this latter point would mean that they do not response-independent food delivery (e.g., Cat- generate a sufficient range of delay values to ania & Keller, 1981; Sizemore & Lattal, 1977; allow a functional relation to be established Williams,1976).Iftheprogrammeddelaysare between responding and delay value. Un- relatively long (30 s), then the differences signaled delays, however, do yield orderly between the two conditions are more equivo- delay gradients, based on either nominal or cal (Gleeson & Lattal, 1987). This latter averaged obtained delays (Sizemore & Lattal, finding, however, simply underlines the fact 1978). Nonresetting delays are indeed vari- that at some point delays become sufficiently able,ratherthanfixed.Theavailableevidence, long that their effects are indistinguishable which is not extensive, suggests that fixed and from those resulting from an absence of a variable delays have different behavioral ef- response–reinforcer dependency. fects (e.g., Cicerone, 1976; Logan, 1960). When delays are variable, mean delay values DELAY OF REINFORCEMENT EFFECTS also may affect responding differently as a function of the distribution of those delays Giventhataneffectofdelaysunconfounded (e.g.,asinthedifferenteffectsonVIrespond- byproceduralvariablescanbeestablished,the ing of an arithmetic or a constant probability second question is: What are the effects? As a distribution of interreinforcer intervals, cf. function of circumstances, delays of reinforce- Catania & Reynolds, 1968). ment can decrease or increase behavior, or Forassessingthebehavioraleffectsofdelays leave it unchanged relative to immediate of reinforcement, Dews (1981) favored reset- reinforcement. Furthermore, the same delay tingdelaysovereithersignaleddelaysinwhich value may have different effects as a function the opportunity to respond was removed (cf. of other parameters of both the delay and at Pierce et al., 1972), or nonresetting delays. least some of the maintaining conditions of Although the resetting delay keeps the ob- reinforcement(butcf.Shahan&Lattal,2005). tained delay value constant, it creates other A primary consideration concerning how problems. One is that reinforcement rates are responding will be affected by delays of determined by whether responses occur dur- reinforcement is the baseline on which the ing the delay. This becomes a problem delay is imposed. The most common proce- particularly with longer delays, because some dure in the experimental analysis of behavior responding during the unsignaled delay typi- is to impose a delay of reinforcement after cally occurs and each response reduces rein- obtaining steady-state responding on some forcement rate, which in turn can reduce schedule with immediate reinforcement. Un- response rates independently of the effects of der these conditions, delays typically reduce the delay. response rates, but, even here, reductions are The response that produces the reinforcer not invariably the outcome. A less common initiates the delay, at the end of which the procedure is to impose delays of reinforce- reinforcer is provided independently of fur- mentintheabsenceofapreviouslyestablished ther responding. In the case of unsignaled operant response. delays, this procedure raises the question of whether simply providing the established ResponseDifferentiationwithDelayedReinforcement reinforcers independently of responding in the Absence of Response Training would have similar effects on behavior—that Skinner (1938) was the first to differentiate is,doesthedependencymatterinformulating operant responses in the absence of immedi- delays of reinforcement? The effects of un- ate reinforcement. He observed that lever- signaleddelayedreinforcementthereforehave press responding of experimentally naive rats been compared to those of response-indepen- was established when reinforcement followed dent reinforcers occurring at the same rate an unsignaled delay of up to 8 s. Lattal and andwiththesametemporaldistributionasthe Gleeson (1990) systematically investigated delayed reinforcers. The effect depends criti- such acquisition. In their first experiment, cally on the duration of the delay. If the they magazine trained experimentally naive, programmed delays are relatively brief (e.g., food-deprived rats and pigeons. Then, with 3 s), delayed reinforcement maintains more some pigeons, tandem fixed-ratio (FR) 1 FT DELAY OF REINFORCEMENT 133 30-s and with other pigeons and rats tandem reinforced approximation, shaping of naive FR1 differential-reinforcement-of-other-behav- pigeons’ key peck responses occurred quickly, ior (DRO) 30-s schedules of reinforcement completed within a single session of 1 hr or were implemented. Under the former sched- less. When, however, a 10-s delay was initiated ule, the first response initiated an unsignaled by activating the shaping switch, responding delay of 30 s for pigeons that terminated with was not shaped in 1 pigeon in the course of food delivery, regardless of whether further morethanten1-hrsessions.Visualobservation respondingoccurredduringthedelayinterval. of the pigeon revealed that during the last The arrangement was similar in the second training sessions, it continuously oriented to schedule, except that every response during the chamber wall containing the response key the delay (30 s for pigeons; 10 s for rats) reset and the grain magazine. The lateral distance thedelaytimer,thereby imposinga10-or30-s of its movements appeared to be constrained period of no operant responses immediately bythe10-sdelayinthatwhenitsbeakwasnear before food delivery. Without response shap- the key, the shaping switch was activated. The ingoranyotherformofresponsetraining,the pigeon then would continue to engage in response (key pecking by pigeons; lever other behavior during the delay interval, pressing by rats) developed in most of the ensuringthattherewasconsiderablevariability animals in periods of time ranging from a few in the responses that occurred just prior to minutes to a few hours. Thereafter, the food delivery. The 10-s delay delineated the response was maintained. Subsequent re- outer limits of the pigeon’s physical distance search ruled out the control of responding from the key. A similar pattern occurred with by such variables as simple exposure to the the 1-s delay, only the distance from the key apparatus(Lattal&Gleeson),evocationofthe allowedbythisdelaybeforereinforcementwas response by food delivery per se, (Lattal & considerably less. Thus, the delay contingency Gleeson; Wilkenfield, Nickel, Blakely, & Pol- seemedtoconstrainbehaviorspatially,butthe ing, 1993), and auditory feedback associated delay was sufficiently long in the 10-s case that with the response (Critchfield & Lattal, key pecking failed to develop over a time 1993; Schlinger & Blakely, 1994). Similar period that was more than adequate to shape response differentiation with delayed rein- key pecking with immediate or even briefly forcement also has been reported using delayed (1-s) reinforcement. different strains of rats (e.g., Anderson & Elcoro, 2007; Hand, Fox, & Reilly, 2006), Imposing Delays on Operant Responding Previously Siamesefightingfish(Elcoro,daSilva,&Lattal, Maintained by Immediate Reinforcement 2008; Lattal & Metzger, 1994), and humans Response differentiation with delayed rein- (Okouchi, 2009). forcement increases operant response rates relative to the baseline condition, which is Response Shaping with Delayed Reinforcement either zero or near zero. The more common The shaping of responding by the differen- effect associated with delays of reinforcement tial reinforcement of successive approxima- is response rate reduction; however, under tionshasbeensuggestedtobemostlikelywith some conditions responding has been ob- immediate differential reinforcement of such served to be unchanged from immediate approximations to the target or criterion reinforcement. Under others, response rates response(Skinner,1953).Giventhepreceding have increased when delays are imposed. findingsofresponseacquisitionintheabsence Delays of reinforcement most often are ofexplicitattemptstotraintheresponse,itwas introduced at full value following an immedi- ofinteresttoattempttoshaperespondingwith ate reinforcement baseline. Leaving aside for delayed reinforcement. The procedure was the moment the complexity of interpretation identicaltoaconventionalshapingprocedure, introduced by signaled delays of reinforce- exceptthattheshapingswitchwasarrangedso ment,Ferster(1953)suggestedthattheeffects that when an appropriate approximation to of signaled delays might be attenuated if the thecriterionresponseoccurred,activatingthe delay is introduced gradually and titrated as a shaping switch initiated a nonresetting un- function of the organism’s behavior, a tech- signaleddelay.Whenreinforcementwaseither nique used with considerable success subse- immediate or delayed by 1 s from the to-be- quently by Terrace (1963) to introduce nega- 134 KENNON A. LATTAL tive discriminative stimuli during discrimina- response rates increase as they do with brief tion training. Using this titration technique, unsignaled delays (Lattal & Ziegler; Richards, Ferster maintained responding of pigeons 1981). underVI60-sscheduleswithblackout-signaled By far, the most typically reported effect of delays of up to 120 s. His data, however, were delay of reinforcement imposed following limited to a sample of cumulative records steady-staterespondingonschedulesofimme- illustrating terminal performance and details diatereinforcementisareductioninresponse of the titrating procedure were not reported. rates relative to those observed when rein- Using a related procedure with baboons as forcement is immediate. In some cases, how- subjects,FersterandHammer(1965)reported ever, conclusions about the effects often have developing sustained responding by baboons been compromised because, particularly in with a 24-hr signaled delay between respond- some of the early experiments in which ing on one of two keys and delivery of different parameters of both the delay and reinforcement following a response on a the maintaining conditions of reinforcement second key when ‘‘the delay in reinforcement were manipulated, they failed to employ was increased slowly, paced with the monkeys’ appropriate control procedures as discussed performance’’ (p. 249). Although responding intheprevioussection.Wheretheappropriate was sustained, it ‘‘differed significantly from controlshavebeenincorporated,theeffectsof [that] usually recorded with immediate rein- the delay depend on parameters of both the forcement’’ (p. 252) and, in a follow-up delay and the reinforcement schedule. The experiment, introducing an 18-hr delay sud- former parameters include delay duration, denly as opposed to gradually showed that whether the delay is fixed or variable (Cicero- ‘‘performance maintained with long delays to ne, 1976), type of signal (Lattal, 1987), and reinforcement does not depend on a gradual, whether the delay is signaled, unsignaled, or paced increase in the length of the delay’’ (p. partially signaled (e.g., Richards, 1981; Schaal 253). &Branch,1988).Thelatterincludethetypeof Imposing brief (around 0.5 s) unsignaled schedule (e.g., Gonzales & Newman, 1976; nonresetting delays between a response and Kendall & Newby, 1978; Morgan, 1972; Ri- the reinforcer often increases, and character- chards, 1981) and parameters of reinforce- istically changes the structure of, responding ment (e.g., Mazur, 1987). An exception is the relative to that observed with immediate finding of Shahan and Lattal (2005) that reinforcement. This has been observed when unsignaled 3-s delays reduced response rates key-peck responses of pigeons were initially maintainedbydifferentratesofreinforcement maintained by immediate reinforcement on proportionally the same. either VI or differential-reinforcement-of-low- rate (DRL) schedules (Arbuckle & Lattal, BEHAVIORAL PROCESSES IN DELAY 1988; Hall, Channell, & Schachtman, 1987; OF REINFORCEMENT Lattal & Ziegler, 1982; Richards, 1981; Size- more&Lattal,1978).Lattalandhiscolleagues Historically, theoretical accounts of delay of have shown that, regardless of whether re- reinforcement have distinguished between a sponse rates increase, there is a consistent primaryandasecondaryorderivedgradientof increase in the number of short (, 0.5 s) reinforcement delay (see Renner, 1964, for a interresponse times. Lattal and Ziegler (1982) review). Hull (1943), for example, asserted suggested that, with brief delays to reinforce- that the derived delay of reinforcement ment, if the first peck of what is to be a series gradient was mediated by conditioned rein- of pecks initiates a delay, this allows the forcement and the primary one was not. remainingpecksinthebursttooccur,thereby Spence (1947) initially treated all delays of making the reinforcer contiguous with a burst reinforcement as involving conditioned rein- of responses. This same effect occurs if the forcement, including their mediation by pro- 0.5-s delays are resetting (Lattal & Ziegler). If, prioceptive stimuli, but later modified his however, that same response starts a delay position to distinguish two types of delay of signaled by a blackout, potential bursts are reinforcement, a chaining and nonchaining more likely to be truncated and, as a result, type,eachinvolvingdifferentbehavioralmech- neither IRT distributions change nor do anisms (Spence, 1956). DELAY OF REINFORCEMENT 135 These historical themes recur in the exper- as is suggested by the observations quoted in imental analysis of delay of reinforcement. the preceding paragraph, responding invari- Response–reinforcer temporal contiguity was ably occurs during the delay, accompanied by assigned a primary role in the effectiveness of astimuluschangeornot,raisingthepossibility reinforcement by Skinner (1948,1953; Ferster that such behavior contributes to the delay of & Skinner, 1957) and thereafter by others reinforcement effect. Baum (1973) more (e.g.,Peele,Casey,&Silberberg,1984;seealso broadly questioned the importance of re- Schneider, 1990). A number of subsequent sponse–reinforcer temporal contiguity in the experiments employed unsignaled delays of maintenance of operant behavior. He pro- reinforcement to examine more systematically posed that delay of reinforcement has its the functional relations between rate of effectsbecausedisruptingresponse–reinforcer response and the disruption of response– temporal contiguity weakens the correlation reinforcer temporal contiguity, in the form of between responding and reinforcement (cf. delay of reinforcement, unencumbered by also Williams, 1976). exteroceptive stimuli accompanying the delay Regardless of the conceptualization of de- (Sizemore & Lattal, 1977, 1978; Williams, lays as disrupting correlations or actual conti- 1976). guity,introducingadelayofreinforcement,by Ferster (1953) appealed to mediating be- definition, relaxes or loosens the response– havior during the delay to account for his reinforcer relation. The response–reinforcer observations of sustained operant responding dependency constrains and focuses respond- under delays of reinforcement up to 120 s, ing (e.g., Staddon, 1979; Timberlake & Alli- where a blackout or other stimulus change son, 1974; Zeiler & Buchman, 1979) when accompanied each delay. Eliminating the reinforcement is immediate. Relaxing this stimulus accompanying the delay, and thus a constraint in a schedule of reinforcement source of immediate conditioned reinforce- allows other forms of behavior to intrude, in ment for operant responses, still led to amannersimilartotheintrusionofnonnative interpretationsintermsofmediatingbehavior species into an otherwise stable ecosystem. during the delay. Azzi et al. (1965) observed Like nonnative species, these new behavioral mediating behavior in the form of ‘‘some sort forms can either compete with the operant of dipper contact, with a regularity similar to behavior or they can complement or even that which typifies ratio responding’’ (p. 161) augment it. during their unsignaled delay periods. A How relaxing but not eliminating the comment by Dews (1960), also studying response–reinforcer dependency affects re- unsignaled resetting and nonresetting delays sponse variability is illustrated by the pigeon’s of reinforcement, underlines the combined behaviorduringtheattempttoshapebehavior influence of contiguity and mediating behav- with delayed reinforcement described in pre- ior: ‘‘[p]resumably, some behavior occurring cedingsection.Asystematicanalysisofsuchan during [the] delay will be fortuitously rein- effect was reported by Escobar and Bruner forced’’ (p. 229). (2007), who studied response differentiation A primary delay of reinforcement gradient by experimentally na¨ıve rats in a chamber basedsimplyonthetemporalrelationbetween containinganarrayofsevenlevers.Thecenter the reinforcer and the last operant response lever was associated with a tandem random- that produced it, that is, a ‘‘pure’’ effect of time FT x-s schedule, where x varied from 1– delay, isolated from behavior, seems un- 32 s for different groups. Responses on the achievable, if not implausible. Certainly, with other six levers were recorded, but had no delays there is a structural or procedural othereffect.Atalldelayvalues,respondingwas temporalseparationbetweenoperantrespons- highest on the center lever. Responding on esandreinforcersthatfollow,butthisdoesnot the other levers was more confined to the necessarily correspond to a functional separa- levers adjacent to the center lever at 0-s delay. tion between responding and reinforcement. Although the effect was variable, with 1–8 s Delays of reinforcement do not dam the delays responding on levers physically more behavior stream. They simply rechannel it. distant from the operative lever tended to Thus, the delay is not a period of behavioral increase.With16-sand32-sdelays,responding emptiness through which time passes. Rather, to all of the levers was low to zero. Both of 136 KENNON A. LATTAL these experiments suggest that longer delays reinforcement reorganized temporal patterns allow greater behavioral variation, that is, the of responding and often even increased opportunity for different response forms to response rates relative to those occurring with intrude and thereby interact with the extant immediatereinforcement.Furthermore,there contingencies. is considerable evidence that events occurring Schaal, Shahan, Kovera, and Reilly (1998) in proximity to reinforcement in chained and provided an example of how the intrusion of tandem schedules, of which operant delays of other behavior can compete with the operant reinforcement are an example, do affect response during a delay of reinforcement responding earlier in those schedules (cf. procedure. In one experiment, pigeons’ key Lattal&Crawford-Godbey,1985;Starin,1987). pecking was maintained on a VI schedule. Lejeune, Richelle, and Wearden (2006) After the interfood interval expired, a peck to observedthatmediatingbehavioroftenoccurs asecond,food,keywasrequiredtoactivatethe in experimental analyses of temporally con- foodhopper.Indifferentconditions,thefood trolled responding, but they questioned keywas orwas not illuminated and afeedback whether it played a necessary role in such sound was or was not produced by responses circumstances. A similar conclusion seems on the food key during the interfood interval appropriate in the case of delay of reinforce- arranged by the VI schedule. Response rates ment in that not all experiments reporting on the VI key were lower when the food key delay of reinforcement effects report system- wasilluminated andwhenthefeedback sound atic patterns of mediating behavior. The was present during the interfood interval. experiments in the preceding paragraphs Schaal et al. suggested that their results are certainly show that relaxing the response– analogous to what happens during unsignaled reinforcer relation affects response variability, nonresetting delays to reinforcement: Rather which in turn affects response rate. Whether than other behavior occurring at the time of such an increase in response variability in all reinforcementbeingadventitiouslyreinforced, cases constitutes evidence of mediating behav- hopper-related observing behavior can be ior seems questionable. explicitly reinforced by hopper access. This Delaysofreinforcementarebothimposedby intermittent reinforcement of observing in contingencies and impose contingencies. Zei- competition with theoperant responses there- ler (1977) distinguished between direct, pro- by reduces operant response rates. These grammedvariablesinreinforcementschedules results also support Azzi et al.’s (1965) and indirect ones. The latter result from the informalobservationoffeeder-directedbehav- interactionofthedirectvariableswithbehavior. ior during and before the delay and related Delay of reinforcement effects are determined observations during signaled delays by Iversen inpartbysuchdirect variables astheschedule (1981). Nor are they unique to delay of of reinforcement and parameters of the delay reinforcement. Nevin (1971), for example, itself. Such effects, however, seem equally exposed pigeons to two-key concurrent fixed- determined by the dynamic generated when interval (FI) VI schedules. Negatively acceler- these direct variables interact with behavior. ated responding developed on the VI key as a Given the difficulties of isolating temporal function of the lapsed time on the concur- contiguity and the ambiguities of mediating rently available FI schedule. That is, the behavior, it seems useful to consider delayed concurrently reinforced FI responding com- reinforcementofoperantbehaviorinfunction- peted with the VI responding, changing the al terms, that is, in terms of the behavioral rate and pattern of the latter. Related results dynamics that emerge from the interaction of were reported by Lattal and Abreu-Rodrigues responding with the formal properties of both (1997) when FT schedules operated concom- the maintaining conditions of reinforcement itantly with VI schedules. andthecharacteristicsofthedelayitself. Examplesofintrudingbehavioraccompany- ing the introduction of delays of reinforce- CONCLUSION ment complementing or augmenting operant behavior have been reported in several exper- Thorndike’s law of effect states that ‘‘[o]f iments as well. As already noted, Lattal and several responses made to the same situation, Ziegler (1982) showed that brief delays of those which are accompanied or closely DELAY OF REINFORCEMENT 137 followed by satisfaction to the animal will, Catania, A. C. (1961). Behavioral contrast in a multiple other things being equal, be more firmly andconcurrentscheduleofreinforcement.Journalof theExperimentalAnalysisofBehavior,6,335–342. connected to the situation, so that, when it Catania, A. C. (1979). Learning. Englewood Cliffs, NJ: recurs, they will be more likely to recur (1911, PrenticeHall. p. 244).’’ Left for future generations was the Catania, A. C., & Keller, K. J. (1981). Contingency, task of exploring the implications of ‘‘closely contiguity,correlation,andtheconceptofcausation. In M. D. Zeiler, & P. Harzem (Eds.), Predictability, followed by’’ for the understanding of rein- correlation, and contiguity (pp. 125–167). Chichester, forcement. The research discussed in this England:Wiley. Perspective describes some of the empirical Catania, A. C., & Reynolds, G. S. (1968). A quantitative and interpretive fruits of Thorndike’s be- analysis of the responding maintained by interval schedulesofreinforcement.JournaloftheExperimental queathal derived from the experimental anal- AnalysisofBehavior,11,327–383. ysis of behavior. Chung,S-H.(1965).Effectsofdelayedreinforcementina Separated from structural confounds associ- concurrentsituation.JournaloftheExperimentalAnalysis ated with their imposition, delaysofreinforce- ofBehavior,8,439–444. ment have diverse behavioral effects, as a Chung,S-H.,&Herrnstein,R.J.(1967).Choiceanddelay ofreinforcement.JournaloftheExperimentalAnalysisof function of the circumstances of their con- Behavior,10,67–74. struction and imposition. All delays involve a Cicerone, R. A. (1976). Preference for mixed versus relaxing of the response–reinforcer temporal constant delay of reinforcement. Journal of the Exper- relation and hence, in a structural sense at imentalAnalysisofBehavior,25,257–261. Critchfield,T.S.,& Lattal,K.A.(1993).Acquisitionofa least, disruption of temporal contiguity be- spatiallydefinedoperantwithdelayedreinforcement. tweentheresponseandreinforcerthatfollows. Journal of the Experimental Analysis of Behavior, 59, Whether this structural change in temporal 373–384. contiguity translates into functional change is Dews, P. B. (1960). Free-operant behavior under condi- tionsofdelayedreinforcement:I.CRF-typeschedules. ambiguous.Both correlational and mediation- Journal of the Experimental Analysis of Behavior, 3, al accounts of delay of reinforcement ques- 221–234. tion, in different ways, the primacy of disrup- Dews,P.B.(1981).Effectsofdelayofreinforcementonthe tions in temporal contiguity in determining rateofsteady-stateresponding.InC.M.Bradshaw,E. delay of reinforcement effects. The ambiguity Szabadi,&C.F.Lowe(Eds.),Quantificationofsteady-state operantbehavior(pp.215–229).Amsterdam:Elsevier. about the functional nature of response– Elcoro, M., da Silva, S. P., & Lattal, K. A. (2008). Visual reinforcertemporalcontiguityincombination reinforcement in the female Siamese fighting fish with the varied effects that delays have as a Betta splendens. Journal of the Experimental Analysis of function of the kinds of variables described Behavior,90,53–60. Escobar, R., & Bruner, C. A. (2007). Response induction previously invites a broader view of delay of during the acquisition and maintenance of lever reinforcement. To wit, delay of reinforcement pressing with delayed reinforcement. Journal of the seems more usefully viewed as a dynamic ExperimentalAnalysisofBehavior,88,29–49. behavioral process resulting from the actions Fantino, E. (1969). Conditioned reinforcement, choice, and the psychological distance to reward. In D. P. of direct and indirect variables on behavior Hendry(Ed.),Conditionedreinforcement(pp.163–191). rather than as simply a static parameter of Homewood,IL:Dorsey. reinforcement. Ferster, C. B. (1953). Sustained behavior under delayed reinforcement. Journal of Experimental Psychology, 45, 218–224. REFERENCES Ferster,C.B.,&Hammer,C.(1965).Variablesdetermin- ingtheeffectsofdelayofreinforcement.Journalofthe Anderson, K. A., & Elcoro, M. (2007). Response acquisi- ExperimentalAnalysisofBehavior,8,243–254. tionwithdelayedreinforcementinLewisandFischer Ferster, C. B., & Skinner, B. F. (1957). Schedules of 344rats.BehaviouralProcesses,74,311–318. reinforcement.NewYork:AppletonCenturyCrofts. Arbuckle, J. L., & Lattal, K. A. (1988). Changes in Gleeson, S., & Lattal, K. A. (1987). Response-reinforcer functional response units with briefly delayed rein- relationsandthemaintenanceofbehavior.Journalof forcement. Journal of the Experimental Analysis of theExperimentalAnalysisofBehavior,48,383–393. Behavior,49,249–263. Gonzales,F.,&Newlin,R.J.(1976).Effectsofadelayed- Azzi,R.,Fix,D.S.R.,Keller,F.S.,&RochaeSilva,M.I. reinforcement procedure on performance under (1965). Exteroceptive control of response under IRT.t schedules. Journal of the Experimental Analysis delayed reinforcement. Journal of the Experimental ofBehavior,26,221–235. AnalysisofBehavior,7,159–162. Hall, G., Channell, S., & Schachtman, T. R. (1987). The Baum,W.M.(1973).Thecorrelation-basedlawofeffect. instrumental overshadowing effect in pigeons: The Journal of the Experimental Analysis of Behavior, 20, roleofresponsebursts.QuarterlyJournalofExperimental 137–153. Psychology,39B,173–188. 138 KENNON A. LATTAL Hand,D. J., Fox, A. T., & Reilly, M. P. (2006). Response Okouchi, H. (2009). Response acquisition with humans acquisition with delayed reinforcement in a rodent withdelayedreinforcement.JournaloftheExperimental model of attention-deficit/hyperactivity disorder. Be- AnalysisofBehavior,91,377–390. havioralBrainResearch,175,337–342. Peele,D.B.,Casey,J.,&Silberberg,A.(1984).Primacyof Hayes,S.C.,&Hayes,L.J.(1993).Appliedimplicationsof interresponse time reinforcement in accounting for current JEAB research on derived relations and rate differences under variable-ratio and variable- delayed reinforcement. Journal of Applied Behavior interval schedules. Journal of Experimental Psychology: Analysis,26,507–511. AnimalBehaviorProcesses,10,149–167. Hull, C. L. (1943). Principles of Behavior. New York: Pierce, C. H., Hanford, P. V., & Zimmerman, J. (1972). Appleton-CenturyCrofts. Effectsofdifferentdelayofreinforcementprocedures Iversen, I. H. (1981). Response interactions with signaled on variable-interval responding. Journal of the Experi- delayofreinforcement.BehaviourAnalysisLetters,1,3–9. mentalAnalysisofBehavior,18,141–146. Kendall,S.B.,&Newby,W.(1978).Delayedreinforcement Renner,K.E.(1964).Delayofreinforcement:Ahistorical offixed-ratioperformancewithoutmediatingextero- review.PsychologicalReview,61,341–361. ceptive conditioned reinforcement. Journal of the Reynolds, G. S. (1961). Behavioral contrast. Journal of the ExperimentalAnalysisofBehavior,30,231–237. ExperimentalAnalysisofBehavior,4,57–71. Kimble, G. A. (1961). Hilgard and Marquis’ Conditioning Richards, R. W. (1981). A comparison of signaled and andLearning.NewYork:AppletonCenturyCrofts. unsignaled delay of reinforcement. Journal of the Lattal,K.A.(1984).Signalfunctionsindelayedreinforce- ExperimentalAnalysisofBehavior,35,145–152. ment.JournaloftheExperimentalAnalysisofBehavior,42, Schaal, D., & Branch, M. (1988). Responding of pigeons 239–253. undervariable-intervalschedulesofunsignaled,brief- Lattal, K. A. (1987). Considerations in the experimental ly signaled, and completely signaled delays to rein- analysisofreinforcementdelay.InM.L.Commons,J. forcement. Journal of the Experimental Analysis of Mazur,J. A.Nevin,& H.Rachlin(Eds.),Quantitative Behavior,50,33–54. studies of operant behavior: The effect of delay and of Schaal,D.W.,Shahan,T.A.,Kovera,C.A.,&Reilly,M.P. interveningeventsonreinforcementvalue(pp.107–123). (1998). Mechanisms underlying the effects of un- NewYork:Erlbaum. signaled delayed reinforcement on key pecking of Lattal, K. A., & Abreu-Rodrigues, J. (1997). Response- pigeons under variable-interval schedules. Journal of independenteventsinthebehaviorstream.Journalof theExperimentalAnalysisofBehavior,69,103–122. theExperimentalAnalysisofBehavior,68,375–398. Schlinger, H. D., & Blakely, E. (1994). The effects of Lattal, K. A., & Crawford-Godbey, C. (1985). Homoge- delayed reinforcement and a response-produced neous chains, heterogeneous chains, and delay of auditory stimulus on the acquisition of operant reinforcement. Journal of the Experimental Analysis of behaviorinrats.PsychologicalRecord,44,391–409. Behavior,44,337–342. Schneider, S. M. (1990). The role of contiguity in free- Lattal, K. A., & Gleeson, S. (1990). Response acquisition operantunsignaleddelayofpositivereinforcement:A with delayed reinforcement. Journal of Experimental briefreview.ThePsychologicalRecord,40,239–257. Psychology:AnimalBehaviorProcesses,16,27–39. Schoenfeld,W.N.,&Cole,B.K.(1972).Stimulusschedules: Lattal,K.A.,&Metzger,B.A.(1994).Responseacquisition Thet-Tsystems.NewYork:Harper&Row. bySiameseFightingFish(Bettasplendens)withdelayed Shahan, T., & Lattal, K. A. (2005). Unsignaled delay of visual reinforcement. Journal of the Experimental Anal- reinforcement, relative time, and resistance to ysisofBehavior,61,35–44. change.JournaloftheExperimentalAnalysisofBehavior, Lattal, K. A., & Ziegler, D. R. (l982). Briefly delayed 83,201–219. reinforcementAn interresponse time analysis. Journal Shull,R.L.,Spear,D.J.,&Bryson,A.E.(1981).Delayorrate oftheExperimentalAnalysisofBehavior,37,407–416. offooddeliveryasdeterminersofresponserate.Journal Lejeune,H.,Richelle,M.,&Wearden,J.H.(2006).About oftheExperimentalAnalysisofBehavior,35,129–143. Skinner and time: Behavior-analytic contributions to Sizemore, O. J. (1976). The relation of response-reinforcer researchon animaltiming.Journal of the Experimental dependency and contiguity to signalled and unsignalled AnalysisofBehavior,85,125–142. delay of reinforcement. Unpublished doctoral disserta- Logan,F.A.(1960).Incentive.NewHaven:YaleUniversity tion,WestVirginiaUniversity. Press. Sizemore, O. J., & Lattal, K. A. (l977). Dependency, Mazur,J. E. (1987). An adjusting procedure for studying temporal contiguity, and response- independent delayed reinforcement. In M. L. Commons, J. E. reinforcement. Journal of the Experimental Analysis of Mazur,J. A.Nevin,& H.Rachlin(Eds.),Quantitative Behavior,27,119–125. analysis of behavior: Vol 5. Effects of delay and of Sizemore,O.J.,&Lattal,K.A.(l978).Unsignaleddelayof intervening events on reinforcement value. Hillsdale, NJ: reinforcementinvariable-intervalschedules.Journalof Erlbaum. theExperimentalAnalysisofBehavior,30,169–175. Moore, J. (1979). Choice and number of reinforcers. Skinner,B.F.(1938).Thebehavioroforganisms:Anexperimental Journal of the Experimental Analysis of Behavior, 32, analysis.NewYork:AppletonCenturyCrofts. 51–63. Skinner, B. F. (1948). ‘‘Superstition’’ in the pigeon. Morgan, M. J. (1972). Fixed-ratio performance under JournalofExperimentalPsychology,38,168–172. conditions of delayed reinforcement. Journal of the Skinner,B.F.(1953).Scienceandhumanbehavior.NewYork: ExperimentalAnalysisofBehavior,17,95–98. MacMillan. Nevin,J.A.(1971).Ratesandpatternsofrespondingwith Spence,K.W.(1947).Theroleofsecondaryreinforcement concurrent fixed-interval and variable-interval rein- indelayedrewardlearning.PsychologicalReview,54,1–8. forcement. Journal of the Experimental Analysis of Spence,K.W.(1956).Behaviortheoryandconditioning.New Behavior,16,241–247. Haven,CT:YaleUniversityPress.

See more

The list of books you might like

Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.