Schedules of Reinforcement

792 Pages·2014·40.418 MB·English

by  B. F. Skinner

SCHEDULES
OF
REINFORCEMENT
SCHEDULES
OF
REINFORCEMENT
by
C. B. FERSTER
Indiana University
Medical Center
and
B. F. SKINNER
Harvard University

B. F. Skinner Foundation
Cambridge, Massachusetts 02138
Forewords I and II Copyright © 1997, 2014 by the B. F. Skinner Foundation
Original copyright © 1957 by Prentice-Hall, Inc.
ISBN 978-0-9899839-5-2
B. F. Skinner Foundation Reprint Series
For information about the series please contact the B. F. Skinner Foundation at
www.bfskinner.org
Acknowledgments for this e-book version
The B. F. Skinner Foundation would like to thank the Behavior Analyst Certification Board (www.bacb.com) for the encouragement and
financial support that made this electronic format possible.
To
Members of the
“PIGEON STAFF”
FOREWORD I
ONE OF B.F. SKINNER’S most important contributions to the experimental analysis of behavior was his discovery of schedules of
reinforcement. In the 1956 paper, “A Case History in Scientific Method,” Skinner described how his short supply of food pellets led him
to ask why he should have to reinforce every bar press made by his rat subjects. His question, asked because of the purely pragmatic issue
that he was running out of self-made pellets on a Saturday afternoon, led him to the discovery of the first “fixed interval” one minute (FI
1) food delivery schedule of reinforcement. (It was fortuitous that Skinner did not use water as a reinforcer during this early experimental
time since he would not have run short.) Until then he had used either a continuous reinforcement program, wherein every response was
reinforced with a food pellet, or extinction, where no responses were reinforced. The number of bar presses was the variable of interest.
The FI 1 schedule arranged for the delivery of a pellet contingent upon the first response occurring after one minute since the last pellet
delivery. Skinner reports that he was amazed that under this schedule the number of responses increased. Next, while attempting to
maintain a stable rate of bar pressing by maintaining a constant level of hunger, he devised and scheduled “fixed ratio” reinforcement and
ran extended-length sessions, some for 24 hours. These two types of intermittent reinforcement schedules produced very different
patterns of responses. Skinner observed the results, and quickly realized the importance of differently scheduled consequences.
Scheduling the availability of consequences with regard to either time, or number of responses, or both, is the subject of this book.
Operants are selected by the complex interaction of individual actions and the timing and frequency of historical and current
consequences. These consequences, together with genetic influences thus determine operant behavior. Reinforcement schedules can
induce adjunctive behavior (which is behavior maintained by a stimulus that gets its reinforcing value from some other ongoing schedule
of reinforcement), and they direct the persistence and patterning of all behavior. Moreover, schedules contribute to a clarification of the
topic of motivation in that they can account for patterns and cycles of behavior. The significance of schedules of reinforcement cannot be
overestimated.
One concomitant development with the discovery of schedules was the evolution of the idea of considering operant behavior as a
different form of “reflex” than that described by Pavlov. Instead of being elicited by an antecedent stimulus this kind of behavior
operated upon the environment to produce consequences. Unlike respondent conditioning, with operants there was no need for individual
trials. The behavior was free to occur at any time in that it was not dependent upon a prior stimulus. For Skinner, this changed the
dependent variable from latency or magnitude to responses per unit time. The total number of responses divided by the time over which
the count was taken equals rate, and rate is the main dependent variable in the experimental analysis of behavior.
The discovery of schedules occurred around the years 1930 and 1931, or about 27 years before the publication of Schedules of
Reinforcement (hereafter Schedules). Schedules was published the same year as Verbal Behavior in 1957, which was also a major
contribution to our discipline, still not fully realized. The Journal of the Experimental Analysis of Behavior (JEAB), which was the first
major professional journal in our discipline, did not appear for two more years. Next to Skinner’s 1938 book Behavior of Organisms,
Schedules was the most frequently cited reference source in the first 4 years of JEAB.
Ferster and Skinner exposed their research subjects to schedule parameters until the birds’ performances were “stable,” although the
authors provide no objective definition of “stability” in Schedules. In a later book on research tactics, Murray Sidman noted that there is
no final answer as to how to determine behavioral stability, but that one calls behavior stable when a behavior does not change over a
sufficient period of time. The experimenter makes an informed judgement, based on his or her experience with subjects in a particular
laboratory, as to whether a subject is showing transition or steady-state performance. Transition is the time when the subject is coming
under the control of new schedule parameters, and stability is more representative of the final performance illustrating schedule control.
Ferster & Skinner speak of transitions and display many records showing transitions from one schedule to another. The precise transition
condition can be known only if the preceding performance is relatively stable and a final or terminal state of stability is regained.
Orderliness of the performance is important if one is to judge the effects of some change in conditions, and experimental analysis and
replication are paramount.
Ferster & Skinner determined stability by visual inspection; and no statistics were used in Schedules. They were able to employ this
practice successfully because their experimental manipulations produced large behavioral changes that were easily seen. Often a change
involved a beginning low rate of responding that then evolved into some high and typically patterned performance. A return to baseline
was rarely reported in Schedules, but subsequent replications and research often included reinstatement of original conditions in a classic
A-B-A single-subject reversal design.
Programming equipment used in Schedules is primitive by today’s standards. It was either hand-made by Skinner, by graduate students
on the pigeon lab staff, or constructed in the shop which was available to Psychology faculty at Harvard. Skinner typically made the first
approximation of a needed piece of equipment from material on hand. The final models were more “professionally” constructed by Ralph
Gerbrands, or by Rufus Grason and Steve Stadler, all of whom were mechanics in the shop and who later went on to found companies.
The prototype of the cumulative recorder currently in use was built by Gerbrands from Skinner’s plans about 1951, and it allowed
Ferster & Skinner to collect the data that are shown and described in Schedules. This recording device evolved from Skinner’s earlier use
of the physiologist’s kymograph to record the occurrence of individual (usually muscle) events. The cumulative recorder is not totally
unlike the invention of the microscope in that it, too, suddenly permitted the detailed inspection of ongoing life, in this case behavior and
its relationship to environmental events. The cumulative record permits immediate inspection and estimation of rate, observation of
changes in rate, as well as observation of the pattern of responding. This fact allows the experimenter to directly observe the effects of the
independent variable and to develop statements of functional relationships as they occur with reference to a stable baseline. Rate has been
the subject of extensive analysis, and keys called “rate meters” accompany cumulative records so as to allow the observer to visually
gauge the rate of responding from the slope of the record. Since cumulative records show moment-to-moment changes in the rate and
pattern of behavior as it occurs, as well as the operation of the food dispenser and other stimulus events, the records serve as
discriminative stimuli to control the behavior of the scientist. Because the behavior and the contingencies are directly observed, such
records reduce the temptation to make hypotheses about inner processes. By inspecting the record as it is being produced one can
instantly determine many variables of influence during the session, and with surveillance video one can also watch the subject in real
time. The experimental subject and the experimenter thus interact in a dynamic way impossible with group research, or work that requires
extensive post hoc statistical analysis to detect effects.
Systematically scheduling reinforcement allows for an analysis of the conditions governing the probability that a given action will
occur at a given time. These conditions specify the relation between independent and dependent variables. The history of contact with the
schedule provides the behavior that is visible under current conditions.
Schedules was one of the first extensive presentations of single-subject behavior research to appear in the literature since Ebbinghaus’s
work 44 years earlier. The demonstration of such control and predictability had never been seen in behavior science before. In this regard
Schedules was a major contributor to the formation of an independent science of behavior relations.
It was in 1976 that B.F. Skinner wrote “Farewell, My Lovely” as he observed in the Journal of the Experimental Analysis of Behavior
the gradual decline in the use of cumulative records to report results. His choice of words for the title of that paper indicated not only the
affection he had for the apparatus he invented but also the disappointment he felt for the decline in single-subject non-averaged behavior
reports. During 1967, in four issues of JEAB, out of 55 papers there were 27 that contained cumulative records (49%); in 1977 there were
21 out of 69 (30%); in 1987 there were 12 out of 53 (22%); and in 1993 there were only two out of 42. (4%). We might predict that
eventually there will be none.
The extensive research results presented in Schedules, which were collected over a three year period, are an incredible
accomplishment, particularly considering the available manpower, the equipment constraints, and the many variations of schedules
developed by Ferster and Skinner. The dedication of the book “To the Pigeon Staff” was intended to acknowledge the many graduate
students who also contributed during the years of the project. There are more than 900 Figures in the book that illustrate original data, and
15 separate schedules are described and demonstrated. The results presented in Schedules illustrate over 70,000 recorded hours and “…
approximately one-quarter billion responses.”
Ferster & Skinner present the effects not only of the schedule itself on acquisition, transition, and steady-state performance but of
many other independent variables as well. For example, drug treatment, brain ablations, added and novel stimuli, response contingent and
non-contingent time out, body weight alterations, satiation, limited hold, pre-feeding, water deprivation, pre-aversive stimuli, and yoked
subject conditions were used (p. 402). One dramatic example of the power of an independent variable affecting schedule controlled
performance is the FI with added clock (p. 277). In this study a visual “clock” stimulus was co-related with the interreinforcement time
during the fixed interval. When performance appeared to be in synchrony with the visual clock, the clock was suddenly reversed, that is,
it ran backward. The result was that the FI behavior “scallop” also inverted. Such a demonstration illustrates how discriminative
environmental variables, when paired with behavioral consequences, come to exert precise control over the actions of an organism.
The discovery and parametric manipulation of critical independent variables has been a milestone in every science. In biology there are
many independent variables including genetic, environmental, and chemical ones. The discovery of schedules of response consequences
as the critical variable in operant behavior was a major contribution to biobehavioral analysis.
The primary purpose of Schedules was “… to present a series of experiments designed to evaluate the extent to which the organism’s
own behavior enters into the determination of its subsequent behavior” (p. 3). A sometimes surprising finding for many experimenters, is
how schedules of reinforcement generate comparable patterns of responding independent of the species producing the behavior. A
schedule produces what the cumulative record portrays, a universal property of behavior, not that which is idiosyncratic to a particular
organism. One obvious advantage that cumulative records provide is that they allow a direct comparison of these behavioral properties
across species. As Skinner notes, once you have allowed for the different ways different species make contact with the environment, what
remains of their behavior shows astonishingly similar properties. Homing pigeons were most frequently used by Ferster & Skinner,
although White Carneaux pigeons were used, and some rat records are also shown. It is not possible to discern the species of the
behaving organism by inspection of its cumulative record.
The commentary provided by Ferster & Skinner about the cumulative records shown in Schedules provides insight and observation
that has yet to be fully exploited. It is not only a description of the records and what is happening but also an integration of the effects of
behavior contacting the environment.
The results of accidents, mistakes, and sheer “luck” during the work are noted by Ferster & Skinner. As everyone who has conducted
similar experiments has experienced, equipment breaks down, and when it does sometimes interesting things happen. A certain amount of
serendipity also occurs in all research that begins with the question “I wonder what will happen when I do this?”
The book contains three introductory chapters in which the authors explain the schedules they studied and the way the records were
obtained and prepared for presentation. Technical information about apparatus construction and operation, and methodology that was
used in conducting the studies is presented in Chapter 3. Valuable information, some that is rarely available even today, describes how to
maintain and train pigeons, as well as details of construction, purpose and use of programming instrumentation and apparatus. As Ferster
& Skinner mention, it is important to plan ahead when collecting data in the form of cumulative records, especially from multiple
chambers, because they accumulate quickly and arranging them for analysis can become a chore. The authors worked out a means of
preparing records by trimming out the white space and “nesting” the remaining segments. In addition to the body of the book, a 140-item
Glossary and an Index are included in the book. This may have been the first presentation of a glossary of operant analysis terms in the
literature.
Since publication, no one has come close, or even attempted, such an extensive analysis of any variable in the experimental analysis of
behavior. However, the results reported in Schedules have been systematically replicated literally hundreds of thousands of times in
thousands of laboratories, classrooms, institutions, clinics and other sites around the world. The fact that intermittent schedules of
reinforcement were first noted in 1938 yet no comprehensive treatment of them was forthcoming until 1957, (and only then by their own
discoverer) illustrates how far ahead of his contemporaries Skinner was. His treatment of verbal behavior in a book with the same name,
a schedule notation system, and the research reported with Robert Epstein in the Columban simulations are other examples of Skinner’s
versatility and creativity. We are still conducting parametric analyses and extensions of some of his seminal work.
Charles B. Ferster was a graduate student scheduled to finish his doctoral program at Columbia University with Fred S. Keller in June,
1950. Skinner asked for Keller’s nomination of someone to work in his lab and Ferster came to Harvard as Skinner’s full-time assistant
on February 1, 1950.
According to Skinner, Ferster was first author of Schedules to underline his equal partnership because if Skinner, as a professor of
psychology at the time, was first, then Ferster would be cast in the role of an assistant and Skinner reports that was not the case. Skinner
always indicated that Ferster was an equal colleague in creating and conducting the work reported in Schedules and therefore he should
receive equal credit. Authorship order was one way of insuring that credit.
Ferster’s and Skinner’s was possibly the most productive and creative five and one-half year collaborative effort we shall ever see in
the experimental analysis of behavior. Fred Keller, Skinner’s lifetime friend and colleague called it “… a landmark in the history of
behavior science.” Skinner himself wrote “It was the high point in my research history.”
Schedules of Reinforcement, and a few other books, notably Keller & Schoenfeld’s 1950 Principles of Psychology, the Holland and
Skinner 1961 program and text Analysis of Behavior, and Sidman’s 1960 Tactics of Scientific Research, served as my own introduction to
basic operant research. We read and used Schedules as a major reference source when I was a graduate student at Arizona State
University in the early 1960s. Jack Michael or Iz Goldiamond did not assign us the book as a text in any particular course, but they
suggested we learn as much from it as possible. And we did, because it was exciting reading and we learned as we collected data that
resembled the curves in the book. We referred to it often, especially the early chapters on technique and instrumentation. Generating VI
schedules could actually be fun. We used the well-known Fibonnacci numbers procedure (p. 335) to generate series of intervals, and
some very long pieces of 16mm film to operate the equipment. We made particular use of the descriptions of the construction of
equipment since there was still not much commercially available in those days and we frequently had to build our own apparatus from
“state surplus” items. To this day I remain a great admirer of fine microswitches, a good soldering iron, and multimeters. The opportunity
to scrounge and build equipment was a “hands-on” experience from which we learned about electrical components and programming,
and also the mechanics of environment-behavior interaction. (We were engaged in serious and productive recycling decades before it
became popular.) Mechner notation was our language and many evenings we tipped a few at Jack’s while discussing a particular rat or
pigeon subject’s cumulative records with Bob Dylan on the “hi-fi” in the background. It was very common to have to break from these
“seminars” in the middle of the night and go to the lab to check an ongoing experiment.
Schedules of Reinforcement has had a profound effect on the development of the experimental analysis of behavior and the
understanding of both human and nonhuman behavior. It will remain a classic in the field.
Carl D. Cheney
Logan, Utah
March, 1997
FOREWORD II
I
SCHEDULES OF REINFORCEMENT (Schedules) is an extraordinary monograph. It is an account of exciting scientific discoveries that were
both important and original. The material was quite unfamiliar except to a small coterie who had been close to the work. A monograph of
this magnitude is normally preceded by a series of technical papers in the scientific literature, describing reasonably coherent fragments
of the work as it progresses, so that people in the field can have some familiarity (which often passes as understanding) with the new
discoveries. But Schedules was not preceded by papers. It appeared full grown in 700 pages of almost entirely original material. To most
psychologists even the nomenclature was unfamiliar, although some terms had been used before. In a word, it was an uncompromising
challenge. Here it is, a mother lode of information on new discoveries: Go ahead and mine it. There are substantive written sections,
mostly in the early chapters, but the bulk of Schedules is the 921 figures and their accompanying description. This atlas of figures
contrasts sharply with the careful, analytical development in earlier books by Skinner (Behavior of Organisms and Science and Human
Behavior) and even more with other books in psychology. An account of why Schedules is such a different book will give some
perspective on the historical importance of the research and may help those approaching the book for the first time to understand it and
appreciate its significance.
Ferster and Skinner discovered the incredible power of schedules of reinforcement to engender patterns of behavior. Their own
behavior was so reinforced by the phenomena associated with schedule-controlled responding that, with the aid of automated equipment,
they did research 24 hours a day, 7 days a week, year after year. Skinner had stable research funds from the Office of Naval Research that
permitted Ferster and him to do uninterrupted research. Skinner did not publish any of this experimental work except for a report of a
paper given at the 1951 Congress of Psychology in Sweden, and Ferster wrote one technical article about how to do research on operant
behavior and published three experimental papers. Rather than stopping research to write reports, new experiments were planned on the
basis of the results of those just conducted. Progress was evident from the capability to do experiments that were not possible or even
conceivable earlier.
In the Festschrift volume for B.F. Skinner, Ferster gives a good description of the activities of the pigeon lab. He properly emphasizes
the effort that went into technical developments and the availability of shop facilities to build equipment. Keys and feeders were tried and
improved in a dozen iterations. The cumulative recording of responses, where each response causes a constant step movement of a pen
perpendicular to the constant rate of movement of the paper, deserves special comment. Four different models of cumulative recorders
were used, starting with one using a Ledex rotary switch as the main stepping mechanism and ending with a recorder built in the
Psychological Laboratories by Ralph Gerbrands and later produced commercially by Gerbrands and Co. in a number of still more
successively improved models. It was the cumulative recorder that permitted the recognition of the powerful effects of schedules. The
information shown in a cumulative record is equally contained in a series of blips corresponding to the steps on a horizontal line of a
polygraph, just as the information in most graphs can all be shown in a table of numbers. But the information conveyed to the observer by
the cumulative record, as with a graph, is far greater. Changes in rate of responding, indicated by changes in slope, are more obvious in
the cumulative record than in a polygraph. The cumulative record shows at a glance the pattern of changes in rate of responding in real
time over periods of hours or longer. The characteristic properties of different schedules would not have been discovered without the
cumulative recorder.
When at last Ferster and Skinner turned to writing an account of their research on schedule-controlled behavior, they described all of it
rather than summarizing the main findings. Dealing with the cabinets filled with cumulative records from experiments over several years
was a Herculean task that would have overwhelmed most people. Ferster and Skinner took to writing Schedules with boundless
enthusiasm. Long before multiple schedule control had been discovered as an experimental phenomenon, it had been Skinner’s practice
to bring his professional activities under strong stimulus control by working without interruption in a particular place. The room with the
cabinets of records was made the writing room. There were log books of the daily experiments, giving the details about schedules,
parameter values, and the subjects that were studied each day. With these books it was possible to retrieve the records for all experiments.
Ferster stopped doing any research (freeing about 10 independent experimental units for use by deserving graduate students), and for a
long period neither Ferster nor Skinner came into the pigeon lab except for a look at the cumulative records of experiments after they had
finished their daily stint of figure preparation.
Ferster’s Festschrift description of the mechanics of preparing the figures captures the flavor of their joint activities. The general
practice was for Ferster and Skinner together to look at cumulative records for each subject studied in a particular experiment and select
records to be photographed. This selection was undoubtedly the most important intellectual activity involved in the creation of Schedules
and its success is indicated by subsequent workers confirming the important characteristic features of schedule performances described in
Schedules, but it is impossible for a reader now to assess how the selections were made or to appreciate the extraordinary talent required
to understand the details of the records and to recognize the salient and replicable features. In Ferster’s account of writing Schedules, he
says “decisions about what to excerpt were made quickly, usually without much discussion, because we were both so familiar with the
records.” Because space limitations made it impossible to show photographs of all records as they were recorded without sacrificing
details, they devised a method for collapsing the time scale by “telescoping” the pen tracings (pp. 26–27, also described by Ferster).
Skinner loved making useful mechanical devices and also took pleasure in working with his hands, cutting out the pen tracings and