ERIC EJ941704: A Parametric Analysis of Errors of Commission during Discrete-Trial Training

2011·0.2 MB·English

by  ERIC

JOURNALOFAPPLIEDBEHAVIORANALYSIS 2011, 44, 611–615 NUMBER3 (FALL2011)
A PARAMETRIC ANALYSIS OF ERRORS OF COMMISSION DURING
DISCRETE-TRIAL TRAINING
FLORENCE D. DIGENNARO REED, DEREK D. REED, CYNTHIA N. BAEZ, AND
HELENA MAGUIRE
MELMARKNEWENGLAND
Weinvestigatedtheeffectsofsystematicchangesinlevelsoftreatmentintegritybyalteringerrors
ofcommissionduringerror-correctionproceduresaspartofdiscrete-trialtraining.Wetaught3
students with autism receptive nonsense shapes under 3 treatment integrity conditions (0%,
50%,or100%errorsofcommission).Participantsexhibitedhigherlevelsofperformanceduring
perfect implementation (0% errors). For 2 of the 3 participants, performance was low and
showednodifferentiationintheremainingconditions.Findingssuggestthat50%commission
errorsmay beasdetrimental as100% commission errors on teachingoutcomes.
Key words: treatment integrity, errors of commission, discrete-trial teaching, procedural
fidelity
_______________________________________________________________________________
Treatment integrity has been conceptualized (Northup, Fisher, Kahng, Harrell, & Kurtz,
as consistent and accurate implementation of a 1997; Rhymer, Evans-Hampton, McCurdy, &
treatment protocol in the manner in which it Watson, 2002). Similarly, Noell, Gresham, and
was designed (Gresham, 1989). Numerous Gansle (2002) and Wilder, Atwell, and Wine
applied studies provide evidence that client (2006) found that lower levels of treatment
gainsarebestachievedwhentreatmentintegrity integrity impeded the acquisition of academic-
is high (e.g., DiGennaro, Martens, & Klein- related skills (e.g., mathematics performance
mann, 2007; DiGennaro, Martens, & McIn- and compliance to directives, respectively).
tyre, 2005). However, in each of these studies, Although the studies mentioned above offer
client gains were simply correlated with ob-
scientifically rigorous examples of the impor-
served integrity levels. The degree to which the
tance of treatment integrity considerations, all
treatment was implemented with fidelity was
of these studies exclusively examined integrity
not systematically manipulated.
errors of omission (i.e., not implementing
Some studies included parametric analyses in
components of a protocol). Recent discussions
which differing levels of integrity were experi-
have expanded the concept of treatment
mentally manipulated to demonstrate a func-
integrity to include errors of commission (i.e.,
tional relation between treatment integrity and
a violation of integrity due to implementation
client outcomes. For example, several studies
of procedures that are not prescribed by the
demonstrated that behavior-reduction strategies
treatment protocol; St. Peter Pipkin, Vollmer,
were most effective when time-out procedures
&Sloman,2010;Vollmer,Roane,Ringdahl,&
were implemented with 100% integrity
Marcus, 1999; Worsdell, Iwata, Hanley,
Thompson, & Kahng, 2000). These studies
Florence D. DiGennaro Reed and Derek D. Reed are
now at the Department of Applied Behavioral Science, produced inconsistent results when behavior-
Universityof Kansas. reduction strategies featured commission errors.
Correspondence concerning this article should be
For example,Vollmeret al.(1999)demonstrat-
addressed to Florence D. DiGennaro Reed, Department
ofAppliedBehavioralScience,UniversityofKansas,4056 ed that omission and commission errors during
Dole Human Development Center, 1000 Sunnyside implementation of differential reinforcement of
Avenue, Lawrence, Kansas 66045 (e-mail: fdreed@
alternative behavior reduced treatment efficacy
ku.edu).
doi:10.1901/jaba.2011.44-611 if a higher rate of reinforcement was delivered
611
612 FLORENCE D. DIGENNARO REED et al.
for problem behavior. However, Worsdell et al. the experimenter’s hand within 5 s of the
(2000) documented therapeutic outcomes dur- discriminative stimulus. Percentage accuracy
ing functional communication training despite was calculated by dividing the total number of
commission errors (i.e., reinforcement of prob- correct responses by the total number of trials
lem behavior). In another study, commission and converting this ratio to a percentage. Data
errors were found to be more detrimental than werecollectedbythefirstauthor(adoctoral-level
omission errors in both a human operant task behavioranalyst)usingpaperandapencilduring
and with children with disabilities in an onetothree5-minsessions3to5daysperschool
educational setting (St. Peter Pipkin et al., week. The number of sessions varied due to the
2010). Based on the mixed results of these schedules of the classrooms and the experiment-
studies, it is clear that additional research is er. The participants did not receive classroom
needed to investigate the impact of commission instruction by the first author outside experi-
errors on responding. To date, no study has mentalsessions.Anindependentsecondobserver
exclusively examined errors of commission simultaneously collected data on participant
during instruction. Using the parametric anal- behaviorandtheexperimenter’simplementation
yses employed by Wilder et al. (2006) to ofdiscrete-trialtraining(i.e.,proceduralfidelity)
examine the effects of omission errors on during 57% (Alek), 35% (Justin), and 41%
preschoolers’ compliance to adult directives, (Salvatore) of sessions. For participant behavior,
we examined the impact of commission errors an agreement was scored when both observers
during discrete-trial instruction on the acquisi- scored student performance identically (i.e., as
tion of receptive identification of nonsense correct or incorrect). Agreement was calculated
shapes. as the number of agreements divided by
agreements plus disagreements, multiplied by
100%. Mean percentage agreement was 99.6%
METHOD
(range, 90% to 100%) for Alek, 99.4% (range,
Participants and Setting 90% to 100%) for Justin, and 99.8% (range,
We recruited three boys (Alek, Justin, and 98%to 100%) for Salvatore.Procedural fidelity
Salvatore) to participate in this study from a data were collected using a task analysis of
privateschoolforindividualswithautism.Allof discrete-trial teaching, error correction, and
the participants were 8 years old and had been consequence manipulation (i.e., appropriate
diagnosed with autism. Alek had also been implementation of errors of commission). Pro-
diagnosed with a variant of Landau-Kleffner cedural fidelity was 99.8% (range, 95% to
syndrome and a seizure disorder. During their 100%) for Alek and 100% for both Justin and
enrollment, participants received discrete-trial Salvatore.
instruction and were accustomed to delayed
reinforcement through the use of a token Design and Procedure
economy. The study took place in the students’ A combined multielement and nonconcur-
classrooms.Eachclassroomhadsixstudents,one rent multiple baseline design across participants
teacher, and two to three teaching assistants. was used to evaluate the effects of differing
levels of treatment integrity (i.e., commission
Response Measurement, Interobserver Agreement, errors) on participants’ performance during
and Procedural Fidelity discrete-trial instruction of receptive nonsense
Thedependent variablewasthepercentage of shapes.
discrete trials with accurate participant respond- Baseline. The experimenter presented three
ing(i.e.,percentageaccuracy),whichwedefined different cards horizontally on a tabletop, each
as placement of the correct target stimulus into depictingadifferentnonsenseshape.Subsequent
ERRORS OF COMMISSION 613
to appropriate attending (i.e., hands on lap or rect response was followed by an error of
tableandeyecontactwiththeexperimenter),the commission.During50%errorsofcommission
experimenterinstructedtheparticipantsto‘‘find (i.e., bouba), every other incorrect response was
[shape]’’ at which time the experimenter simul- rewarded with a token and social praise.1
taneously extended her hand to receive a card. Commission errors were not made during the
We allowed participants up to 5 s to exhibit a 0% errors of commission condition (i.e.,
response. The experimenter did not deliver manoo). That is, this condition was associated
programmed consequences (i.e., tokens, social with perfect integrity levels. Each session
praise, and error correction) contingent on consisted of 10 consecutive trials of one shape
performance. Each session consisted of 10 associated with that condition. The order of
consecutivetrialsofthesameshape.Thelocation conditions was counterbalanced across sessions.
of shapes on the tabletop was varied after each
trial and was determined a priori by computer
RESULTS AND DISCUSSION
randomization.
Figure 1presentstheparticipants’percentage
Consequence manipulation. The purpose of
accuracy across all sessions. During baseline,
this phase was to examine the effects of
performances were generally low across all
treatmentintegrity manipulation on participant
conditions. During the consequence manipula-
performance. Participants received instruction
tion, Alek’s performance across the three
as described in baseline; however, the experi-
conditions was initially undifferentiated. How-
menter rewarded unprompted correct responses
ever, after five sessions in each condition, His
with a token and social praise. In addition,
percentage accuracy was high and stable during
least-to-most prompting was used as an error-
the 0% errors of commission condition (M 5
correction procedure. Specifically, when partic-
90%) and low (M 5 42%) during the 50%
ipantsmadeerrors,theexperimenterprovideda
errors of commission condition. By the end of
gestural prompt to the correct shape followed
the study, his performance in the 50% errors
by physical guidance at the hand if the gestural
condition did not differ from performance
prompt did not evoke correct responding.
during the 100% errors condition (M 5
Correct responding with the prompt was
31%). Like Alek, Justin’s performance was
followed by a neutral statement only (e.g.,
initiallyundifferentiated,butafterapproximate-
‘‘that’s [shape]’’). Thus, prompted correct
lyfivesessions,heexhibitedahigherpercentage
responses were not followed by praise and a
accuracy when the experimenter made no
token.Theconsequencemanipulationconsisted
commission errors (M 5 47%). Performance
of implementing discrete-trial teaching at three
during the 50% and 100% errors conditions
different integrity levels (e.g., 100%, 50%, and
wassimilar(M524%inbothconditions).We
0%commissionerrors).Eachintegritylevelwas
observed differential outcomes immediately for
associatedwithonenonsenseshape(kiki:100%
Salvatore. His percentage accuracy was highest
errors; bouba: 50% errors; manoo: 0% errors).
during the0%commission errorscondition (M
A commission error consisted of reinforcing an
5 92%). He averaged 45% and 10% accuracy
incorrect response by providing a token and
social praise before the error-correction proce-
dure was implemented. Thus, errors were 1 Actualpercentagesoferrorsofcommissionwere52%,
reinforced as prescribed by each condition (see 52%,and53%forAlek,Justin,andSalvatore,respective-
below) followed by error correction using least- ly. Obtained percentages deviated slightly from pro-
grammed percentages due to the fact that student
to-most prompting. During the 100% errors of
performancedictatedhowmanycommissionerrorswould
commission condition (i.e., kiki), every incor- be possible in10trials.
614 FLORENCE D. DIGENNARO REED et al.
Figure1. Percentage accuracyonthe receptiveidentification task during discrete-trial teachingacross baseline and
consequence manipulation conditions foreach participant.
ERRORS OF COMMISSION 615
during the 50% and 100% commission errors integrity and student outcomes to identify the
conditions, respectively. level of integrity needed to prevent declines in
These findings replicate previous research and studentperformance.Theseresultscouldinform
suggest that relatively higher degrees of treatment practitionersaboutacceptablecriterionlevelsfor
integrityyieldbetteracquisition(Noelletal.,2002; stafftrainingandongoingevaluation.
Wilder et al., 2006). That is, commission errors
appear to degrade performance just as omission
REFERENCES
errorsdo.ThedegradinglevelsofaccuracyforAlek
DiGennaro, F. D., Martens, B. K., & Kleinmann, A. E.
imply that persistent low integrity could adversely
(2007). A comparison of performance feedback
affect performance even for those students who procedures on teachers’ treatment implementation
showinitialprogressundersuboptimalconditions. integrity and students’ inappropriate behavior in
special education classrooms. Journal of Applied
Most important, however, the present study
Behavior Analysis,40,447–461.
suggests that making some commission errors DiGennaro, F. D., Martens, B. K., & McIntyre, L. L.
contingent on incorrect responses may be just as (2005). Increasing treatment integrity through nega-
tive reinforcement: Effects on teacher and student
detrimental as making many commission errors
behavior. SchoolPsychology Review, 34, 220–231.
duringdiscrete-trial instruction. Gresham,F.M.(1989).Assessmentoftreatmentintegrity
Several limitations exist that should be ad- in school consultation and prereferral intervention.
School Psychology Review,18,37–50.
dressed in future research. First, all of the
Noell, G. H., Gresham, F. M., & Gansle, K. A. (2002).
participants had previous exposure to errorless Does treatment integrity matter? A preliminary
teaching strategies that were not used in this investigation of instructional implementation and
mathematics performance. Journal of Behavioral
study. The degree to which this affected perfor-
Education,11, 51–67.
mance is unknown, but we suspect that Justin’s
Northup, J., Fisher, W., Kahng, S., Harrell, R., & Kurtz, P.
relativelylowerperformanceevenunderoptimal (1997). An assessment of the necessary strength of
trainingconditionsmayhavebeeninfluencedby behavioraltreatmentsforseverebehaviorproblems.Journal
ofDevelopmentalandPhysicalDisabilities,9,1–16.
thetypeofteachingstrategyweselected.Second,
Rhymer,K.N.,Evans-Hampton,T.N.,McCurdy,M.,&
the assignment of one shape to each integrity Watson, T. S. (2002). Effects of varying levels of
conditionwasnotcounterbalancedacrosspartic- treatment integrity on toddler aggressive behavior.
Special Services intheSchools,18,75–81.
ipants; thus, faster acquisition in one condition
St.PeterPipkin,C.S.,Vollmer,T.R.,&Sloman,K.N.
may have been due to the specific instructional (2010). Effects of treatment integrity failures during
targetassociatedwiththatcondition.Finally,itis differential reinforcement of alternative behavior: A
translational model. Journal of Applied Behavior
unlikely that practitioners who work in applied
Analysis,43,47–70.
settingswillexhibiterrorpatternssimilartothose Vollmer,T.R.,Roane,H.S.,Ringdahl,J.E.,&Marcus,
that we programmed in this preliminary study. B. A. (1999). Evaluating treatment challenges with
differential reinforcement of alternative behavior.
That is, the procedural errors we examined may
Journal ofAppliedBehavior Analysis, 32,9–23.
notberepresentativeofthosetypicallycommitted Wilder,D.A.,Atwell,J.,&Wine,B.(2006).Theeffects
inappliedsettings,and,asaresult,theecological of varying levels of treatment integrity on child
compliance during treatment with a three-step
validity ofour findingsmay becompromised.It
prompting procedure. Journal of Applied Behavior
will be valuable, then, for future studies to
Analysis,39,369–373.
examine the effects of other types of procedural Worsdell, A. S., Iwata, B. A., Hanley, G. P., Thompson,
fidelity errors during discrete-trial instruction. R.H.,&Kahng,S.(2000).Effectsofcontinuousand
intermittent reinforcement for problem behavior
Moreover,researchsuggeststhatpractitionersare
during functional communication training. Journal
unlikely to implement procedures with 100% of AppliedBehaviorAnalysis,33,167–179.
accuracy in practice settings (e.g., DiGennaro et
Received May 25,2010
al., 2007). Thus, future studies should examine
Final acceptanceJanuary 5, 2011
the relation between decrements in treatment Action Editor,Thomas Higbee