ERIC EJ889223: The Effects of Self-Monitoring on the Procedural Integrity of a Behavioral Intervention for Young Children with Developmental Disabilities

2010·0.41 MB·English

by  ERIC

JOURNALOFAPPLIEDBEHAVIORANALYSIS 2010, 43, 315–320 NUMBER2 (SUMMER2010)
THE EFFECTS OF SELF-MONITORING ON THE PROCEDURAL
INTEGRITY OF A BEHAVIORAL INTERVENTION FOR YOUNG
CHILDREN WITH DEVELOPMENTAL DISABILITIES
JOSHUA B. PLAVNICK, SUMMER J. FERRERI, AND ANGELA N. MAUPIN
MICHIGANSTATEUNIVERSITY
Theeffectsofself-monitoringontheproceduralintegrityoftokeneconomyimplementationby
3 staff in a special education classroom were evaluated. The subsequent changes in academic
readiness behaviors of 2 students with low-incidence disabilities were measured. Multiple
baselines across staff and students showed that procedural integrity increased when staff used
monitoring checklists, and students’ academic readiness behavior also increased. Results are
discussedwithrespecttotheuseofself-monitoringandtheimportanceofproceduralintegrityin
publicschoolsettings.
Keywords: autism,developmentaldisabilities,earlychildhoodspecialeducation,procedural
integrity,self-monitoring, token economy
_______________________________________________________________________________
Proceduralintegrityisthedegreetowhichan elementary school teachers. Improved integrity
intervention is implemented as intended (e.g., was correlated negatively with student problem
Najdowski et al., 2008). Accurate implementa- behavior.Thesestudiesareunique,becausethey
tion of evidence-based interventions allows report educator and student data for an
data-based decision making regarding the intervention in a public school setting. Howev-
effectiveness of an intervention and increases er, an external researcher was needed to provide
the likelihood of improving student outcomes thefeedbacknecessaryforchangestoprocedural
(for a review, see Hagermoser-Sanetti & integrity; ongoing external support may not be
Kratochwill, 2008). Effective training practices, feasible in many school settings.
such as didactic instruction, behavioral rehears- Self-monitoring is a procedure that has been
al, and brief coaching, must be combined with shown to increase the accuracy with which
contingencies following training to ensure direct service providers implement a variety of
accurate implementation of behavioral inter- protocols in human service organizations (Allen
ventions (DiGennaro, Martens, & Kleinman, & Blackston, 2003; Richman, Riordan, Reiss,
2007; DiGennaro, Martens, & McIntyre, Pyles, & Bailey, 1988). Seligson-Petscher and
2005). In these two studies, performance Bailey (2006) examined the effects of self-
feedback and negative reinforcement were monitoringontheaccurateimplementationofa
combined to increase the procedural integrity behavior intervention plan by public school
of a behavioral intervention implemented by personnel. Despite increases in procedural
integrity, experimenters provided performance
feedback throughout all conditions, making it
This investigation was funded by the Office of Special
Education and Rehabilitative Services of the U.S. impossible to isolate the effects of self-monitor-
Department of Education (Grant H325D030060). We ingonproceduralintegrity.Thepurposesofthe
thankMary Mariagefor her collaboration andsupport.
currentstudywere(a)toidentifytheeffectsofa
Address correspondence to Joshua Plavnick, c/o Sum-
mer Ferreri, Counseling Educational Psychology and self-monitoring checklist on the procedural
Special Education, College of Education, 340 Erickson, integrity of token economy implementation by
MichiganStateUniversity,EastLansing,Michigan48824
early childhood special education (ECSE)
(e-mail: [email protected]).
doi:10.1901/jaba.2010.43-315 classroom staff and (b) to identify any changes
315
316 JOSHUA B. PLAVNICK et al.
in academic readiness behaviors of ECSE a 30-s whole-interval procedure and MotivAi-
studentswhenexperimentalmanipulationswere der vibrating timers to signal the end of an
applied to token economy implementation. interval. Engagement in academic readiness
behavior was scored when a student demon-
strated both target behaviors for an entire
METHOD
interval. Staff participants delivered token
Participants, Setting, and Materials
reinforcers to students for engaging in these
Three Caucasian staff members and 2 behaviors following a pretraining baseline.
Hispanic students participated in this research Observations were 10 to 15 min in length and
study. Ingrid (teacher), Teri, and Rita (para- occurred during small- and large-group activi-
professionals) were educators in a 10-hr per ties 2 or 3 days each week.
week ECSE program. Ingrid requested experi- Two independent observers scored 28% of
menter support for 2 students in the ECSE staff and 38% of student observations to
program who often required one-to-one adult evaluate interobserver agreement. Agreements
support to participate in group activities. Toby were identified when both observers scored the
(4 years old) had been diagnosed with autism same response for a component (staff partici-
and Kendra (3 years old) with Williams pants) or interval (student participants). Dis-
syndrome and specific language impairment. agreements were identified when interobserver
responses varied for a component or interval.
Measurement and Interobserver Agreement
Total agreements were divided by the sum of
Staff. Experimenters used a discrete categori-
agreementsplusdisagreementsandconvertedto
zation procedure (Kazdin, 1982) to score the
a percentage. Mean agreement for procedural
percentage of token economy components
integrity was 84% (range, 64% to 100%), 84%
implemented correctly. A component was
(range, 64% to 91%), and 95% (range, 82% to
scored correct if it occurred in the correct order
100%) for Rita, Teri, and Ingrid, respectively.
and as written on a token economy procedural Mean agreement for academic readiness behav-
checklist (available from the first author). An ior was 83% (range, 60% to 100%) and 94%
item was scored incorrect if it was not (range, 70% to 100%) for Toby and Kendra,
implemented as written or was implemented respectively. Low agreement scores occurred
in the wrong order, or if another response was during initial observations and were corrected
implementedinplaceoftheindicatedresponse. by the observers briefly discussing response
Experimenters conducted 15-min observations definitions.
2 to 3 days each week. Paper and pencil were
used to record data during small- and large- Design and Procedure
group activities. The effects of self-monitoring on procedural
Students. The dependent measures were the integritywereanalyzedusingamultiplebaseline
percentage of intervals students engaged in two design similar to that in DiGennaro et al.
academic readiness behaviors (i.e., appropriate (2005). A second multiple baseline design
sitting and vocalizing) simultaneously. Appro- across students was used to assess the effects of
priate sitting was defined as sitting in a staff- any changes in staff behavior on student
designated location and in a manner instructed behavior.
by staff with minimal movement for the entire Staff baseline.Staffbaselineconsistedofthree
interval. Appropriate vocalizing was defined as phases: pretraining, training, and implementa-
talking at or below conversational volume (i.e., tion. Consultants developed a token economy
could not be heard from more than 3 m away). procedure prior to the pretraining phase and
Datawerecollectedwithpaperandpencilusing measured the number of components complet-
EFFECTS OF SELF-MONITORING 317
ed when staff participants addressed student iable in effect for students was the token
engagement in academic readiness behavior as economy system as implemented by staff
theyhadthroughouttheschoolyearpriortothe participants in each condition.
start of research activities. Following pretrain-
ing, the experimenters conducted the training
RESULTS AND DISCUSSION
phase, which consisted of two 1-hr training
Figure 1 shows the percentage of token
sessions and used procedures similar to those of
economy steps implemented correctly across
DiGennaro et al. (2007) to train staff partici-
staff participants. Token economy implementa-
pants to implement each item on the token
tion did not occur during pretraining; the
economy checklist. The first session involved
combined procedural integrity for all staff was
staff participants only and consisted of didactic
0%. During training, implementation, and self-
review, modeling, and role play. The second
monitoring, the mean percentage of token
session involved staff and student participants.
economy steps implemented correctly for all
During the second training session, the exper-
participantswas70%(range,45%to82%),45%
imenter first modeled token economy imple-
(range, 0% to 91%), and 84% (range, 45% to
mentation with student participants and then
100%),respectively.Figure 2 showstherelation
provided coaching and feedback to staff
between levels of procedural integrity and
participants who implemented the procedure.
student behavior across experimental conditions
Once a staff participant demonstrated proce-
andparticipants.Meanengagementinacademic
dural integrity of 80% for one token economy
readiness behaviors during pretraining, training,
session,coachingandfeedbackwereterminated,
implementation, and self-monitoring was 25%
and the staff participant implemented the
(range, 20% to 40%), 95% (range, 80% to
intervention independently. The implementa-
100%), 52% (range, 0% to 100%), and 89%
tion phase continued for each staff participant
(range, 20% to 100%), respectively.
until procedural integrity was stable or down-
The results suggest that self-monitoring can
ward trending, and the experimenter and improve the implementation of a token econ-
individualstaffparticipantcouldmeettoreview omy system in a public school setting. Consis-
the self-monitoring procedure. tent with the findings of DiGennaro et al.
Staff self-monitoring. Prior to beginning self- (2007), behavioral interventions were not
monitoring, the experimenter met briefly with implemented accurately following initial train-
each staff participant and explained the mon- ing. This investigation extends previous proce-
itoring checklist, reviewed the token economy dural integrity research by demonstrating the
procedure,andanswered anyquestions. During effectiveness of self-monitoring for increasing
self-monitoring, staff participants continued to posttraining procedural integrity levels with
implement the token economy and were minimal support provided by external agents.
instructed to complete a token economy Future researchers should evaluate long-term
checklist(identicaltothatoftheexperimenters) implementation while employing school per-
following two token sessions of their choice sonnel (e.g., school psychologist) to collect
eachday.Completedchecklistswereleftinafile self-monitoring data and evaluate procedural
folderfor the experimenters to collect following integrity.
observations. Although these results demonstrate positive
Student conditions. Experimenters observed outcomes, some limitations should be consid-
and recorded the occurrence or nonoccurrence ered. First, staff and student participants
of academic readiness behavior during all demonstrated variability in responding follow-
experimental conditions. The independent var- ing the introduction of self-monitoring proce-
318 JOSHUA B. PLAVNICK et al.
Figure 1. The percentage of accurately completed token economy steps by each staff participant throughout all
conditions. Phases in baseline are pretraining, training, and implementation, and the intervention condition is self-
monitoring.
dures. The cause of variability was unknown, nents as opposed to the overall procedural
although procedural integrity may have been integrity within a token economy session.
controlled by competing demands, and aca- Future researchers could isolate the effects of
demic readiness behavior could have been individual components on student outcomes.
controlled by the accurate or inaccurate imple- Second, Ingrid’s accelerating trend in procedur-
mentation of specific token economy compo- al integrity during the implementation condi-
EFFECTS OF SELF-MONITORING 319
Figure2. Thepercentageofintervalsstudentsengagedinacademicreadinessbehaviors duringcorresponding staff
conditions. Phases in baseline are pretraining, training, and implementation, and the intervention condition is self-
monitoring.
tion suggests that an unknown variable con- efficacy of self-monitoring for supporting
tributed to this increase. This may have been accurate token economy implementation in a
due to observation of other educators imple- public school setting. In addition, student
menting the procedure accurately or to practice participants demonstrated improvements in
effectsovertheextendedbaselineperiod.Future academic readiness behaviors following the
research could address this limitation by introduction of the token economy. The use
examining the effects of observing accurate of self-management practices in school settings
implementationonratesofproceduralintegrity. may be an effective and efficient approach to
Despite these limitations, this investigation improve implementation of school-based inter-
extends previous research by demonstrating the ventions.
320 JOSHUA B. PLAVNICK et al.
REFERENCES Kazdin, A. E. (1982). Single-case research designs. New
York:Oxford.
Allen,S.J.,&Blackston,A.R.(2003).Trainingpreservice Najdowski,A.C.,Wallace,M.D.,Penrod,B.,Tarbox,J.,
teachers in collaborative problem solving: An inves- Reagon, K., & Higbee, T. S. (2008). Caregiver-
tigationoftheimpactonteacherandstudentbehavior conducted experimental functional analyses of inap-
change in real-world settings. School Psychology propriate mealtime behavior. Journal of Applied
Quarterly,18,22–51. BehaviorAnalysis,41,459–465.
DiGennaro, F. D., Martens, B. K., & Kleinman, A. E. Richman, G. S., Riordan, M. R., Reiss, M. L., Pyles, D.
(2007). A comparison of performance feedback M., & Bailey, J. S. (1988). The effects of self-
procedures on teachers’ treatment implementation monitoring and supervisor feedback on staff perfor-
integrity and students’ inappropriate behavior in mance in a residential setting. Journal of Applied
special education classrooms. Journal of Applied BehaviorAnalysis,21,401–409.
BehaviorAnalysis,40,447–461. Seligson-Petscher, E., & Bailey, J. S. (2006). Effects of
DiGennaro, F. D., Martens, B. K., & McIntyre, L. L. training, prompting, and self-monitoring on staff
(2005). Increasing treatment integrity through nega- behavior in a classroom for students with disabil-
tive reinforcement: Effects on teacher and student ities. Journal of Applied Behavior Analysis, 39, 215–
behavior.School Psychology Review,34,220–231. 226.
Hagermoser-Sanetti,L.M.,&Kratochwill,T.R.(2008).
Treatment integrity in behavioral consultation: Mea-
surement, promotion, and outcomes. International Received November 4,2008
Journal of Behavioral Consultation and Therapy, 4, Final acceptance August16,2009
95–114. Action Editor,Chris Ninness