ERIC ED602611: Reliability and Validity of a Spanish Language Assessment of Children's Social-Emotional Learning Skills

2018·0.22 MB·English

by  ERIC

Reliability and Validity of a Spanish Language
Assessment of Children’s Social-Emotional Learning
Skills
Authors: Jaclyn M. Russo (ORCID: 0000-0001-6237-1956), Clark McKown (ORCID: 0000-
0001-9694-1179), Nicole M. Russo-Ponsaran (ORCID: 0000-0002-9430-152X), Adelaide M.
Allen
Publication Year: 2018
Journal: Psychological Assessment
Publisher: https://www.apa.org/pubs/journals/pas/
PsychologicalAssessment ©2017AmericanPsychologicalAssociation
2018,Vol.30,No.3,416–421 1040-3590/18/$12.00 http://dx.doi.org/10.1037/pas0000508
BRIEF REPORT
Reliability and Validity of a Spanish Language Assessment of Children’s
Social-Emotional Learning Skills
Jaclyn M. Russo Clark McKown, Nicole M. Russo-Ponsaran,
UniversityofVirginia and Adelaide Allen
RushUniversityMedicalCenter
y.
s.adl FewSpanishlanguagetoolsareavailableforassessingimportantsocial-emotionallearning(SEL)skills.The
herbro presentstudypresentsevidenceofthepsychometricpropertiesofaSpanish-languageversionofSELweb
s
blied (SELweb-S),aweb-basedsystemforassessingchildren’sabilitytorecognizeothers’emotionsandperspec-
punat tives,solvesocialproblems,andengageinself-control.Withasampleof524studentsinGradesKto3,we
itsalliededissemi emcxoeamnmtpiomnseoidtdeuthaleesssreeesxlsihmaibbeiinltitetydscamonordedsvearealixtdehitityboitohefidgShhEiiLgnhwteerrnbea-lSlia.cboTilnhistiyiss,tse(tncuc)dyyaaspsnredosvsmimdoeeddnetreavmtiedoe6dn-umcleeotnshctahotrte(easm)fpiinotdraaivlitsdhtuaebaolirleiattsyics,ea(slblsy)-
ofob coherentfactorstructure,and(d)performanceonSELweb-Sassessmentmoduleswaspositivelyrelatedto
oneott teacher-reportedSELskills.FindingsarediscussedintermsoftheimportanceofdirectassessmentsofSEL
orsn skills in languages other than English. In addition, we highlight the importance of abiding by rigorous
ationandi recommendationsintheliteratureforthetranslationandculturaladaptationofassessments.
cier
os
ssu PublicSignificanceStatement
Aal
aldu Thisstudyadvancesthedirectassessmentofsocial-emotionallearning(SEL)skillsinschool-agechildren
ologicindivi baywperbe-sbeanstiendgdeivreidcetnacseseosfsmreleinatbioliftySaEnLdsvkaillildsitfyorofcahiSldpraennisihn-lGanrgaudaegseKvetrosi3o.nTohfiSsEsLtuwdyebh(iSgEhlLigwhetsb-tSh)e,
chhe importanceofdevelopingdirectassessmentsofSELskillsinlanguagesotherthanEnglish.
yt
Psof
ne
as
cu Keywords:directassessment,social-emotionalcomprehension,transadaptation,reliability,validity
erial
mn
Aso
eer
hp
bytthe Social-emotional learning (SEL) skills include the thinking to later academic, behavioral, mental health, and even economic
pyrightedsolelyfor sbkoidllysoafnrdesbeeahracvhiohrassnseheodwendtthoactrSeEatLespkoisliltsivinecchoinldnheoctoidonasr.eArellaartgede P(oIuilaltincnotoam,ise2,s0K1(ae2n.)gs..a,RsM,efPlceeCcnltneinsllygalnvtdha,eniAian,ccoarncekda,sWe&defMsotcoVursririsgooinnni,Sa2)E0Lh0a,6vf;eoSuaradbsootpalttee&ds
coed comprehensive SEL standards, and more than 10 states have
sd
in
entnte incorporatedaspectsofSELintotheirKto12standards(CASEL,
documcleisi TJahcilsynarMtic.leRuwsasso,pCuebnlitsehrefdorOAndlivnaenFceirdstSJtuudnyeo1f9,T2e0ac1h7i.ngandLearning, 2m0a1n5y).eFvoidreendcuec-abtaosresdcoSmEmLitpterodgtroamnusrthuarvinegbceheinldrcerne’asteSdE(LDsuksiellns-,
Thissarti UAndievlaeirdsietyAolfleVn,irgDienpiaa;rtCmleanrkt MofcKBoehwanv,ioNraiclolSeciMen.cResu,ssRou-PshonUsanraivne,rsaintdy blaugrsy,bCehaliinnd,.DAolmthiotruogvhicsht,a&ndWardesissabnedrgp,r2o0g1ra5m).sHaorweepvreorl,ifoenreatainrega,
Thi MedicalCenter. there are few usable, scalable, and informative systems for mea-
This research was supported by the Institute of Education Sciences
suringimportantSELskills.Evenfewerassessmentsystemshave
through Grant R305A110143 to Rush University Medical Center. The
been created for non-English-speaking students. The purpose of
opinionsexpressedarethoseoftheauthorsanddonotrepresentviewsof
theInstituteortheU.S.DepartmentofEducation.ClarkMcKownisthe thisstudyistodescribethepsychometricpropertiesofaSpanish-
founder and president of xSEL Labs, a company that licenses and sells languagetranslationofSELweb(SELweb-S),aweb-basedsystem
SELweb.Thisfinancialinterestbeganafterthisarticlewasacceptedfor for directly assessing important SEL skills (McKown, Russo-
publication.Theauthorsaregratefulforthehelpofthestudents,teachers, Ponsaran,Johnson,Russo,&Allen,2016).
administrators,andfieldstaffwhoparticipatedinthisstudy;themembers SELskillsincludementalprocessesinvolvedinthecomprehen-
ofthetranslationcommittee;andvoiceactorJohnnyArena.Wealsothank
sionofsocial-emotionalinformationandbehaviorsinvolvedinthe
JasonK.Johnsonforhissignificantcontributionstothisproject.
executionofsocialinteractions(Lipton&Nowicki,2009).Social-
CorrespondenceconcerningthisarticleshouldbeaddressedtoJaclynM.
emotional (SE) comprehension includes the ability to recognize
Russo,CenterforAdvancedStudyofTeachingandLearning,University
ofVirginia,405EmmetStreetSouth,RuffnerHall,Room221,Charlot- others’ emotions (“social awareness”) and perspectives (“social
tesville,VA22904.E-mail:[email protected] meaning), solve social problems (“social reasoning”), and some
416
SELASSESSMENTTRANSADAPTATION 417
aspects of self-control. SEL skills develop early in childhood will be culturally appropriate (Peña, 2007). Instead, translation
(Blair & Raver, 2015) and are strongly and positively correlated should be accompanied by a cultural adaptation process so that
with age (McKown, Allen, Russo-Ponsaran, & Johnson, 2013). questionsarefamiliarandappropriateforthestudentscompleting
Furthermore,SELskillsarevitalinbuildingachild’sfoundation theassessment(Peña,2007).Toachieveanappropriatetranslation,
for later learning in school (Blair & Raver, 2015). Specifically, acommitteeoffluentspeakersshouldassesstheculturalrelevance
self-controlandsocialcompetencesignificantlypredictinitiallev- ofthetranslatedassessmentitems,recommendmodifications,and
els of math and reading scores between kindergarten and second come to consensus about the wording of translated items; this
grade,andbetweenthirdandsixthgradeaftercontrollingforchild process is called transadaptation (van Widenfelt, Treffers, de
IQ,age,ethnicity,andmaternaleducationlevel(McClellandetal., Beurs,Siebelink,&Koudijs,2005).
2006). In light of the rapidly growing Spanish-speaking population in
Further,moreprosocialengagementinfirstgradeisassociated the United States and increased awareness of the need for strong
withhighermathachievementinfourthgrade(Luo,Hughes,Liew, SEL assessment in young children, we translated, culturally
& Kwok, 2009). Given the strong link between SEL skills and adapted, and validated a Spanish version of SELweb (“SELweb-
y. children’sacademicachievement,itissurprisingthattherearefew S”). The purpose of this article is to present evidence of the
shers.broadl tionoglstotoolsdifroerctalyssaesssseinsgsSSEELcosmkiplrlsehtehnastioarneinapspcrhooporilas.teMfoosrtuesxeisitn- pyssyesch(ComFAetsr;icAprrboupcekrlteie,s20o1f2S)EwLewreebu-sSe.dCtoontefsirtmthaetohryypfoatchteosrisanthaal-t
publinated schoolsareteacher-reportmeasuresthatarewellsuitedtoassess- scores from SELweb-S would reflect four moderately correlated
dmi ingobservablesocial-emotionalbehavior.Thosetoolsarenotwell factors. Cronbach’s alphas were calculated to test the hypothesis
salliedisse shueintseidontobmeceaaussuereththeeymreeqnutiarleporobcseesrvseesrsthtoatmmaakkeeauphiSgEhcleovmeplroef- tmhaotdufalecsto.rFsicnoarlley,retlwiaob-illietvieesl whioeuraldrcehxiccaeledlin.8e0arfomroadllelSsE(LHwLeMb-sS;
ite
ofob inference(Denham,2006;Ringwalt,2008).Directassessments,in Raudenbush, Bryk, & Congdon, 2004) were run to show that
oneott which children solve tasks that reflect comprehension skills, are students’scoresonSEL-webSexhibitedcriterion-relatedvalidity,
orsn better suited to assess the mental processes that make up SE as demonstrated by a positive association between age and
i
ationand comApdrderheessnisnigonth(iMsgcaKpo,wSEnLewteabl.,is2a01w3e)b.-basedsystemdesignedto SteEacLhweerbr-eSposrctoorefss,oacniadlsbkeitlwlse.en performance on SELweb-S and
cier
sous assess children’s SE comprehension from kindergarten through
s
ologicalAindividual taphbrioirlbditlyegmtroasd,reea.cnodFginevniezgeaagossethesiensrmss’eeelnfmt-comotinootdnrouslla.ensTdodpimreerecsaptlseyucrteaivsesemess,osstoicolhvnielrdseroceconiga’s-l Participants Method
he nition skill, for example, children view pictures of faces and
ch
yt
Psof indicate what each child is feeling from their facial expression. Data were collected in the fall of 2013 from three urban ele-
anse TwolargefieldtrialsoftheEnglishversionofSELwebfoundthat mentaryschoolsinalargeMidwesternmetropolitanarea;theRush
cu
erial (a)SELwebscoresexhibitedhighreliability,(b)SELwebmodules University Institutional Review Board approved all study proce-
Amson demonstrated a theoretically coherent factor structure, (c) factor dures.Participantsincluded524nativeSpanish-speakingstudents
heper scoresdemonstratedconvergentanddiscriminantvalidity,and(d) in bilingual classrooms in kindergarten through third grade
ythe controlling for IQ and demographic characteristics, performance (32.3%, 24.6%, 20.0%, and 22.7%, respectively). Of the 524
bt
yrightedolelyfor oaocnfcaetchpaetdaenamcsseie,cstsaemcahcehineetvremrmeopedonurt,lteaosnfdwsoancseigapalotsisvkiteiillvlyse,lryaenlaadtsesmdouctoilatittpeeldaechiwneidrtihrceappteooerrsrt s7t.u1d6eynetsa,rs28o2ldw(SerDe(cid:2)boy1s.0(953y.e8a%rs)).and children were, on average,
ps
coed ofproblembehaviors(McKownetal.,2016).SELwebprovidesa Measures
isnd feasibleandpsychometricallysoundassessmentofSEcomprehen-
entnte sioninchildren. SELweb. SELweb modules are described briefly in the fol-
mi
docucleis othHeorwtheavnerE,nthgeliashbsmenecaenosftdhiarteclitttalseseissskmneonwtsnoafbSoEutLninonl-aEnnggulaigshes- ltoowcoilngansdectsicoonr.inAgdpertoacileeddurdeessccraipntiboen ofofutnhde aindmMinciKstorawtinonetpraol-.
hisarti speaking children’s SE comprehension skills and reflects a limi- (2016).Groupadministrationoccurredinschoolsettings,eitherin
Ts
hi tation in the area of SEL assessment. To the best of the authors’ a computer lab or in classrooms on laptops. Children completed
T
knowledge, no research exists describing the development of as- the assessment independently, on average taking 40 to 45 min to
sessments of SE comprehension skills in non-English-speaking complete.
populations. However, during the 2013–2014 school year, 76.5% Socialawareness. Sixphotographsofchildfaceswithneutral
(approximately3.8millionstudents)ofallEnglish-languagelearn- facialexpressions,includingthreegirlsandtwoethnicminorities,
ersspokeSpanish,accountingfor7.7%ofallpublicK-12students. wereusedtocreatefacialemotionrecognitionstimuli.WithFace-
Thus, Spanish is the most prevalent language next to English in Gen software (Singular Inversions, 2005), digitized photographs
American schools today (Kena et al., 2016). To expand its reach werealteredintohigh-intensitydisplaysofhappy,sad,angry,and
andbenefit,SELdirectassessmentsshouldthereforebeavailable frightened. For each face and emotion, 10 faces were created,
inSpanish. rangingfromlow-tohigh-intensityaffectdisplays,formingapool
Translating an assessment from one language to another re- of240images,oritems.Fromthisitempool,fourtestformswere
quires a multistep process (Bracken & Barona, 1991). Simply created,eachwith40items,andchildrenrandomlyreceivedoneof
translatingandbacktranslatinganassessmentmayresultinaccu- the four test forms. Item difficulties were determined from an
ratelytranslatedinstructions,butitdoesnotensurethetranslation initialitemtryoutwith1,239childrenusingRaschmodeling(Item
418 RUSSO,MCKOWN,RUSSO-PONSARAN,ANDALLEN
difficultyrange(cid:2)(cid:3)3.2to5.1logits).Faceswereassignedtotest nobilingualmembersoftheresearchteam,atranslationcommittee
formstoensureabalanceofemotions,itemdifficulties,andchild wasassembled,asdescribedinthefollowingsection.
faceswithinagivenform.Aftereachfacewaspresented,children FollowingtherecommendationsofvanWidenfeltetal.(2005),
clicked to indicate whether the face reflected happy, sad, angry, the authors convened a translation committee of five members.
scared, or just okay. To adjust for differences in test form diffi- Committee members were bilingual in English and Spanish and
culties and thereby equate scores, item scores were scored, heldpositionsaspsychologists,schooladministrators,orteachers.
summed,andstandardizedwithinform. Becausetheyoccupieddifferentprofessionalrolesandhaddiffer-
Social meaning. Consistent with existing measures (e.g., entexperiencesasnativeSpanishspeakers,eachcommitteemem-
Happé, 1994), we created 12 illustrated and narrated vignettes in berbroughtanimportantperspectivetothediscussionoftransla-
which a character is disappointed, scared, sarcastic, lying, hiding tions.
feelings,orharboringafalsebelief.Aftereachstory,childrenwere Eachcommitteememberreviewedthetranslatedscriptpriorto
asked a question whose correct answer required accurate infer- agroupmeeting.Memberswereaskedtorankeachtranslatedline
ences about the story character’s mental state. Item scores were on a 4-point Likert scale of how well the line maintained the
y. summed across vignettes and standardized within form to place meaning and intent of the original line (4 (cid:2) good with no reser-
shers.broadl scoSroecsiafrlomreathsoendiinfgfe.renWtfeorcmresatoendaficvoemimllounstrmateetdrica.nd narrated vcaotuilodnsb,e3im(cid:2)prgoovoedd,,1bu(cid:2)t anlteeerdnsatiimvepsroavreempeonst)s.ibAlen,y2li(cid:2)neorekcaeyi,vbinugt
publinated vignettes involving ambiguous provocation situations and five an average score of 2 or lower was flagged as unacceptable and
dmi involvingpeerentry.Eachformalsoincludedthreevignetteswith prioritizedtodiscusswiththegroupduringthemeeting.Inaddi-
salliedisse gchirillsdraenndsetlherceteedwfirtohmbomyusltaipslep-rcohtoaigcoenoisptsti.onAsf:te(ra)eaanchatvtriigbnuetitoten, ttihoenc,oamnymliitnteeer.aDteidsclorewpearntthraantinag4swwaesrealrseosoeldvieteddinantwdoap3p-rhorvgerdobuyp
ite
ofob about the problem’s origin (benign or hostile intent), (b) a social meetings, starting with lines rated as a “1” or “2” and finishing
oneott goal, and (c) solution preference. Scores for each question were with those rated a “3.” The project team edited and finalized the
orsn summed across vignettes and standardized within test form to scriptaccordingtothedecisionsmadebythetranslationcommit-
i
ationand efoqrumatsewscitohressix. Tvoigrneedttuecseeraecshp.oEndacehntffoartmiguinec,lwudeecdretharteedeafimvebitgeus-t teeF.inally,abilingualvoiceactorrecordedthescript.Projectteam
cier
sous ous provocation vignettes and three peer entry vignettes. Each members were present for all recording sessions. After all lines
s
Aal vignettewasincludedonthreeforms.Childrenrandomlyreceived were recorded, they were processed and incorporated into the
ologicalindividu oneSeolff-fciovnetrteoslt.foWrmes.developedachoice-delaytask(Kuntsi,Ste- sisyhstveemrs.iTonheantrdanaslllamtioemnbceormsmagitrteeeedrtehvaitewtheedatshseescsommenptlewteadsSrepaadny-
he venson, Oosterlaan, & Sonuga-Barke, 2001) and a frustration- foradministration.
ch
yt
Psof tolerance task (Bitsakou, Antrop, Wiersema, & Sonuga-Barke,
ne 2006).Forthechoice-delaytask,childrenearnedpointsforsend-
caus Results
erial ingrocketshipstospace;theslowerandthereforemoretediousthe
Amson rocket ship, the more points the child earned. Children had 10
heper opportunities to score as many points as possible. For the Missing Data
ythe frustration-tolerancetask,childrenweretoldtoidentifywhetheror
yrightedbolelyfort ncgooeuttl“tdwstiouncs9kh0”apsb.eeFsfoowrreesremevoeidvraeinlngtiitceoamnlsat,onthdtehteocongmeextptuatsietermmwa,natyshpecrrooerbgreyracmtinamdsuectdhinetogy dstrueTdneyaw(cihh.oee.sr,es1cp2oa9mreopnflte(thste)edp5rD2o4EviScdhSeiAdld-crmoenninswienqhtuoetopstaiproatnirctniiapciiarpteeasdtefoiinrnttahhneosasetdudcdh-yoil)n-.
copeds mildfrustration. Children with and without DESSA-mini scores did not differ
isnd TheDevereuxStudentStrengthsAssessment(DESSA)-Mini. significantly in sex, age, perspective-taking skills, problem-
entnte TeacherscompletedoneoffourequivalentformsoftheDESSA- solvingskills,orself-control.Childrenwhoseteacherscompleted
mi
docucleis mnaiinrei i(nNwaghliicehri,thLeefBreuqfufee,nc&yoSfhabpeihraov,io2r0s1r1e)f,leacntin8g-itSeEmLqsukeilsltsioanre- tehmeotDioEnSSrAec-omginniitiosncorteadsk.1t9hanstanchdialrddrendevwiaitthioonuthiDgEheSrSAon-mtihnei
hisarti rated. Items are scored on a 5-point scale (i.e., never to very scores(p(cid:5).05).
Ts
hi frequently). The DESSA-mini provides one overall social-
T
emotionalcompetencescore,theSocial-EmotionalTotal(Naglieri
Reliability
et al., 2011). The average internal consistency across forms was
(cid:4)(cid:2).88(range(cid:4)(cid:2).85to.91). Internal consistency. To test reliability hypotheses, we cal-
culated the internal consistency of scores. For Social Meaning, a
single score measure, we computed Cronbach’s alpha. Social
Procedures
Awareness, Social Reasoning, and Self-Control scores were de-
TheEnglishscript,includingallspokendialogueandanyprint rivedfrommultipleformsandscores.Internalconsistencycoeffi-
text,wastranslatedintoSpanishandthenbacktoEnglishbythe cients for each score were calculated and averaged across forms.
Rush University Medical Center translation service. Next, the Factor score reliabilities were estimated using procedures de-
studyauthorsreviewedtheEnglishscriptanditsback-translation scribed by Nunnally and Bernstein (1994, p. 271). Composite
for accuracy. Specifically, study authors independently reviewed scorereliabilitiesweredeemedacceptableandrangedfrom.67to
theentirescriptandmadedetailednoteswheretheybelievedthe .94.ComprehensivescorereliabilitiesaresummarizedinTable1
translation deviated from the original intent of the script. Having (i.e.,r ).
yy
SELASSESSMENTTRANSADAPTATION 419
Table1 tion and perspective-taking both reflect understanding others, we
ScoreReliabilities next tested a two-factor model with Emotion Recognition and
Perspective-Taking scores loading on one factor, and the Social
SELwebcompositescore ryya r12b Problem-Solving and Self-Control scores on a second factor.
Model fit was again poor (CFI (cid:2) .76, RMSEA (cid:2) .136, 90% CI
Social-Emotional(SE)Comprehension .94 .73
SocialAwareness .91 .56 [.124, .149]). Because problem solving and self-control may be
SocialMeaning .67 .68 distinctfromoneanother,wenexttestedathree-factormodel,with
SocialReasoning .85 .64 Emotion Recognition and Perspective-Taking scores loading on
Self-Control .81 .51
one factor, Self-Control scores on a second factor, and Social
SELwebmoduleandscore (cid:4) r Problem-Solving scores on a third factor, and the fit was good
12
(CFI(cid:2).91,RMSEA(cid:2).078,90%CI[.064,.092]).Thefitofthe
SocialAwareness four-factormodelwasexcellent(CFI(cid:2).99,RMSEA(cid:2).036,90%
Happy .71 .36
CI [.017, .053]). That model, depicted inFigure 1, was a signifi-
Sad .75 .43
shers.broadly. ASSconacgrireaydl-Meaning/Perspective-Taking ...686817 ...446598 cwaintShtolycthiabeleftAotewurra-frfieatncttoeosrsth,meSooddcaeitala,lt(cid:6)Mh(cid:7)aen2a/ndthifne(cid:8)ga,4lSt0eo,rcnpiaa(cid:5)tlivRe.e0sa5s()oa.lnlincgo,mapnadriSsoelnfs-
dpubliminated SoPPciooassliittRiivveeeasASoonttcirniiabgluGtioonal ..6686 ..5543 Csioonntrloatlelnotavdaerdiaobnlea(CseFcIo(cid:2)nd-.o9r7d,eRrMSoScEiaAl-E(cid:2)m.o0t3io4n,a9l0C%omCpIr[e.h0e1n5-,
salliedisse SePlfo-CsiotinvteroSlolutionSelection .77 .54 .t0h5is1]a)n.aTlyhseisS(“oScEialc-Eommpootisointea”l)Cwoamspcroemhepnusteiodnasfaactworeisgchotreedfsruomm,
ofitobe DFreulsatyraotifoGnrTaotilfeicraanticoen ..6834 ..4422 withfactorloadingsastheweightingvalue.Thisscorewasusedin
ationoroneandisnott sNatIaonbtteiel.ritnyaSloEcfoLSnEs(cid:2)iLswtseoencbciyacloo-mefmSpoEostLiiotweneaabnldcleomamronpdionusgliet.escaonrdesm.odulescores. b6-month ciSsnoreigcltfei-arSCli-oooecnmni-atrorleotlilaAo,tnewwdaaelvrceuaonlsimeedsipstth,ryeeSahSnoeaEnclisyaicosloenmMs.ipnAeoalcsnoriitninetgegra,wisoiSntaho-nrcfeioaalcavltteoeRrdraelsvlacasoionlrindedsiiinctryaget,foalnreaacnoltdf--
cier
sous yses.
s
alAdual Six-month stability. From the total sample, 209 children
Psychologicoftheindivi tacbhroeeemtswepperldeeeansteteatdnettsoeStidEenvgLianwwlueTaabsateb1al67e-6sme1dcoao(nyin.tsedh.(,mrtairmenages)eu.i(cid:2)rnBeme1tch8eaen3utsstsoeptra1icbn8hig6liil.tddyMra,eyanesna)dnw.Wfeiirnneetdeuirrnsavegnads-l ee RecHoSagapndpi (cid:2) y on ..7756**
canuse domly assigned to Social Aware1n2ess and Social Reasoning test Recogni(cid:2)on .73* Social
erial forms,forthoseassessmentmodules,temporal-stabilityestimates e Angry Awareness
Amson reflected a mix of alternate forms and test–retest reliability. Six- Recogni(cid:2)on 81*
bythetheper mcoomthposstiatbeilsictyorescdoermesonwsetrraetimngodtheerastetrownigtehsttheestiSmoactiealoRfteeamsopnoirnagl e RecSocagrnei(cid:2)do n .43*
pyrightedsolelyfor Vstaabliilditiyty(i.e.,.68). e PerTsapkeinc(cid:2)g ve 1.0* MSeoacniainl g .45*
coed
isnd Factor structure. We hypothesized that SELweb-S scores
mentinte would fit a four-factor model of social-emotional comprehension .56*
docucleis ianndwSheiclfh-CSoonctiraollArewflaercetnceosrsr,eSlaotecdiallaMtenetanvianrgia,bSleosci(asleeReMascoKnoinwgn, .52*
hisarti et al., 2016). To test this hypothesis, we used Amos 21.0.0 (Ar- e Posi(cid:2)ve .61*
Ts A(cid:3)ribu(cid:2)on
hi buckle,2012)torunconfirmatoryfactoranalyses(CFAs).Forthe
T single-indicator latent variable reflecting perspective-taking, the e Social .65* Social .83*
Goal Reasoning
errorvariancewasmodeledas1minustheinternalconsistencyof .76*
the indicator variable score (Kline, 2005). We compared our e Posi(cid:2)ve
hypothesized model with three plausible alternatives to evaluate Solu(cid:2)on .69*
theextenttowhichourhypothesizedmodelwassuperiortothose
alternatives. Because simpler CFA models were nested within
more complex models, the change in chi-square goodness of fit e Delay of .48*
Gra(cid:2)fica(cid:2)on
wasusedtocomparethefitofthemodelswithoneanother(Kline, Self
2005). e Frustra(cid:2)on .59* Control
Tolerance
The fit of a one-factor model, in which all scores loaded on a
single factor, was poor (comparative fit index [CFI] (cid:2) .78, root Figure 1. Confirmatory model of social-emotional comprehension.
mean square error of approximation [RMSEA] (cid:2) .130, 90% CFI (cid:2) comparative fit index; RMSEA (cid:2) root mean square error of
confidence interval [CI] [.117, .142]). Because emotion recogni- approximation;CI(cid:2)confidenceinterval.(cid:2)p(cid:5).05.
420 RUSSO,MCKOWN,RUSSO-PONSARAN,ANDALLEN
Criterion-related validity. We ran two kinds of analyses to
test criterion-related validity hypotheses. First, we ran two-level
HLMs (Raudenbush et al., 2004), with classroom as the Level 2
unit, evaluating the relationship between age and each social-
emotionalcomprehensionscore.Wefoundthatagedemonstrated
a curvilinear relationship related to social-emotional comprehen-
sionscoresasshowninTable2.Specifically,children’sscoresin SE Comp
social awareness, social meaning, social reasoning, self-control, Awareness
e
and overall SELweb-S performance increased with age. In addi- cor Meaning
S
tion, the age-related increase in these skills declined with age. z Reasoning
Figure 2 shows that each of the skills measured by SELweb-S Self-Control
improvedwithage,buttheslopeoftherelationshipbetweenage
andperformancealsoleveledoffwithage.
y. To further evaluate criterion-related validity, we ran two-level
shers.broadl HcoLmMpsreehveanlsuioatninsgcotrheearnedlatthioenDshEiSpSbAe-twmeineinTesaccohres,occoianlt-reomllointigonfoarl
publinated age and sex. Social-emotional comprehension scores and the Age
dmi DESSA-miniTscorewerestandardizedfortheseanalysessothat
itsallieedisse ccooemffpirceihenentssiorenfleacntdingDEthSeSAas-smoicniaitisocnorbesetwcaenenbesociniatel-repmreotetidonaasl Fcoigmuprreeh2e.nsMiono.deSlE-in(cid:2)fersroecdiarle-leamtiootnisohniapl.betweenageandsocial-emotional
ofob standardizedregressioncoefficients.AsshowninTable2,eachof
oneott thesocial-emotionalcomprehensionscoreswassignificantlyasso-
orsn ciatedwithteacherreportofsocial-emotionalcomprehension. ture,and(d)performanceonSELweb-Sassessmentmoduleswas
i
ationand Discussion pHoLsMitivreelsyulrteslaptreodvtiodeteuascehresro-rfepSoErLtewdeSbE-SLwskitihllsc.oAnfdiddietniocneatlhlya,tothuer
cier
sous translated version of this assessment is measuring what it was
Asal SELweb addresses a need for direct assessments of SEL skills designedtomeasure(i.e.,SEcomprehension)withinapopulation
hologicaleindividu apSpotapptrueolspa,trtiihoaenteafouoftrhuonsrasetivdineevsceShlpooaponelidss.haI-nnsdpceofainkesliidndgeterasstttieuoddneaonftSstphaiennifsathsh-te-lagnrUognwuiaitnegdge owasfistohncariteaisvtiueolntsSpfbraeontmwisehoe-unsrpeEthanekginltiegsahc-chshpeierldarrkeeipnno.grAptod(pid.ueitl.ia,otinDoanEllSyoS,fAcahn-midldinraeil)nig,natnhedde
ch
Psyoft version of SELweb. As such, the purpose of this study was to SELweb-SwasmoderateindicatingthattheSELskillsmeasured
ne describe the process by which SELweb was transadapted into throughSELweb-Saredistinctfromthebehavioralcomponentsof
as
ericalu Spanish and to present evidence of the reliability and validity of SELasmeasuredthroughteacherreport.Finally,ourresultshigh-
eAmerson SEULswinegb-tSh.e transadaptation process, we translated and culturally lmigahntceed,aunsdtreornscgorreinlagtitohneshimippboerttawneceenoafgceoamnpdarSiEngLwaecbh-iSldp’serpfoerr--
hp
ythe adapted a complex developmental assessment. Study findings formance across SEL skills with their age-mates. Overall, our
yrightedbolelyfort htehsigeeishrltiihgnahtettn(tadh)eaditnpSduiEvrLpidwouseaebl.-aSSspseyeciseisflmidcseanlrlteylm,iarobedsleuullsetscsoseruxephspibothirtateetddamoreuorvdhaeylriapdtoefthoto-r saktsnusodewsyslmpedreogndetutchoeafdtsiasocpaivsayalc-ilehamobmloeteifotornircaualsllecyoamstospucrnaedlheeSinnpsasioncnihs,ohotlhlasen.Igfmuirapsgtoerttodaniroteluycrt,
iscopndeds h(big)hcionmteprnoaslitceonassissetsesnmcyenatndscmooredsereaxtehi6b-imteodnthhigtehmrpeolriaablisltiatyb,ili(tcy), SthEeLirwSepba-Snisahll-oswpesatkeiancghesrtsutdoendtisr’ecStlEy,Lfesaksiilblsly,wainthdovuatlindeiteydiansgsetsos
entnte assessmentmodulescoresfitatheoreticallycoherentfactorstruc- speakSpanishthemselves.
mi
docucleis meAasltuhroeudgbhythSeEreLwweebre-Snobestiwgneeifnictahnet1d2if9fecrehnilcderseninwShEoLsespkailrlesnatss
Thissarti THaiebrlear2chicalLinearModelsoftheAssociationBetweenSocial- consentedandthosechildrenwhoseparentsdidnotconsent,alow
hi consent rate reflects a limitation of our study. Additionally, our
T EmotionalComprehensionandDESSA-miniTScore,
studydesigndidnotincludeaversionoftheassessmentthatwas
ControllingforAgeandSex
translated by other means, so we cannot say definitively that the
SELwebpredictors transadaptation process is superior to other translation methods.
However,giventhestrongevidenceofscorereliabilityandvalid-
SELweb Social Social Social Self-
ity, we are confident that this method of translation is highly
Variable overall awareness meaning reasoning control
effective.Therigorousrequirementsofthetransadaptationprocess
Intercept (cid:3).12 (cid:3).14 (cid:3).13 (cid:3).10 (cid:3).11 maydetermanytestdevelopersfromusingittotranslatetestsinto
Age (cid:3).08 .04 (cid:3).07 .01 (cid:3).10
another language. However, we believe that following these rec-
Sex .14 .19 .20 .18 .15
SELweb ommendations is likely to result in better measurement systems
score .36(cid:2) .25(cid:2) .28(cid:2) .20(cid:2) .37(cid:2) and,assuch,westronglysuggestthatothertestdevelopersadhere
tothisprocedure.
Note. SEL(cid:2)social-emotionallearning;DESSA-mini(cid:2)DevereuxStu-
dentStrengthsAssessmentMini.Coefficientsarestandardized. In addition, our analyses did not account for the nesting of
(cid:2)p(cid:5).05. children in classrooms and schools. Because the degrees of free-
SELASSESSMENTTRANSADAPTATION 421
dominthemodelsexceededthenumberofclassrooms,wecould Kuntsi, J., Stevenson, J., Oosterlaan, J., & Sonuga-Barke, E. (2001).
notuseamoreadvancedmodelingstrategysuchastheComplex Test–retestreliabilityofanewdelayaversiontaskandexecutivefunc-
Samples Facility in Mplus (Muthén & Muthén, 2012). In the tionmeasures.BritishJournalofDevelopmentalPsychology,19,339–
future,itwillbeimportanttoreplicatethisworkwitheitherlarger 348.http://dx.doi.org/10.1348/026151001166137
Lipton,M.,&Nowicki,S.(2009).Thesocial-emotionallearningframe-
samplesofclassroomsorindependentsamplesofchildren.
work(SELF):Aguideforunderstandingbrain-basedsocial-emotional
learningimpairments.TheJournalofDevelopmentalProcesses,4,99–
Conclusion
115.
Luo, W., Hughes, J. N., Liew, J., & Kwok, O. (2009). Classifying aca-
The push for reliable and valid direct assessments of social-
demicallyat-riskfirstgradersintoengagementtypes:Associationwith
emotional comprehension that are appropriate for use in schools
long-term achievement trajectories. The Elementary School Journal,
continues to grow. As a result, educational researchers aim to
109,380–405.http://dx.doi.org/10.1086/593939
createeasilyimplementabledirectassessments.SELwebwascre-
McClelland,M.,Acock,A.,&Morrison,F.(2006).Theimpactofkinder-
atedtoaddressthisneed.Tothebestofourknowledge,thisstudy garten learning-related skills on academic trajectories at the end of
y. isthefirsttotranslateandculturallyadaptacomputer-automated, elementaryschool.EarlyChildhoodResearchQuarterly,21,471–490.
s.adl scalable,directassessmentofsocial-emotionalcomprehensioninto http://dx.doi.org/10.1016/j.ecresq.2006.09.003
herbro Spanish.Assuch,wewereabletoprovideimportantinformation McKown, C., Allen, A. M., Russo-Ponsaran, N. M., & Johnson, J. K.
s
blied toteachersforagroupofstudentswhootherwisewouldnothave (2013). Direct assessment of children’s social-emotional comprehen-
itsalliedpuedisseminat beenabletocompletetheRasesfeesrsmenecnet.s Mc.As1Kio.0on3(w.270/nPa1,0s6Cy0)c3..,h3WRo4l3ueo5sbgs-iobc-aaPsloedAnssaassreassenssm,sNmen.etMn,t2.,o5fJ,och1hn1isl5do4rne–,n1J’1.s6Ks6o..,cRhiatult-pses:m/o/d,oxJti..o,dno&ai.loArcglol/em1n0-,
ofob Arbuckle,J.L.(2012).Amos(Version21.0.0).Chicago,IL:SPSS,Inc. prehension. Journal of Psychoeducational Assessment, 34, 322–338.
et
onot Bitsakou,P.,Antrop,I.,Wiersema,J.R.,&Sonuga-Barke,E.J.(2006). http://dx.doi.org/10.1177/0734282915604564
orsn Probing the limits of delay intolerance: Preliminary young adult data Muthén,L.K.,&Muthén,B.O.(2012).Mplususersguide(7thed.).Los
i
sociationuserand Blaforidor,sm,C1t.h5,e1&,D3Re8laa–vy4erF4,.rCuhs.tttrCpa:.t/i/(od2nx0.T1d5aos)i..koSr(gDc/he1oF0oT.l1)0r.e1Ja6od/uji.nrjnneasesluomafneNdthes.u2erl0of0-sr5ce.ig0eu6nl.ca0et3iMo1ne:thA- NaASgtlnriegenreigl,etJsh.,sACA.A,ssL:eeAsBsumuthfefonert,–.PM.iAni.,(&DESShSaApi-rmoi,nVi)..BL.e(w2i0s1v1il)l.eD,NevCe:reKuaxpSlatund.ent
Asal developmental psychobiological approach. Annual Review of Psychol- Nunnally,J.C.,&Bernstein,I.H.(1994).Psychometrictheory(3rded.).
ologicalindividu Bro0ac1gk5ye,2n26,16B,.7A1.1,–&731B.arhotntpa:,//Adx..d(1o9i.9o1rg)./1S0t.a1t1e4o6f/anthneuraervt-ppsryocched-0u1re0s81f4o-r PeNñcraeo,wsEs-.YcDuolr.tku(,r2aN0l0Y7re:)s.MeLacorGcshtr.ainwC-thHrialindlls.lDateiovenl:oMpmeethnot,do7l8o,gi1c2a5l5c–o1n2s6id4e.rahttitopn:/s/dinx
he
ycth translating, validating and using psycho-educational tests in cross- .doi.org/10.1111/j.1467-8624.2007.01064.x
Psof cultural assessment. School Psychology International, 12, 119–132. Raudenbush, S. W., Bryk, A. S., & Congdon, R. (2004). HLM 6 for
canuse http://dx.doi.org/10.1177/0143034391121010 Windows[Computersoftware].Skokie,IL:ScientificSoftwareInterna-
erial CASEL.(2015).Statescanscorecardproject.Retrievedfromhttp://www tional.
mn
Aso .casel.org/state-scan-scorecard-project Ringwalt,S.(2008).Developmentalscreeningandassessmentinstruments
heper Denham, S. (2006). Social-emotional competence as support for school with an emphasis on social and emotional development for young
ythe readiness: What is it and how do we assess it? Early Education and children ages birth through five. Chapel Hill, NC: The University of
yrightedbolelyfort DueDseeednv1be7ul0ro1yp_,m4Le.,nCt,ali1n7,,S.5,7D–o8m9i.trohvtitcph:,//Cd.x,.&doWi.oerisgs/b1e0r.g1,2R0.7(/2s011555)6.6W9h3a5t SaNhboooolr,dthTT.CeJac.r,ho&nliincPaail,aFAnPtsaGs,isRCta.hnCilcd.e(DC2e0ev1ne2tel)or..pPmatetnetrnInssotiftustceh,oNoaltrioeandailnEeasrslyfoCrehcialds-t
ps doesevidence-basedinstructioninsocialandemotionallearningactu- achievementandsocioemotionaldevelopmentattheendofelementary
scoded ally look like in practice? Retrieved from http://www.casel.org/state- school. Child Development, 83, 282–299. http://dx.doi.org/10.1111/j
mentiinten Hasptpané,dFar.dGs-.fEo.r-(s1o9c9ia4l)-.aAndn-aedmvoatniocnedalt-elesatronfinthge/oryofmind:Understanding Sin.1g4u6la7r-8I6n2v4e.r2s0io1n1s..01(627080.5x). Facegen Main Software Development Kit
hisdocuarticleis ohDfaenvsdetiolcorayppmpceehdna,traaalcntdeDrsin’soortrmhdoeaulrgsch,htsi2l4da,rnedn12fa9ene–dl1in5agd4su.ltbhsyt.tpJa:ob/u/lderxn.aadulotoiis.fotirAcg,u/1tmi0sem.1n0taa0ln7ldy/ van(KVoWeurdisdiiojesnn,fEe3l..t1,()2.B0V.0Ma5n)..c,oTTurrvaenefrsf,learBtsiC,onP:.AaDnudt.h,codrreo.sBs-ecuurlst,urEa.l,aSdiaepbtealtiinokn,oBf.aMss.e,s&s-
Ts
hi BF02172093 ment instruments used in psychological research with children and
T
Kena,G.,Hussar,W.,McFarland,J.,deBrey,C.,Musu-Gillette,L.,Wang, families. Clinical Child and Family Psychology Review, 8, 135–147.
X., . . . Dunlop Velez, E. (2016). The Condition of Education 2016 http://dx.doi.org/10.1007/s10567-005-4752-1
(NCES 2016–144). Washington, DC: U.S. Department of Education,
NationalCenterforEducationStatistics.Retrievedfromhttp://nces.ed.
gov/pubsearch ReceivedAugust22,2016
Kline,R.B.(2005).Principlesandpracticeofstructuralequationmod- RevisionreceivedApril20,2017
eling(2nded.).NewYork,NY:GuilfordPress. AcceptedMay4,2017 (cid:2)