ERIC ED543664: Teachers' Knowledge about Early Reading: Effects on Students' Gains in Reading Achievement

2011·0.42 MB·English

by  ERIC

This article was downloaded by: [Joanne F. Carlisle]
On: 29 September 2011, At: 02:35
Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered
office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Journal of Research on Educational
Effectiveness
Publication details, including instructions for authors and
subscription information:
http://www.tandfonline.com/loi/uree20
Teachers’ Knowledge About Early
Reading: Effects on Students’ Gains in
Reading Achievement
Joanne F. Carlisle a , Ben Kelcey b , Brian Rowan a & Geoffrey Phelps
c
a University of Michigan, Ann Arbor, Michigan, USA
b Wayne State University, Detroit, Michigan, USA
c Educational Testing Service, Princeton, New Jersey, USA
Available online: 28 Sep 2011
To cite this article: Joanne F. Carlisle, Ben Kelcey, Brian Rowan & Geoffrey Phelps (2011): Teachers’
Knowledge About Early Reading: Effects on Students’ Gains in Reading Achievement, Journal of
Research on Educational Effectiveness, 4:4, 289-321
To link to this article: http://dx.doi.org/10.1080/19345747.2010.539297
PLEASE SCROLL DOWN FOR ARTICLE
Full terms and conditions of use: http://www.tandfonline.com/page/terms-and-conditions
This article may be used for research, teaching, and private study purposes. Any
substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,
systematic supply, or distribution in any form to anyone is expressly forbidden.
The publisher does not give any warranty express or implied or make any representation
that the contents will be complete or accurate or up to date. The accuracy of any
instructions, formulae, and drug doses should be independently verified with primary
sources. The publisher shall not be liable for any loss, actions, claims, proceedings,
demand, or costs or damages whatsoever or howsoever caused arising directly or
indirectly in connection with or arising out of the use of this material.
JournalofResearchonEducationalEffectiveness,4:289–321,2011
Copyright©Taylor&FrancisGroup,LLC
ISSN:1934-5747print/1934-5739online
DOI:10.1080/19345747.2010.539297
INTERVENTION,EVALUATION,ANDPOLICYSTUDIES
Teachers’ Knowledge About Early Reading:
Effects on Students’ Gains in Reading Achievement
JoanneF.Carlisle
1 UniversityofMichigan,AnnArbor,Michigan,USA
1
0
2
r BenKelcey
e
b WayneStateUniversity,Detroit,Michigan,USA
m
e
pt
e BrianRowan
S
9 UniversityofMichigan,AnnArbor,Michigan,USA
2
5
2:3 GeoffreyPhelps
0 EducationalTestingService,Princeton,NewJersey,USA
at
]
e
sl
rli
a
C Abstract: This study developed a new survey of teachers’ knowledge about early reading and
F. examinedtheeffectsofteachers’knowledgeonstudents’readingachievementinGrades1to3in
e
n alargesampleofMichiganschools.Usingstatisticalmodelsthatcontrolledforteachers’personal
n
a and professional characteristics, students’ prior reading achievement, and the clustering of high-
o
[J knowledge teachers in schools and school districts with particular demographic composition, we
y
b foundthattheeffectsofteachers’knowledgeaboutearlyreadingonstudents’readingachievement
ed weresmall.In1stgrade,studentsinclassroomsheadedbyhigherknowledgeteachersperformedbetter
d
a onyear-endtestsofreadingcomprehensionbutnotwordanalysis.In2ndand3rdgrades,theeffects
o
nl of teachers’ knowledge on either measure of students’ reading achievement were not statistically
w
significant.Althoughthestudysuggestsnewformsofstatisticalanalysisthatmightproducebetter
o
D estimatesoftheeffectsofteachers’knowledgeonstudents’readingachievement,furtherresearchis
neededtoimprovetheconceptualandpsychometricpropertiesofmeasuresofteachers’knowledge
ofreadingandtoinvestigatetherelationoftheirknowledgeandtheirinstructionalpractices.
Keywords: Teachers’knowledge,earlyreading,studentachievement
Researchshowsthataftercontrollingfordifferencesinstudents’previouslearningandhome
background,studentachievementvarieswidelyfromclassroomtoclassroomatthesame
gradelevelwithinaschool(e.g.,Scheerens&Bosker,1997).Mountingevidencesuggests
that at least part of this variation in student achievement results from stable “teacher
effects,” commonly defined as the fixed or random effects of specific teachers on their
students’achievementgainsacrossseveralyearsofobservation(Nye,Konstantopoulos,&
AddresscorrespondencetoJoanneF.Carlisle,UniversityofMichigan,SchoolofEducation,610
EastUniversityAvenue,AnnArbor,MI48109-1259,USA.E-mail:[email protected]
290 J.F.Carlisleetal.
Hedges,2004;Rivkin,Hanushek,&Kain,2005;Rowan,Correnti,&Miller,2002;Sanders
& Horn, 1994). Of current interest to education researchers is the extent to which these
teachereffectsonstudentachievementarisebecauseofvariationinteachers’pedagogical
andcontentknowledgeinthesubjectareatheyareteaching(e.g.,Hill,Ball,&Schilling,
2008).Thisquestionisofparticularimportancebecauseresearchshowsthatotherindices
ofteachers’professionalknowledge(e.g.,degreeattainment,certificationstatus)areonly
weakly related to student achievement (e.g., Croninger, Rice, Rathbun, & Nishio, 2003;
Wayne&Youngs,2003).
Thestudyreportedherecontributestothislineofworkbydiscussinganewmeasure
of teachers’ knowledge about early reading and by reporting on an empirical study that
used this measure to examine the effects of teachers’ knowledge on students’ reading
achievementinabout900first-throughthird-gradeclassroomsinMichigan.Aswediscuss
ingreaterdetailnext,thestudywasdesignedtoresolveanumberofuncertaintiesarising
1 from previous research on the effects of teachers’ knowledge on students’ early grades
1
0
2 readingachievement.Asweshallsee,theseuncertaintiesrevolvearoundhowtomeasure
r
e teachers’ knowledge for teaching early grades reading and how to estimate the effects of
b
m suchknowledgeonstudents’readingachievementinlightofvariousconfoundingfactors
e
pt inthematchingofstudentstoteacherswithinandbetweenschools.
e
S
9
2
5
3 THEPROBLEM
2:
0
at Formorethanadecade,researchershavearguedthattobeeffective,earlygradesreading
]
sle teachersneedarelativelyhighlevelofknowledgeabout“thelinguisticfoundations”ofearly
rli reading(Moats,2009b).Moats(1994,1999)developedanearlyandinfluentialapproachto
a
C measuringteachers’knowledgeinthisdomainknownastheInformalSurveyofLinguistic
e F. Knowledge.Thissurveyincludeditemsdesignedtomeasureteachers’contentknowledge
nn about the relations between the spoken and written aspects of language; about the sound
a
o structureofwords;andaboutrelatedtopicsingrammar,morphology,andorthography.A
J
y [ decade after Moats (1994) introduced this measure, many of the items from her original
b
d survey continue to be used in studies of teachers’ knowledge of early reading—and for
e
d good reason. As Piasta, Connor, Fishman, and Morrison (2009) noted in justifying their
a
o use of items from Moats’s survey, “Current theories of reading emphasize the necessity
nl
w of the alphabetic principle to link phonological, orthographic, and semantic knowledge,
o
D particularlyinthebeginningstagesofliteracy”(p.225).Thus,theyreasonedthatteachers’
knowledge of the alphabetic principle and of mappings between language and print was
essentialforeffectiveearlyreadinginstruction.
ResearchershaveuseditemsfromMoats’s(1994)surveytoaddressseveralinterrelated
researchquestionsabouttheteachingofearlygradesreading.Somestudieshaveexamined
theextenttowhichteachersactuallyknowaboutthelinguisticfoundationsofearlyreading;
othershaveinvestigatedwhetherspecificprofessionaldevelopmentprogramscanincrease
teachers’ knowledge in this domain; still others have asked whether increasing teachers’
linguisticknowledgeleadstomoreemphasisonexplicitinstructioninphonemicawareness,
phonics,orothercode-relatedaspectsofreading;andafewstudieshaveexaminedwhether
teachers with greater knowledge in this area have a more positive impact on students’
readingachievementthandoteacherswithlessknowledgeinthisarea.
Thefindingsfromthisbodyofresearchaddress(butleaveopen)anumberofimportant
questions about the nature of teachers’ knowledge about teaching early grades reading,
abouthowtomeasurethisconstruct,andaboutwhetherteachers’knowledgeinthisdomain
Teachers’KnowledgeAboutEarlyReading 291
isrelatedtoteachingeffectiveness,asmeasuredbygainsinstudents’readingachievement.
Forexample,severalstudieshaveshownthattheaverageteacherofearly-gradesreading
lacks strong knowledge about the linguistic foundations of reading (e.g., Bos, Mather,
Dickson, Podhajski, & Chard, 2001; McCutchen, Abbott, et al., 2002; Moats, 1994). In
addition, a growing body of evidence has shown that teachers can be taught linguistic
knowledge through programs of professional development (e.g., Bos, Mather, Narr, &
Babur,1999;Foorman&Moats,2004;Garetetal.,2008;McCutchen,Abbott,etal.,2002;
Spear-Swerling & Brucker, 2003, 2004). Additional evidence suggests that professional
developmentcanaffectteachingpractice,withresearchtendingtoshowthatteacherswho
participateinprofessionaldevelopmentaimedatincreasingknowledgeaboutthelinguistic
foundations of reading also provide students with more explicit instruction in phonemic
awareness,phonics,andothercode-relatedareasofreading(Bosetal.,1999;McCutchen,
Abbott,etal.,2002;Garetetal.,2008).
1 Whatislessclearfromresearchistheextenttowhichteachers’knowledgeaboutthe
1
0
2 linguisticfoundationsofreadinghasaneffectonstudents’readingachievementintheearly
r
e grades.Infact,theaccumulatedevidence,acrossdifferentstudies,suggeststhattheeffects
b
m on students’ reading achievement of teachers’ knowledge in this area might be limited
e
pt tocertain domains ofreading performance (e.g.,Moats,2009b). Positiveevidence ofthe
e
S effects of teachers’ knowledge on students’ reading achievement can be inferred from a
9
2 studybyBosetal.(1999),whichfoundthatstudentsofteacherswhoreceivedprofessional
5
3 developmentaimedatincreasingtheirknowledgeofthelinguisticfoundationsofreading
2:
0 showed greater achievement gains in some of these areas (e.g., phonemic awareness)
at than did students whose teachers did not receive this same professional development.
]
sle It is important to recognize, however, that this study estimated the effect of teachers’
rli participationinaprofessionaldevelopmentprogramonstudents’readingachievement,not
a
C theeffectoftheirknowledgeaboutreading.Thesameistrueofotherstudies.Forexample,
F. Garet et al. (2008) conducted a large randomized field trial of professional development
e
n program emphasizing (in part) the linguistic foundations of reading. These researchers
n
a
o foundthatteacherswhoparticipatedintheprogramscoredhigheronatestoftheir“code-
J
y [ related” knowledge of reading than did teachers who did not participate in the program,
b
d although the students of participating teachers did not show statistically greater gains in
e
d reading achievement compared to students of teachers who did not attend the program.
a
o Thisstudyalsodidnotexaminetheeffectsofteachers’knowledgeaboutreadingontheir
nl
w students’outcomes.
o
D Inaddition,studiesoftherelationshipbetweenteachers’knowledgeofthelinguistic
foundations of reading and students’ achievement have shown inconsistent results. For
example, in contrast to Bos et al. (1999), Spear-Swerling and Brucker (2004) found that
studentswhoweretutoredbyteacherswithhigherknowledgeofthelinguisticfoundations
ofreadingachievedhigherwordreadingscoresthandidstudentstutoredbyteacherswith
lower scores, but this effect did not occur on students’ test scores in the areas of letter-
soundcorrespondence,readingofirregularwords,orspelling.Inanotherstudy,McCutchen,
Harry,andcolleagues(2002)reportedpositivecorrelationsbetweenmeasuresofteachers’
linguisticknowledgeandkindergarteners’word-readingachievement,buttheseresearchers
did not find a relationship between teachers’ linguistic knowledge and first and second
graders’ achievement in the domains of vocabulary, reading comprehension, spelling, or
writingfluency.
Insummary,theresultsofthesevariousstudiespresentaquandary.Researchsuggests
thatearly-gradesreadingteachershavelimitedknowledgeofthelinguisticfoundationsof
readingandthatprofessionaldevelopmentcanincreaseteachers’knowledgeinthisdomain.
292 J.F.Carlisleetal.
The open question, however, is whether increasing teachers’ knowledge in this domain
will improve students’ reading achievement. The pattern of uneven (and modest) effects
previouslydescribedledFoormanandMoats(2004)tosuggesttheneedforfurtherresearch
into the measurement and effects of teachers’ knowledge for early grades reading—an
area of research that we also see as important. The study reported here was designed to
developanewmeasureofteachers’knowledgeofearly-gradesreadingandtodevelopan
empirical approach to estimating the causal effects of teachers’ knowledge on students’
reading achievement using nonexperimental data. In the sections that follow, we explain
ourapproachtoinvestigatingtheseissues.
APPROACHTOMEASUREMENTISSUES
1
1 Weconcludedfromourreviewofresearchonteachers’knowledgeaboutearly-gradesread-
0
2 ingthatasomewhatdifferentapproachtomeasuringteachers’knowledgewaswarranted.
r
be As discussed next, this entailed addressing three interrelated problems: (a) the domains
m
e ofknowledgetobeassessedinmeasuresofteachers’knowledgeofearly-gradesreading,
pt (b) the types of knowledge to be assessed within these domains, and (c) the resulting
e
S
9 psychometricpropertiesofanymeasureswedeveloped.
2
5
3
2:
0
at DomainsandTypesofTeacherKnowledge
]
e
sl Webeginbydiscussingthedomainsandtypesofknowledgetobeassessedinourstudy.
arli Ourreviewoftheliteraturesuggestedthatexistingmeasuresofteachers’knowledgeabout
C
F. earlyreadinghadtwomainproperties.First,mostreportedmeasuresfocusedonjustone
e of several domains of specialized knowledge that teachers might need in order to teach
n
n early-gradesreadingeffectively,namely,teachers’knowledgeofthelinguisticfoundations
a
o
J ofearlyreading.Inourview,itmakessensetoassumelinguisticknowledgeisanimportant
[
y component of teachers’ knowledge for teaching reading in the early grades. However,
b
d knowledge in this particular domain would seem to be relevant mainly to code-related
e
d
a instruction. There is good reason to focus on teachers’ knowledge beyond this limited
o
nl domain(Snow,Burns,&Griffin,1998).Forexample,mostbalancedreadingprogramsin
w
o the early grades recognize the need to build students’ oral language, not only to develop
D
phonemicawarenessanddecodingskillsbutalsotopromotevocabulary,fluencyinword
recognitionandtextprocessing,andreadingcomprehension(Pressleyetal.,2001;Snow,
Griffin,&Burns,2005).Forthisreason,wewouldarguethatreadingresearchersneedto
expandtheirmeasuresofteachers’knowledgeforreadinginstructionbeyondanexclusive
focusonlinguisticfoundations.1
Asecondproblemwithmostcurrentapproachestomeasuringreadingteachers’knowl-
edge is the focus on a particular type of knowledge: teachers’ content knowledge, de-
finedhereasknowledgeofaparticularacademicbodyofwork—inthiscase,linguistics.
1Notallmeasuresofteachers’knowledgeforteachingearly-gradesreadingreviewedinthisarticle
have focused only on the linguistic foundations of reading, although this is true of most of the
measuresusedinthepublicationscitedinourliteraturereview.Inparticular,itisworthnotingthat
themeasureofteachers’knowledgedevelopedbyGaretetal.(2008)wascarefully(andmoreorless
evenly)balancedacrossthefiveareasofreadingdiscussedinSnowetal.(1998).Themeasureused
byFoormanandMoats(2004)wasalsoreportedtofocusonallfiveoftheseareas.
Teachers’KnowledgeAboutEarlyReading 293
McCutchen, Harry, et al. (2002), for example, described the academic nature of items in
Moats’s (1994) original Informal Survey of Linguistic Knowledge when they noted that
“aperfectscoreonthe[Moats]surveyisdifficulttoachievewithoutconsiderablelinguis-
tic training” (p. 214). An important question for reading research, however, is whether
academic knowledge of this sort is the only form of knowledge needed to teach early-
gradesreadingeffectively.Tobesure,teachersneedcontentknowledgetoteacheffectively
(Shulman,1986),butthepossessionofacademicknowledgedoesnotassurethatteachers
willbeeffectiveinteachingtheirassignedsubjects.Inreading,forexample,teachersmight
be able answer a number of difficult questions about English phonology correctly and
stillnotknowhowtoeffectivelyteachchildrenwhoarehavingrealproblemsgraspingthe
conceptofphonemesinwords.AsShulman(1986,1987)pointedoutinhisseminaldiscus-
sionsofpedagogicalcontentknowledge,morethancontentknowledgeisneededtoteach
effectively.Snowetal.(2005)referredtothisknowledgeas“usable”knowledge,knowl-
1 edgethatis“embeddedinpractice”(p.11).Hill,Rowan,andBall(2005)referredtosuch
1
0
2 knowledge as a “specialized” form of content knowledge—that is, a deep understanding
r
e ofbothdisciplinaryknowledgeandwaysthatsuchknowledgecanberepresentedtofoster
b
m studentlearning.Fromthisperspective,wearguethatmeasuresofteachers’knowledgein
e
pt anyacademicdomainshouldassessnotonlyteachers’academicknowledgebutalsotheir
e
S understandingofhowthatknowledgemightbeusedeffectivelyinpractice.
9
2
5
3
2:
at 0 AnAlternativeApproachtoMeasurement
]
e
sl Theworkreportedinthisarticlebuildsontwoadditionalinsightsfromresearchconducted
arli by others. One comes from prior research showing that the knowledge of early-grades
C
F. readingteacherscanbemeasuredalongtwoprimarydimensions—knowledgerelevantto
e the teaching of word reading and knowledge relevant to the teaching of reading compre-
n
n hension. In particular, research conducted by Phelps and Schilling (2004) and by Garet
a
Jo etal.(2008)providedevidencethatknowledgeinthedomainsofwordreadingandread-
[
y ingcomprehensiondefinetwomeasurabledomainsofteachers’knowledgeaboutreading.
b
d Based on this insight, the work reported here aimed at developing a measure of reading
e
d teachers’ knowledge that included questions focused on both word reading and reading
a
o
nl comprehension.2
w
Another key insight comes from efforts to measure teachers’ knowledge in fields of
o
D
researchotherthanreading.Inparticular,thelineofworkonmathematicsteachers’knowl-
edgeconductedbyHeatherHillandcolleagues(e.g.,Hill&Ball,2004;Hill,Schilling,&
Ball,2004)demonstratestwoimportantpointsthatweattemptedtobuildoninthework
reportedhere.Theirworkshowsthatinadditiontomeasuringteachers’academiccontent
knowledge,itispossibletomeasureteachers’knowledgeofpedagogyandstudentlearning
in specific areas of the school curriculum. Further, it is possible to develop assessment
itemsthatsituateteachers’knowledgeforteachingininstructionalcontexts.Thus,inour
study, rather than asking teachers how many phonemes are in certain words (as is done
inmanystudiesofteachers’readingknowledge),wedevelopeditemsthataskedteachers
2Otherresearchershaveproposedmeasuringadditionalaspectsofteachers’knowledge.Forex-
ample,PalincsarandDuke(2004)arguedthatknowledgeoftextsandgenresisessentialforteachers
ofreading,andCunningham,Perry,Stanovich,andStanovich(2004)includednotonlyameasureof
phonologyandphonicsbutalsoameasureofteachers’knowledgeofchildren’sliterature,although
thelatterdidnotaccountforstudents’readingperformance.
294 J.F.Carlisleetal.
to determine whether a student’s spelling errors indicate difficulty identifying sounds in
words.Itemsofthissortsituateteachers’contentknowledgeininstructionalcontexts.
Finally,ourcurrentapproachgrewoutofourpreviousresearchonreadingteachers’
knowledgeandreflectsanevolutioninourmeasurementefforts.Inaninitialstudyofthe
connection between reading teachers’ knowledge for teaching and student achievement
(Carlisle et al., 2009), we developed a measure to assess teacher’s knowledge of the lin-
guistic foundations of reading as disseminated at professional development seminars on
Moats’s (2003) Language Essentials for Teachers of Reading and Spelling that were at-
tendedbyteachersinReadingFirstschoolsinMichigan.Thatmeasure,whichwecalled
Language and Reading Concepts, included twice as many items focused on phonology,
phonics,andgrammarasitdiditemsfocusedonreadingcomprehensionandvocabulary.
Moreover, like many teacher assessments in the field of reading, the items in this initial
measure assessed academic content knowledge (e.g., “Which of the following words has
1 a prefix?”) without situating that knowledge in instructional contexts. In this study, we
1
0
2 estimatedtheeffectofteachers’knowledgeofreadingonstudents’readingachievement,
r
e controlling for the sociodemographic characteristics of students in a classroom and for
b
m severalcharacteristicsofteachers’professionalpreparationforteaching(e.g.,certification
e
pt status,educationalattainment).Thestudentoutcomesweretheperformancesoffirst,sec-
e
S ond, and third graders on two subtests of the Iowa Tests of Basic Skills (ITBS): word
9
2 analysisandreadingcomprehension.Theresultsshowednostatisticallysignificanteffects
5
3 ofteachers’knowledgemeasuredinthiswayonstudents’covariateadjustedachievement
2:
0 inwordanalysisorreadingcomprehension.
at Ininterpretingtheresultsofthisfirststudy,wehypothesizedthatthelackofstatistically
]
sle significanteffectsofourmeasureonstudents’readingachievementmightbeattributable
rli (in part) to the approach we had taken to measuring teachers’ knowledge, in particular,
a
C thefocusinourmeasureonassessingteachers’decontextualized,academiccontentknowl-
F. edgeandtheoverrepresentationofitemsassessingteachers’knowledgeofthealphabetic
e
n code and aspects of teaching word reading. As a result, in a subsequent study, we devel-
n
a
o opedameasureofteachers’knowledgethatdifferedfromthisinitialmeasureinthreeways
J
y [ (Carlisle et al., 2008). First, we included items designed to situate teachers’ knowledge
b
d in classroom practices. Second, the new measure had a better balance of items focused
e
d on word reading and comprehension. Finally, the measure was based not on the contents
a
o of a particular program of professional development but rather on experts’ judgments of
nl
w theknowledgethatteachersneededtoteachbeginningreadingeffectively.Usingpropen-
o
D sity score matching (Rosenbaum & Rubin, 1983) to identify and contrast comparable
classrooms and teachers, we estimated the effects of this new measure on classroom-to-
classroom variation in students’ reading achievement. The results of this second study
indicated the presence of a small, positive effect on students’ ITBS reading achievement
scoresinGrades1and2(withstandardizedregressioncoefficientsofb=.05),butnoeffect
ofteachers’knowledgeonthismeasureofstudents’readingachievementinGrade3.This
result suggested that the measure of teachers’ knowledge that we developed emphasized
knowledge likely to have relevance for teaching reading effectively in first and second
gradesbutnotinthirdgrade.
THECURRENTSTUDY
Thepresentstudywasdesignedtoaddresswhatwesawastwoshortcomingsinthestudies
that we (and others) have carried out. First, we again revised our measure of teachers’
knowledge for early-grades reading so that items focused less on measuring teachers’
Teachers’KnowledgeAboutEarlyReading 295
academic content knowledge and more on teachers’ use of their content knowledge to
make decisions about instruction or to analyze students’ performance on reading/writing
tasks. Second, because teachers with extensive knowledge about reading might not be
distributedequallyacrossschools,wealsoadjustedthepropensityscoremethodsweused
to statistically control for the clustering of high-knowledge teachers in certain schools
and to control for the potential influence that schools might have on students’ academic
achievement.Wethenusedthisanalyticstrategytoestimatetheeffectofteachers’reading
knowledgeonstudents’readingachievement.
Thecurrentstudyhastworesearchquestions:Whatisthereliabilityanddimensionality
ofthemeasureofteachers’knowledgethatwedeveloped(Teachers’KnowledgeofReading
andReadingPractices,orTKRRP)?Towhatextentdoesteachers’knowledgeaboutreading,
asdemonstratedonthismeasure,affectstudents’gainsinreadingachievementoveraschool
year?Ourfirstquestionfocusedonthepsychometriccharacteristicsofournewlydeveloped
1 testofteachers’knowledge.Wesawthisasacriticalfirststepinourresearch,especially
1
0
2 because so few prior studies examined the psychometric properties of the measures they
r
e used, a problem that could affect study outcomes. The second step involved developing
b
m an empirical strategy to estimate the effects of teachers’ knowledge on students’ reading
e
pt achievement more directly than has been done in experimental studies of professional
e
S developmentprogramswhileaddressingthecomplexissuesofcausalinferencethatarise
9
2 innonexperimentalstudies.
5
3 With respect to this second problem, several issues are critical. The first is that in
2:
0 Americanschools,studentswhofacethegreatestchallengesinlearningtoread(i.e.,poor
at and minority students with lower levels of entry-level achievement) are also taught by
]
sle theleastqualifiedteachers(Darling-Hammond,2004).Inthissituation,wecanexpectto
rli find that teachers’ knowledge is related to students’ achievement simply because more
a
C knowledgeable teachers are clustered within schools that serve students who generally
F. make larger gains in reading achievement. This clustering of particular types of teachers
e
n and students in particular types of schools motivated our use of a multilevel approach to
n
a
o propensityscorematching.Thisapproachisdiscussedinmoredetailnext.
J
y [ Asecondissueconcernshowtoestimatetheeffectofteachers’knowledgeinlightof
b
d findings that teachers’ knowledge affects their instructional practices in ways that could
e
d improvestudents’readingachievement(e.g.,Bosetal.,2001).Ourapproachtoestimatinga
a
o teacherknowledgeeffectinlightofthisendogenousprocessistocarefullymatchteachers
nl
w on a large number of student, classroom, and school covariates known from previous
o
D researchtoaffectstudents’readingachievementbutnottomatchteachersintermsoftheir
instruction.
Afinalissuearisesbecauseouranalyticmethodsrelyonobservational(i.e.,nonexper-
imental)data.Inthissituation,itcanalwaysbearguedthatourestimatedeffectofteachers’
knowledgeonstudents’achievementissubjecttoomittedvariablesbias.Forthisreason,we
examinedtherobustnessofourcausalinferencesabouttheeffectsofteachers’knowledge
onstudents’achievementbyconductingasensitivityanalysis.Thisanalysisaddressedthe
question of the extent to which our estimates of the effects of teachers’ knowledge on
students’achievementmightbealteredinlightofanyfailuretoincludeparticularkindsof
unmeasuredvariablesinourstatisticalmodel.
SAMPLE,DATASOURCES,ANDMEASURES
The current study examines these issues by studying a sample of teachers who worked
in Reading First schools in Michigan during the 2006–2007 school year. Reading First
296 J.F.Carlisleetal.
(Part B of No Child Left Behind Act of 2001) was specifically designed to improve the
readingachievementofkindergartenthroughthird-gradestudentsinhigh-povertyschools
withchronicunderachievementinreading(U.S.DepartmentofEducation,2002).
ResearchSample
Theparticipantsinthisstudywerevolunteersfromthepopulationofteachersworkingin
Michigan’s Reading First schools during the 2006–2007 school year.3 About 72% of the
Grade1throughGrade3teachersintheseschoolsvolunteeredtoallowresearcherstouse
surveyresultsinresearchstudies.Collectively,the1,101volunteerteacherstaughtin138
1 schoolsandinstructed16,439students.Oftheteacherswhoagreedtoparticipate,297first-
01 grade,275second-grade,and292third-gradeteachershadsufficientstudentachievement
2
r data to be included in the study. Although we were unable to conduct this study with
e
b the full population of Michigan Reading First teachers, we did have available data from
m
e both the population of Reading First teachers and the research sample. This allowed us
pt
e to compare the characteristics of the two groups and determine the extent to which the
S
9 volunteer sample differed from the larger population of teachers. On nearly all measures
2
5 we used to assess differences across the groups, the two groups showed no statistically
3
2: significant differences (see Tables 1 & 2). An exception, however, was that the research
0
at samplehadahigheraveragescore(of+.25SD)onourmeasureofteachers’knowledge.
] Table1presentsdemographicinformationonthestudentstaughtbyteachersincludedinthe
e
rlisl researchsampleandstudentstaughtbyteachersinthelargerpopulation.Table2presents
a informationabouttheteachersintheresearchsampleandinthislargerpopulation.
C
F.
e
n
n
a
o
J
y [ SourcesofData
b
d
e
d Two types of student achievement data were used in this study: a classroom reading as-
a
o sessmentthatwasusedasapretestmeasureofstudents’achievementandastandardized
nl
w achievement test that was used as both a pre- and a posttest measure of achievement.
o
D We also included measures of students’ sociodemographic characteristics in our statis-
tical models. Data on students came from Michigan’s Single Record Student Database
(http://www.michigan.gov/cepi). Data on teachers included teacher scores on our mea-
sureofteachers’knowledgeanddataonteachers’professionalandpersonalbackground.
Thesedatawerecollectedfromasurveyinstrumentcalled“Teacher’sQuest”administered
threetimesayeartoReadingFirstteachersinMichigan.Finally,schoolanddistrictdemo-
graphicandorganizationaldataweregatheredfromtheMichiganDepartmentofEducation
website(http://www.michigan.gov/mde).Thesesourcesofdataaredescribednext.
3ToqualifyforReadingFirstfundinginMichigan,districtshadtomeeteligibilityrequirementsof
lowreadingachievement(i.e.,40%ormoreoffourth-gradestudentsscoringbelowtheproficiency
cutpointonthestateassessment,MichiganEvaluationofAcademicPerformance,Reading)for2of
thepreceding3yearsandlowincome(e.g.,1,000ormorestudentsfromfamiliesbelowthepoverty
line).
fe of
pl s
m st
Sa 131407731439003801011351077959 Te
ch .0.0.0.0.0.0.0.0.0.0.63.0.09.79.77 wa
3 ear 111 Io
s d
ade Re an
Gr ge,
a
en s,
ulatio .013.012.006.074.012.036.000.420.001.000.6216.051.0912.7908.7750 score
op 111 S)
P L
E
B
DI
de (
1 pl s
01 am 145212081162415 kill
2 S 110714030065989 S
mber 2 earch .0.0.0.0.0.0.0.0.0.0.38.0.96.167.163 eracy
pte de Res Lit
9 Se Gra Early
d by [Joanne F. Carlisle] at 02:35 2 andpopulation Grade1 bcResearchSamplePopulation ..010011..011013..004005..071075..012011..040036..000000..039043..001001..001001..23733690..050051..84679704..1498516659..1491416286 ptionofDynamicIndicatorsofBasic=y;ORFOralReadingFluency.=fN8,719.5,488.
Downloade earchsample aPopulation .009.012.004.073.011.036.000.042.001.001.2256.051.8456.14958.14872 withtheexceWordFluenc=eN5,231.
ntsintheres RFscore proportions=Nonsense=dN8,904.
Table1.Characteristicsofstude DisabilityLimitedEnglishproficiencySpecialeducationFreeorreducedlunchHispanicWhiteHawaiianAfricanAmericanAsianAmericanIndianDIBELSfallgrade1/2/3NWF/OMaleAverageage(inmonths)ITBS–ReadingcomprehensionITBS–Wordanalysis Note.CharacteristicsrepresentBasicSkills(ITBS)scores.NWF===abcNNN9,187.5,720.
297