ERIC ED504703: The Accountability Illusion: New Mexico

2009·0.26 MB·English

by  ERIC

New Mexico
Executive Summary in most other states, meaning it holds more sub- N
groupsaccountableforperformance.2 e
TheintentoftheNoChildLeftBehind(NCLB)Actof w
2001istoholdschoolsaccountableforensuringthatall (cid:1)Thesmallernsizeappearstobeafactorinthenumber M
of their students achieve mastery in reading and math, of schools making AYP in New Mexico, despite the
e
with a particular focus on groups that have traditionally state’slowoverallcutscoresinreadingandlowannual x
beenleftbehind.UnderNCLB,statessubmitaccounta- proficiencytargetsinmathandreading(e.g.,thestate i
c
bilityplanstotheU.S.DepartmentofEducationdetailing demands that only 35% of students in grades six o
therulesandpoliciestobeusedintrackingtheadequate througheightreachmathproficiencyin2008).
yearlyprogress(AYP)ofschoolstowardthesegoals.
(cid:1) Looking across the 28 state accountability systems
examined in the study, we find that the number of
ThisreportexaminesNewMexico’sNCLBaccountabil-
elementary schools making AYP in New Mexico is
ity system—particularly how its various rules, criteria,
exceededin12othersamplestates(NewMexicoties
andpracticesresultinschoolseithermakingAYPornot
with New Hampshire and Maine, each with 4 ele-
makingAYP.ItalsogaugeshowtoughNewMexico’ssys-
mentary schools making AYP). New Mexico is one
temiscomparedwithotherstates.Forthisstudy,wese-
of 10 states with 2 middle schools each that made
lected36schoolsfromvariousstatesaroundthenation,
AYPinthesample(seeFigure1).
schools that vary by size, achievement, and diversity,
among other factors, and determined whether each
wouldmakeAYPunderNewMexico’ssystemaswellas
underthesystemsof27otherstates.Weusedschooldata
andproficiencycutscore1estimatesfromacademicyear
2005–2006, but applied them against New Mexico’s TherearesomeinterestingdynamicsthatplaceNew
AYP rules for academic year 2007–2008 (shortened to
Mexiconearthemiddleofthestatedistributionin
“2008”inthisreport).
termsofthenumberofschoolsmakingAYP.Thisisa
statewhichhasseveralrigorousrequirements
Herearesomekeyfindings:
combinedwithmorelenientones.Forexample,New
(cid:1)Weestimatethat14of18elementaryschoolsand Mexico’scutscoresinmathareclosetoorabovethe
16 of 18 middle schools in our sample failed to 50thpercentile,whilereadingcutscoresmostly
makeAYPin2008underNewMexico’saccounta- hoveraroundthe30thpercentile.Somorerigorin
bility system. This high failure rate is partly ex-
mathiscoupledwithlessrigorinreading.New
plainedbyoursample,whichintentionallyincludes
Mexico’s99percentconfidenceintervalprovides
some schools with relatively large populations of
schoolswithgreaterleniencythanthemore
low-performing students. But it’s also partly ex-
commonlyused95percentconfidenceintervalfound
plained by New Mexico’s minimum n size for sub-
groups, which tends to be smaller than those used inotherstates.However,NewMexico’sminimum
subgroupsizeis25,whichissmallerthanmostother
statesweexamined.ThismeansthatschoolsinNew
1 A cut score is the minimum score a student must receive on
Mexicowillhavemoreaccountablesubgroupsthan
NWEA’sMeasuresofAcademicProgress(MAP)thatisequivalentto
performingproficientontheNewMexicoStandardsBasedAssess- wouldsimilarschoolsinotherstates,makingit
ments.
difficultforlargeschoolswithmanyaccountable
2Keepinmind,however,thatschoolsizeandnsizearerelated(e.g.,
smallnsizesmakesenseforsmallschools). subgroupstomakeAYPthere.
1 THOMASB.FORDHAMINSTITUTE
18
o 16
c P
Y
i A 14
x ng
ki
e Ma 12
M ols
o 10
h
w Sc
ple 8
e m
a
N ofS 6
er
mb 4
u
N
2
0
Massachuse s Nevada Idaho NorthDakota Kansas Washington Wyoming Indiana SouthCarolina Montana Florida Vermont NewJersey NewHampshire Maine NewMexico Delaware Colorado RhodeIsland Georgia Illinois Ohio Minnesota Michigan California Texas Arizona Wisconsin
Elementary Schools Middle Schools
Figure1.NumberofsampleschoolsmakingAYPbystate
Note:MiddleschoolswerenotincludedforTexasandNewJersey;absenceofamiddleschoolbarinthosestatesmeans“notapplicable”asopposedtozero.Stateslike
IdahoandNorthDakota,however,havezeropassingmiddleschools.
(cid:1)Nearlyalloftheschoolsinoursamplethatfailedto performance is lower. In other words, schools with
makeAYPinNewMexicoaremeetingexpectedtar- greater diversity and size face greater challenges in
getsfortheiroverallpopulations3butfailedbecause makingAYP.Thisisthecaseinotherstatesaswell.
oftheperformanceofindividualsubgroups,partic-
ularlystudentswithdisabilities(SWDs)andEnglish (cid:1)Astrongpredictorofwhetherornotaschoolmakes
AYP under New Mexico’s system is whether it has
languagelearners.
enoughEnglishlanguagelearnerstoqualifyasasep-
(cid:1) As in other states, middle schools in New Mexico arate subgroup. Every single school with a limited
hadgreaterdifficultyreachingAYPthandidelemen- English proficient (LEP)4 subgroup failed to make
tary schools, primarily because their student popu- AYP.Likewise,mostoftheschools(especiallyatthe
lationsarelargerandthereforehavemorequalifying middleschoollevel)withenoughqualifyingSWDs
subgroups—notbecausetheirstudentachievement failedtomeettheirAYPtargets.5
islowerthanintheelementaryschools.
Introduction
(cid:1) Middle schools with fewer subgroups attained AYP
moreeasilyinNewMexicothanmiddleschoolswith TheProficiencyIllusion(Croninetal.2007a)linkedstu-
more subgroups, even when their average student dentperformanceonNewMexico’stestsandthoseof25
3It’simportanttonotethatstudentsinsubgroupsnotmeetingtheminimumnsizesarestillincludedforaccountabilitypurposesintheoverall
studentcalculations;theyaresimplynottreatedastheirownsubgroup.
4Notethatweuse“LEPstudents”and“Englishlanguagelearners”interchangeablytorefertostudentsinthesamesubgroup.
5SWDsaredefinedasthosestudentsfollowingindividualizededucationplans.WeshouldalsonotethatoursubgroupfindingsforLEPstudents
andSWDsmaybemorenegativethanactualfindings,mostlybecauseofthelikelydifferencesbetweenhowLEPstudentsandSWDsaretreated
inMAP,theassessmentweusedinthisstudy,andintheNewMexicoStandardsBasedAssessments,thestandardizedstatetest.Specifically,theU.S.
DepartmentofEducationhasissuednewNCLBguidelinesinrecentyearsthatexcludesmallpercentagesofLEPstudentsandSWDsfromtaking
thestatetestorthatallowthemtotakealternativeassessments.Inthisstudy,however,novalidMAPscoreswereomittedfromconsideration.
TheAccountabilityIllusion 2
other states to the Northwest Evaluation Association’s States also determine the minimum number of students
(NWEA’s) Measures of Academic Progress (MAP), a thatmustconstituteasubgroupinorderforitsscorestobe
computerized adaptive test used in schools nationwide. analyzedseparately(alsocalledtheminimumn[numberof N
e
Thissinglecommonscalepermittedcross-statecompar- students in sample] size).The rationale is that reporting
w
isons of each state’s reading and math proficiency stan- theresultsofverysmallsubgroups—fewerthan10pupils,
dardstomeasureschoolperformanceundertheNoChild for example—could jeopardize students’ confidentiality M
Left Behind (NCLB) Act of 2001.That study revealed and risk presenting inaccurate results. (With such small e
x
profounddifferencesinstates’proficiencystandards(i.e., groups,randomevents,likeonestudentbeingoutsickon
i
howdifficultitistoachieveproficiencyonthestatetest), testday,couldskewtheoutcome.)Becauseofthisflexibil- c
o
andevenacrossgradeswithinasinglestate.
ity,stateshavesetwidelyvaryingnsizesfortheirsubgroups,
fromasfewas10youngsterstoasmanyas100.
Our study expandsonTheProficiencyIllusion by exam-
ining other key factors of state NCLB accountability Manystateshavealsoadoptedconfidenceintervals—ba-
plansandhowtheyinteractwithstateproficiencystan- sicallymarginsofstatisticalerror—totrytoaccountfor
dards to determine whether the schools in our sample potential measurement error within the state test. In
made adequate yearly progress (AYP) in 2008. Specifi- somestates,thesemarginsarequitewide,whichhasthe
cally, we estimated how a single set of schools, drawn effectofmakingiteasiertoachieveanannualtarget.
fromaroundthecountry,wouldfareunderthediffering
rulesfordeterminingAYPin28states(theoriginal25in AlloftheseAYPrulesvarybystate,whichmeansthata
The Proficiency Illusion plus 3 others for which we now schoolthatmakesAYPinWisconsinorOhio,forexam-
have cut score estimates). In other words, if we could ple,mightnotmakeitunderSouthCarolina’sorIdaho’s
somehow move these entire schools—with their same rules(U.S.DepartmentofEducation2008).
mix of characteristics—from state to state, how would
they fare in terms of making AYP? Will schools with What We Studied
high-performing students consistently make AYP? Will
Wecollectedstudents’MAPtestscoresfromthe2005–
schoolswithlow-performingstudentsconsistentlyfailto
2006academicyearfrom18elementaryand18middle
make AYP? If AYP determinations for schools are not
schoolsaroundthecountry.WealsocollectedtheNCLB
consistentacrossstates,whatleadstotheinconsistencies?
subgroupdesignationsforallstudentsinthoseschools—
inotherwords,whethertheyhadbeenclassifiedasmem-
NCLBrequireseverystate,asaconditionofreceiving
bersofaminoritygrouporasEnglishlanguagelearners,
TitleIfunding,toimplementanaccountabilitysystem
amongothersubgroups.
that aims to get 100% of its students to the proficient
levelonthestatetestbyacademicyear2013–2014.In
Theschoolswerenotselectedasarepresentativesample
theinterveningyears,statessetannualmeasurableob-
of the nation’s population. Instead, we selected the
jectives (AMOs).This is the percentage of students in
schoolsbecausetheyexhibitedarangeofcharacteristics
each school, and in each subgroup within the school on measures such as academic performance, academic
(suchaslowincome6orAfricanAmerican,amongoth- growth, and socioeconomic status (the latter calculated
ers), that must reach the proficient level in order for bythepercentageofstudentsreceivingfreeorreduced-
the school to make AYP in a given year. The AMOs pricelunches).Appendix1containsacompletediscus-
varybystate(asdo,ofcourse,thedifficultyofthepro- sionofthemethodologyforthisprojectalongwiththe
ficiencystandards). characteristicsoftheschoolsample.7
6Low-incomestudentsarethosewhoreceiveafreeorreduced-pricelunch.
7Wegaveallschoolsinoursamplepseudonymsinthisreport.
3 THOMASB.FORDHAMINSTITUTE
70
o 60
c
xi ng 50
e nki
a 4400
M RR
e Reading
w en l 30 Math
c
e Per 20
N
10
0
Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8
Figure2.NewMexicoreadingandmathcutscoreestimates,expressedaspercentileranks(2006)
Note:ThisfigureillustratesthedifficultyofNewMexico’scutscores(orproficiencypassingscores)foritsreadingandmathtests,aspercentilesoftheNWEAnorm,in
gradesthreethrougheight.Higherpercentileranksaremoredifficulttoachieve.AllofNewMexico’scutscoresinreadingarebelowthe50thpercentile,butthecut
scoresinmathareclosetoorabovethe50thpercentile.
Table1.NewMexicoAYPrulesfor2008
Subgroupminimumn Race/ethnicity:25
SWDs:25
Low-incomestudents:25
LEPstudents:25
CI Appliedtoproficiencyratecalcula-ons?
Yes;99%CIused
AMOs Baselineproficiencylevelsasof2002(%) 2008targets(%)
READING/LANGUAGEARTS
Grade3 n/a 59
Grade4 30 59
Grade5 n/a 59
Grade6 n/a 53
Grade7 n/a 53
Grade8 39 53
MATH
Grade3 n/a 44
Grade4 35 44
Grade5 n/a 44
Grade6 n/a 35
Grade7 n/a 35
Grade8 33 35
Sources:U.S.DepartmentofEducation(2008);CouncilofChiefStateSchoolOfficers(2008).
Abbreviations:SWDs=studentswithdisabilities;LEP=limitedEnglishproficiency;CI=confidenceinterval;AMOs=annualmeasurableobjectives;n/a=notapplicable
TheAccountabilityIllusion 4
18 15
ol
16 o
h N
10 c
S
14 n e
ets 12 5 cei w
g n
r a
erofTa 180 0 erform Me
b P
mm 6 -5 tnt x
Nu de i
4 u c
-10 St o
2 ge
a
r
0 -15 ve
A
1) 1) 1) 7) 5) 3) 7) 7) 5) 9) 1) 2) 3) 7) 8) 0) 8) 4)
( ( ( ( ( ( ( ( ( ( 2 2 ( ( ( 1 2 1
Clarkson Maryweather Few Nemo IslandGrove ohnF.Kennedy Scholls Hissmore WolfCreek Mayberry ayneFineArts( Winchester( Coastal Paramount ForestLake Marigold( Roosevelt( KingRichard(
J W
TargetsPassed TargetsFailed Average Student Performance
Figure3.AYPperformanceoftheelementaryschoolsampleunderNewMexico’s2008AYPrules
Note:ThisfigureindicateshoweachoftheelementaryschoolswithinthesamplefaredunderNewMexico’sAYPrules(asdescribedinTable1).Thebarsshowthe
numberoftargetsthateachschoolhastomeetinordertomakeAYPunderthestate’sNCLBrules,andwhethertheymetthem(darkblue)ordidnotmeetthem(light
blue).Themoresubgroupsinaschool,themoretargetsitmustmeet.Underthestudyconditions,aschoolthatfailedtomeettheAMOsforevenasinglesubgroupdidn’t
makeAYP,soanylightbluemeansthattheschoolfailed.ForestLake,forexample,met7ofits8targets,butbecauseitdidn’tmeetthemall,itdidn’tmakeAYP.Schools
areorderedfromlowesttohighestaveragestudentperformance(shownbytheorangetriangles),whichismeasuredbytheaverageMAPperformanceofstudents
withintheschool;itsscaleisshownontherightsideofthefigure.Scoresbelowzero(whichisthegradelevelmedian)denotebelow-grade-levelperformanceand
scoresabovezerodenoteabove-grade-levelperformance.Oneunitdoesnotequalagradelevel;however,thehigherthenumber,thebettertheaverageperformance
andthelowerthenumber,theworsetheaverageperformance.Thenumberinparenthesesaftereachschoolnameindicatesthenumberofstates(outof28)inwhich
thatschoolwouldhavemadeAYP.
ProficiencycutscoreestimatesfortheNewMexicoStan- weappliedtoelementaryandmiddleschoolsinthecur-
dardsBasedAssessments(NMSBA)aretakenfromThe rentstudy.NewMexico’sminimumsubgroupsizeis25,
ProficiencyIllusion(asshowninFigure2),whichfound which is smaller than most other states we examined.
that New Mexico’s definitions of proficiency generally ThismeansthatschoolsinNewMexicowillhavemore
ranked below average compared with the standards set accountable subgroups than would similar schools in
bytheother25statesinthatstudy.Thesecutscoreswere otherstates.
usedtoestimatewhetherstudentswouldhavescoredas
proficientorbetterontheNewMexicotest,giventheir Further, although most states also apply confidence in-
performance on MAP. Student test data and subgroup tervals (or margins of statistical error) to their measure-
designationswerethenusedtodeterminehowthese18 ments of student proficiency rates, New Mexico’s 99%
elementary and 18 middle schools would have fared confidenceintervalgivesschoolsgreaterleniencythanthe
underNewMexicoAYPrulesfor2008.Inotherwords, more commonly used 95% confidence interval. So, for
the school data and our proficiency cut score estimates instance, although schools are supposed to get 59% of
arefromacademicyear2005–2006,butweareapplying their grade 3 students (and 59% of grade 3 students in
themagainstNewMexico’s2008AYPrules. eachsubgroup)totheproficientlevelonthestatereading
test,applyingtheconfidenceintervalmeansthatthereal
Table1showsthepertinentNewMexicoAYPrulesthat targetcanbelower,particularlywithsmallergroups.8
8Wealsoconductedananalysistoshowtheeffectofconfidenceintervalsonthereadingandmathproficiencyratesforelementaryandmiddle
schools.Wedescribethoseresultslaterinthereport.
5 THOMASB.FORDHAMINSTITUTE
20 12
ol
18 10 o
o h
c
c 16 8 S
n
xi ets 14 6 cei
g n
e Tar 12 4 ma
M of 10 2 or
w mberm 8 0 ntPerft
u 6 -2 e
e N d
u
N 4 -4 St
e
2 -6 g
a
r
0 -8 ve
A
0) 0) 0) 5) 0) 0) 1) 1) 0) 0) 2) 0) 2) 1) 2) 0) 3) 5)
( ( ( 1 ( ( ( ( ( ( ( ( ( ( ( 2 ( (
McBeal arringerCharter MLAndrew Pogesto( McCord Tigerbear Chesterfield Filmore Barban Kekata Hoyt BlackLake LakeJoseph Zeus OceanView WalterJones( Artemus Chaucer
B
Targets Passed Targets Failed Average Student Performance
Figure4.AYPperformanceofthemiddleschoolsampleunderNewMexico’s2008AYPrules
Note:ThisfigureshowshoweachofthemiddleschoolswithinthesamplefaredunderNewMexico’sAYPrules(asdescribedinTable1).Thebarsshowthenumberoftargets
thateachschoolhadtomeetinordertomakeAYPunderthestate’sNCLBrules,andwhethertheymetthem(darkblue)ordidnotmeetthem(lightblue).Themoresubgroups
inaschool,themoretargetsitmustmeet.Underthestudyconditions,aschoolthatfailedtomeettheAMOsforevenasinglesubgroupdidnotmakeAYP,soanylightblue
meansthattheschoolfailed.Artemus,forexample,met10ofits12targets,butbecauseitdidn’tmeetthemall,itdidn’tmakeAYP.Schoolsareorderedfromlowesttohighest
averagestudentperformance(shownbytheorangetriangles).ThisismeasuredbytheaverageMAPperformanceofstudentswithintheschool,anditsscaleisshownon
therightsideofthefigure.Scoresbelowzero(whichisthegradelevelmedian)denotebelow-grade-levelperformanceandscoresabovezerodenoteabove-grade-level
performance.Oneunitdoesnotequalagradelevel;however,thehigherthenumber,thebettertheaverageperformanceandthelowerthenumber,theworsetheaverage
performance.Thenumberinparenthesesaftereachschoolnameindicatesthenumberofstates(outof28)inwhichthatschoolwouldhavemadeAYP.
1
0.9
0.8
0.7
e
Rat 0.6
y
nc 0.5
e
cifi 00.44
o
r
P 0.3
0.2
0.1
0
Clarkson Maryweather Few Nemo IslandGrove hnF.Kennedy Scholls Hissmore WolfCreek Mayberry ayneFineArts Winchester Coastal Paramount ForestLake Marigold Roosevelt KingRichard
o W
J
Math Proficiency Rate MathProficiency RatewithCI MathTarget
Figure5.ImpactoftheconfidenceintervalonelementaryschoolmathproficiencyratesunderNewMexico’s2008AYPrules
Note:Thisfigureshowsthereportedproficiencyrateforthestudentpopulationasawholeandtheimpactoftheconfidenceintervalonmeetingannualtargets.The
darkerportionsofthebarsshowtheactualproficiencyrateachieved,whilethelighter(upper)portionsofthebarsshowthemarginoferrorascomputedbythe
confidenceinterval.Thefigureshowsthatoneofthesampleelementaryschools(Maryweather)wasassistedbytheconfidenceinterval.Annualtargets(theorange
lines)areconsideredtobemetbytheconfidenceintervaliftheyfallwithinthelightblueportion.
TheAccountabilityIllusion 6
1
0.9
N
0.8
e
0.7 w
e
Rat 0.6
y M
nc 0.5
e e
iciof 00.44 x
Pr 0.3 i
c
0.2 o
0.1
0
McBeal ngerCharter MLAndrew Pogesto McCord Tigerbear Chesterfield Filmore Barban Kekata Hoyt BlackLake LakeJoseph Zeus OceanView WalterJones Artemus Chaucer
arri
B
Math ProficiencyRate MathProficiency RatewithCI MathTarget
Figure6.ImpactoftheconfidenceintervalonmiddleschoolmathproficiencyratesunderNewMexico’s2008AYPrules
Note:Thisfigureshowsthereportedproficiencyrateforthestudentpopulationasawholeandtheimpactoftheconfidenceintervalonmeetingannualtargets.The
darkerportionsofthebarsshowtheactualproficiencyrateachieved,whilethelighter(upper)portionsofthebarsshowthemarginoferrorascomputedbythe
confidenceinterval.Thefigureshowsthatthreesamplemiddleschools(McBeal,MLAndrew,andPogesto)wereassistedbytheconfidenceinterval.Annualtargets(the
orangelines)areconsideredtobemetbytheconfidenceintervaliftheyfallwithinthelightblueportion.
Note that we were unable to examine the impact of vals) to determine whether the overall school population
NCLB’s“safeharbor”provision.Thisprovisionpermits and any qualifying subgroups achieved the AMOs. We
aschooltomakeAYPevenifsomeofitssubgroupsfail, deemedthataschoolmadeAYPifitsoverallstudentbody
as long as it reduces the number of nonproficient stu- andallitsqualifyingsubgroupsmetorexceededitsAMOs.
dentswithinany failingsubgroup by at least 10%rela- Again,Appendix1suppliesfurthermethodologicaldetail.
tivetothepreviousyear’sperformance.Becausewehad
accesstoonlyasingleacademicyear’sdata(2005–2006),
How Did the Sample Schools Fare
wewerenotabletoincludethisinouranalysis.Asare-
under New Mexico’s AYP Rules?
sult,it’spossiblethatsomeoftheschoolsinoursample
that failed to make AYP according to our estimates
Figure 3 illustrates the AYP performance of the sample
wouldhavemadeAYPunderrealconditions.
elementaryschoolsunderNewMexico’s2008AYPrules.
Only 4 of 18 elementary schools (Winchester,
Furthermore,attendanceandtestparticipationratesare
Marigold, Roosevelt, and King Richard) made AYP.
beyondthescopeofthestudy.Notethatmoststatesin-
ThetrianglesinFigure3showtheaverageacademicper-
cludeattendanceratesasanadditionalindicatorintheir
formance of students within the school, with negative
NCLBaccountabilitysystemforelementaryandmiddle
valuesindicatingbelow-grade-levelperformanceforthe
schools. In addition, federal law requires 95% of each
average student, and positive values indicating above-
school’sstudents—and95%ofthestudentsineachsub-
grade-level performance. All passing schools are in the
group—toparticipateintesting.
righthalfofthefigure,meaningthatthehighestaverage
Toreiterate,then,AYPdecisionsinthecurrentstudyare performingstudentswerefoundintheseschools.
modeledsolelyontestperformancedataforasingleaca-
demic year. For each school, we calculated reading and Figure 4 illustrates the AYP performance of the sample
math proficiency rates (along with any confidence inter- middleschoolsunderthe2008NewMexicoAYPrules.
7 THOMASB.FORDHAMINSTITUTE
Table2.Elementaryschoolsubgroupperformanceofsampleschoolsunderthe2008NewMexicoAYPrules
wMexico SPCSHEUODOOLNYM MatOverallh ProficiencyRRateeading MOverallR MSWDsR MLEPStudentsR MLow-incomeStudentsR MAAR MAsianR MHispanicR MAI/ANR MWhiteR AYPTargetsRequired TargetsMET %ofTargetsMet SchoolMetAYP? NumberofstatesinwhichschoolmetAYP?
e Clarkson 33.4% 42.3% N N N N N N N N N N 10 0 0% N 1
N Maryweather 42.9% 51.1% Y N N N N N Y N Y Y Y N Y Y 14 7 50% N 1
Few 48.1% 54.3% Y Y N N N N Y Y Y Y Y Y Y Y 14 10 71% N 1
Nemo 48.8% 67.9% Y Y N N Y Y Y Y Y Y 10 8 80% N 7
IslandGrove 50.0% 67.5% Y Y N N N N Y Y N Y Y Y 12 7 58% N 4
JFK 55.8% 61.2% Y Y Y N Y Y Y N Y Y 10 8 80% N 3
Scholls 66.4% 69.5% Y Y Y N Y Y Y Y Y Y Y Y Y Y 14 13 93% N 7
Hissmore 65.8% 73.3% Y Y N N Y Y Y Y Y Y 10 8 80% N 7
WolfCreek 59.2% 67.6% Y Y N N N N Y Y Y Y Y Y 12 8 67% N 5
AliceMayberry 64.1% 75.4% Y Y N N Y Y Y Y Y Y 10 8 80% N 9
WayneFineArts 59.2% 83.3% Y Y Y Y N Y Y Y 8 7 88% N 21
Winchester 66.0% 79.1% Y Y Y Y Y Y Y Y Y Y 10 10 100% Y 22
Coastal 70.9% 76.0% Y Y Y N Y N Y Y Y Y Y Y Y Y 14 12 86% N 3
Paramount 72.1% 76.1% Y Y Y Y N N Y Y Y Y Y Y 12 10 83% N 7
ForestLake 81.0% 84.9% Y Y Y N Y Y Y Y 8 7 88% N 8
Marigold 82.4% 87.0% Y Y Y Y Y Y Y Y Y Y Y Y Y Y 14 14 100% Y 10
Roosevelt 85.2% 92.2% Y Y Y Y Y Y Y Y 8 8 100% Y 28
KingRichard 81.1% 89.5% Y Y Y Y Y Y Y Y Y Y Y Y 12 12 100% Y 14
Abbreviations:M=math;R=reading;N=no;Y=yes;SWDs=studentswithdisabilities;AA=AfricanAmerican;Asian/PacificIslander=Asian;Hispanic/Latino=
Hispanic;AmericanIndian/AlaskaNative=AI/AN.
Note:Schoolsareorderedfromlowest(Clarkson)tohighest(KingRichard)averagestudentperformanceasmeasuredbycombinedandweightedmathandreading
performanceontheMAPassessment(notshownintable).Ablankspaceunderneathasubgroupmeansthatsubgroupcontainedfewerthantheminimumnumberof
studentsrequiredforevaluation,soitwasn’tcounted.A“Y”inbluemeansthatthegroupmettheAMOsandan“N”inpeachmeansthatthegroupdidnotmeettheAMOs.
Thetworightmostcolumnsshow(1)whetherthatschoolmetAYP(i.e.,itmetthetargetsforitsoverallpopulationandallrequiredsubgroups);and(2)thetotalnumber
ofstatesinthestudyforwhichthatschoolmetAYP.
Of 18 middle schools in our sample, only 2 made the AMO needed to meet AYP.These figures show that
AYP—one low-performance school (Pogesto) and one oneofthesampleelementaryschools(Maryweather)and
high-performanceschool(WalterJones),bothofwhich threemiddleschools(McBeal,MLAndrew,andPogesto)
haverelativelyfewqualifyingsubgroups. are assisted by the confidence intervals. However, of the
latterthree,onlyPogestoalsomeetsallofitssubgrouptar-
Figures5and6indicatethedegreetowhichschools’overall getsinordertomakeAYP(seeFigure4).
mathproficiencyratesareaidedbytheconfidenceinterval
forelementaryandmiddleschools,respectively.Onthese Theeffectofconfidenceintervalsonschools’proficiency
figures,thedarkbluebarsshowtheactualproficiencyrates ratesinreadingismuchthesame(notshown).Inreading,
ateachschool,andthelightbluebarsshowthedegreeto justoneelementaryschool(Few)andonemiddleschool
whichtheseproficiencyratesare“increased”bytheappli- (McBeal)mettheoveralltargetwiththeconfidenceinter-
cation of the confidence interval.The orange lines show val,butweknowfromFigures3and4thatbothschools
TheAccountabilityIllusion 8
Table3.Middleschoolsubgroupperformanceofsampleschoolsunderthe2008NewMexicoAYPrules
SPCSHEUODOOLNYM MatOverallh ProficiencyRRateeading MOverallR MSWDsR MLEPStudentsR MLow-incomeStudentsR MAAR MAsianR MHispanicR MAI/ANR MWhiteR AYPTargetsRequired TargetsMET %ofTargetsMet SchoolMetAYP? NumberofstatesinwhichschoolmetAYP? NewMex
McBeal 32.0% 52.7% Y Y N N N N N N N Y Y Y N N N Y Y Y 18 8 44% N 0 i
c
BarringerCharter 36.1% 57.1% N Y N N N Y N N Y Y Y Y 12 6 50% N 0 o
MLAndrew 31.9% 55.9% Y Y N N N N N N N N N Y Y Y 14 5 36% N 0
Pogesto 31.5% 66.7% Y Y Y Y Y Y 6 6 100% Y 15
McCordCharter 36.3% 59.2% Y Y N N N N N N N N Y Y Y 13 5 38% N 0
Tigerbear 42.3% 56.9% Y Y N N Y Y N N Y Y 10 6 60% N 0
Chesterfield 44.0% 58.6% Y Y N N Y Y Y N Y Y 10 7 70% N 1
Filmore 44.9% 67.4% Y Y N N N N Y Y Y Y Y Y 12 8 67% N 1
Barban- 44.5% 62.8% Y Y N N N N N N Y Y Y Y 12 6 50% N 0
Kekata 54.6% 66.7% Y Y N N N N Y Y Y N Y N Y Y 14 8 57% N 0
Hoyt 51.1% 69.2% Y Y N N Y Y Y Y Y Y 10 8 80% N 2
BlackLake 57.9% 69.2% Y Y N N Y N Y Y Y N Y Y Y Y Y Y 16 12 75% N 0
LakeJoseph 52.2% 74.3% Y Y N N N N Y Y Y Y Y Y 12 8 67% N 2
Zeus 58.2% 70.5% Y Y N N N N Y Y Y Y Y Y Y Y 14 10 71% N 1
OceanView 57.7% 80.9% Y Y Y Y N N Y Y Y Y Y Y Y Y 14 12 86% N 2
WalterJones 68.6% 80.6% Y Y Y Y Y Y Y Y 8 8 100% Y 20
Artemus 65.0% 79.2% Y Y Y N Y Y Y Y Y N Y Y 12 10 83% N 3
Chaucer 70.2% 85.3% Y Y N Y Y N Y Y Y Y Y Y Y Y Y Y 16 14 88% N 5
Abbreviations:M=math;R=reading;N=no;Y=yes;SWDs=studentswithdisabilities;AA=AfricanAmerican;Asian/PacificIslander=Asian;Hispanic/Latino=
Hispanic;AmericanIndian/AlaskaNative=AI/AN.
Note:Schoolsareorderedfromlowest(McBeal)tohighest(Chaucer)averagestudentperformanceasmeasuredbycombinedandweightedmathandreading
performanceontheMAPassessment(notshownintable).Ablankspaceunderneathasubgroupmeansthatsubgroupcontainedfewerthantheminimumnumberof
studentsrequiredforevaluation,soitwasn’tcounted.A“Y”inbluemeansthatthegroupmettheAMOsandan“N”inpeachmeansthatthegroupdidnotmeettheAMOs.
Thetworightmostcolumnsshow(1)whetherthatschoolmetAYP(i.e.,itmetthetargetsforitsoverallpopulationandallrequiredsubgroups);and(2)thetotalnumber
ofstatesinthestudyforwhichthatschoolmetAYP.
still failed to meet targets for some of their subgroups. hasfewertargetstomeetbecauseithasfewersubgroups.
Overall, the application of the confidence interval had Thesefiguresdonot,however,indicatewhichsubgroups
onlymodestimpactonfinalAYPdecisionsforthesample failedorpassedinwhichschool.Informationonindivid-
elementaryandmiddleschoolsinNewMexico.9 ualsubgroupperformanceappearsinTables2and3for
elementaryandmiddleschools,respectively.
Where Do Schools Fail?
Tables2and3showwhichsubgroupsqualifiedforeval-
Figures3and4illustratethatschoolswithlowormid- uation at each school (i.e., whether the number of stu-
dling performance can still pass AYP when the school dents within that subgroup exceeded the state’s
9Inthecurrentanalyses,confidenceintervalswereappliedtoboththeoverallstudentpopulationandtoalleligiblesubgroupsinoursample
schools.Thus,theultimateimpactoftheconfidenceintervalmaybelargerthantheimpactdepictedinFigures5and6.However,wechosenot
toshowhowtheconfidenceintervalimpactedsubgroupperformancebecauseitwouldhaveaddedgreatlytothereport’slengthandcomplexity.
9 THOMASB.FORDHAMINSTITUTE
Table4.Summaryofsubgroupperformanceofsampleelementaryschoolsunder2008NewMexicoAYPrules
Numberofschoolswith Numberofschoolswhere Numberofschoolswhere
o SUBGROUP qualifyingsubgroups subgroupfailedtomeetmath subgroupfailedtomeetreading
c target target
i
x
Studentswithdisabili"es 16 8 12
e
M StudentswithlimitedEnglish
10 6 7
proficiency
w
Low-incomestudents 18 1 2
e
N
African-Americanstudents 9 1 1
Asian/PacificIslanderstudents 1 0 0
Hispanicstudents 12 2 2
AmericanIndian/AlaskaNa"ve
0 0 0
students
Whitestudents 17 0 0
Table5.Summaryofsubgroupperformanceofsamplemiddleschoolsunder2008NewMexicoAYPrules
Numberofschoolswith Numberofschoolswhere Numberofschoolswhere
SUBGROUP qualifyingsubgroups subgroupfailedtomeetmath subgroupfailedtomeetreading
target target
Studentswithdisabili"es 16 14 14
StudentswithlimitedEnglish
11 9 10
proficiency
Low-incomestudents 18 5 4
African-Americanstudents 11 5 7
Asian/PacificIslanderstudents 5 0 0
Hispanicstudents 14 3 3
AmericanIndian/AlaskaNa"ve
1 1 0
students
Whitestudents 18 0 0
minimum n), and whether that subgroup passed or rules,andthetotalnumberofstateswithinthestudyin
failed. Although all schools are evaluated on the profi- whichthatschoolmetAYP.
ciency rate of their overall population, potential sub-
groups that are separately evaluated for AYP include Theschool-by-schoolfindingsinTables2and3showthat:
SWDs,studentswithLEP,low-incomestudents,andthe
following race/ethnic categories: African American, (cid:1) Almost all schools met their reading and math tar-
Asian/Pacific Islander, Hispanic/Latino, American In- getsfortheiroverallschoolpopulation.
dian/AlaskaNative,andwhite.Tables2and3alsoshow
whetheraschoolmetAYPunderthe2008NewMexico (cid:1) Just two elementary schools (Clarkson and Mary-
TheAccountabilityIllusion 10