ERIC ED504680: The Accountability Illusion: Arizona

2009·0.24 MB·English

by  ERIC

Arizona
Executive Summary (cid:1) Several sample schools made AYP in Arizona that A
failed to make AYP in most other states. This is r
i
TheintentoftheNoChildLeftBehind(NCLB)Actof probablybecauseArizona’sproficiencystandardsare z
2001istoholdschoolsaccountableforensuringthatall o
relativelyeasycomparedtootherstates(especially
n
theirstudentsachievemasteryinreadingandmath,with
inreading).AnotherreasonisthatArizona’sdefini- a
aparticularfocusongroupsthathavetraditionallybeen
tions for subgroups are grade-based rather than
leftbehind.UnderNCLB,statessubmitaccountability
school based, resulting in fewer accountable sub-
planstotheU.S.DepartmentofEducationdetailingthe
groups(i.e.,aschoolmusthaveatleast40individ-
rules and policies to be used in tracking the adequate
uals within a grade for that group to be evaluated).
yearlyprogress(AYP)ofschoolstowardthesegoals.
Arizonaalsousesaverygenerousconfidenceinterval
(ormarginoferror).
ThisreportexaminesArizona’sNCLBaccountabilitysys-
tem—particularlyhowitsvariousrules,criteriaandprac-
ticesresultinschoolseithermakingAYP—ornotmaking
AYP. It also gauges how tough Arizona’s system is com-
pared with other states. For this study, we selected 36
schools from various states around the nation, schools
Arizonahasseveraluniquecharacteristicswhich
thatvarybysize,achievement,anddiversity,amongother
contributetothelargenumberofschoolsmaking
factors,anddeterminedwhethereachwouldmakeAYP
AYPinthestate.Infact,onlyoneotherstateinthe
underArizona’ssystemaswellasunderthesystemsof27
other states. We used school data and proficiency cut study(Wisconsin)deemsthatmoreschoolsmake
score1estimatesfromacademicyear2005–2006,butap- AYPthanArizonadoes.Oneofthefactors
pliedthemagainstArizona’sAYPrulesfortheacademic contributingtothisistherulesetgoverning
year2007–2008(shortenedto“2008”inthisreport).
subgroupsize.Unlikemoststates,Arizonaconsiders
eachgradeseparatelywhendeterminingwhethera
Herearesomekeyfindings:
subgroupmeetsthecriteriaforaccountability,which
(forArizona)isatleast40students.Forinstance,a
(cid:1)Weestimatethat3of18elementaryschoolsand10
middleschoolinArizonawiththreegradescould
of 18 middle schoolsinoursamplefailed to make
havealmost120African-Americanstudents,all
AYPin2008underArizona’saccountabilitysystem.
performingpoorly,andstillmakeAYPaslongas
Among the 28 accountability systems examined in
thestudy,there'sonlyonestatewheremoreschools therearefewerthan40African-Americanchildrenin
makeAYPthaninArizona(Wisconsin).Thismakes eachgrade.Anotherfactorcontributingtothehigh
TheGrandCanyonStateoneoftheleastrestrictive numberofschoolsmakingAYPisArizona’s
intermsofAYPpassagerates(seeFigure1.)2 99percentconfidenceinterval(i.e.,statistical
marginoferror).Thisprovidesschoolswithgreater
leniencythanthe95percentconfidenceinterval
1Acutscoreistheminimumscoreastudentmustreceiveonthe
usedbymostotherstatesinthestudy.Finally,
Arizona'sInstrumenttoMeasureStandards(AIMS)inordertobe
consideredproficientunderArizona'saccountabilitysystem. Arizona’sproficiencystandards(orcutscores)are
2NotethatArizonareceivedfullapprovalfromtheU.S.Department
relativelyeasyintheearlygrades,comparedtoother
ofEducationtoimplementastudentgrowthmodelforthe2006-
2007schoolyear.Thecurrentanalysis,whichdrawsondatafrom states.Infact,ingrades3-5,thereadingcutscoreis
2005–2006,doesnotinanywayuseorincorporatestudentgrowth
modelcalculations. inthe25thpercentilerange.
1 THOMASB.FORDHAMINSTITUTE
18
a 16
n P
Y
A 14
o g
n
z ki
i Ma 12
r ols
A o 10
h
Sc
ple 8
m
a
ofS 6
er
mb 4
u
N
2
0
Massachuse s Nevada Idaho NorthDakota Kansas Washington Wyoming Indiana SouthCarolina Montana Florida Vermont NewJersey NewHampshire Maine NewMexico Delaware Colorado RhodeIsland Georgia Illinois Ohio Minnesota Michigan California Texas Arizona Wisconsin
Elementary Schools Middle Schools
Figure1.NumberofsampleschoolsmakingAYPbystate
Note:MiddleschoolswerenotincludedforTexasandNewJersey;absenceofamiddleschoolbarinthosestatesmeans“notapplicable”asopposedtozero.Stateslike
IdahoandNorthDakota,however,havezeropassingmiddleschools.
(cid:1)Nearlyalloftheschoolsinoursamplethatfailedto are larger and therefore have more qualifying sub-
make AYP in Arizona are meeting expected targets groups—not because their student achievement is
for their overall populations, but failing because of lowerthanintheelementaryschools.4
theperformanceofindividualsubgroups—particu-
(cid:1) A strong predictor of a school making AYP under
larlystudentswithdisabilities(SWDs)atthemiddle
Arizona’ssystemiswhetherithasenoughSWDsto
schoollevel.3
qualifyasaseparatesubgroup.Incaseswherethere
wereenoughstudentstoconstituteaseparateSWD
(cid:1)InArizona,asinmoststates,schoolswithfewersub-
subgroup,everyschoolwithonefailedtomakeAYP.
groupsattainAYPmoreeasilythanschoolswithmore
subgroups,evenwhentheiraveragestudentperform-
Introduction
anceislower.Inotherwords,schoolswithgreaterdi-
versityandsizefacegreaterchallengesinmakingAYP. TheProficiencyIllusion(Cronin,etal.2007a)linkedstu-
dentperformanceonArizona’stestandthoseof25other
(cid:1)Asinotherstates,middleschoolshavegreaterdiffi- states to the Northwest Evaluation Association’s
culty reaching AYP in Arizona than do elementary (NWEA)MeasuresofAcademicProgress(MAP),acom-
schools,primarilybecausetheirstudentpopulations puterizedadaptivetestusedinschoolsnationwide.This
3SWDsaredefinedasthosestudentsfollowingindividualizededucationplans.WeshouldalsonotethatoursubgroupfindingsforLimited
Englishproficient(LEP)studentsandSWDsmaybemorenegativethanactualfindings,mostlybecauseofthelikelydifferencesbetweenhow
LEPstudentsandSWDsaretreatedinMAP,theassessmentweusedinthisstudy,andinArizona’sInstrumenttoMeasureStandards(AIMS),
thestandardizedstatetest.Specifically,theU.S.DepartmentofEducationhasissuednewNCLBguidelinesinrecentyearsthatexcludesmall
percentagesofLEPstudentsandSWDsfromtakingthestatetestorthatallowthemtotakealternativeassessments.Inthisstudy,however,
novalidMAPscoreswereomittedfromconsideration.
4It’simportanttonotethatstudentsinsubgroupsnotmeetingtheminimumnsizesarestillincludedforaccountabilitypurposesintheoverall
studentcalculations;theysimplyarenottreatedastheirownsubgroup.
TheAccountabilityIllusion 2
singlecommonscalepermittedcross-statecomparisons berofstudentsinsample]size).Therationaleisthatre-
ofeachstate’sreadingandmathproficiencystandardsto portingtheresultsofverysmallsubgroups—fewerthan
measure school performance under the No Child Left tenpupils,forexample—couldjeopardizestudents’con- A
r
Behind (NCLB) Act of 2001.That study revealed pro- fidentialityandriskpresentinginaccurateresults.(With i
z
found differences in states’ proficiency standards (i.e., suchsmallgroups,randomevents,likeonestudentbeing
o
howdifficultitistoachieveproficiencyonthestatetest), out sickon test day, could skew theoutcome.) Because n
andevenacrossgradeswithinasinglestate. of this flexibility, states have set widely varying n sizes a
for their subgroups, from as few as 10 youngsters to as
OurstudyexpandsonTheProficiencyIllusionbyexam-
manyas100.
ining other key factors of state NCLB accountability
plansandhowtheyinteractwithstateproficiencystan-
Manystateshavealsoadoptedconfidenceintervals—ba-
dards to determine whether the schools in our sample
sicallymarginsofstatisticalerror–toaccountforpoten-
made adequate yearly progress (AYP) in 2008. Specifi-
tial measurement error within the state test. In some
cally, we estimated how a single set of schools, drawn
states,thesemarginsarequitewide,whichhastheeffect
fromaroundthecountry,wouldfareunderthediffering
ofmakingiteasiertoachieveanannualtarget.
rulesfordeterminingAYPin28states(theoriginal25in
The Proficiency Illusion plus 3 others for which we now
AlloftheseAYPrulesvarybystate,whichmeansthata
have cut score estimates). In other words, if we could
somehowmovetheseentireschools--withtheirsamemix schoolthatmakesAYPinWisconsinorOhio,forexam-
of characteristics—from state to state, how would they ple,mightnotmakeitunderSouthCarolina’sorIdaho’s
fare in terms of making AYP? Will schools with high- rules(U.S.DepartmentofEducation2008).
performing students consistently make AYP? Will
schools with low-performing students consistently fail
What We Studied
tomakeAYP?IfAYPdeterminationsforschoolsarenot
consistentacrossstates,whatleadstotheinconsistencies? We collected students’ MAP test scores from the
2005–2006 academic year from 18 elementary and 18
NCLB requires every state, as a condition of receiving
middleschoolsaroundthecountry.Wealsocollectedthe
TitleIfunding,toimplementanaccountabilitysystem
NCLB subgroup designations for all students in those
that aims to get 100% of its students to the proficient
schools—inotherwords,whethertheyhadbeenclassi-
levelonthestatetestbyacademicyear2013–2014.In
fied as members of a minority group, such as English
the intervening years, states set annual measurable ob-
languagelearners,6amongothersubgroups.
jectives (AMOs).This is the percentage of students in
each school, and in each subgroup within the school
Theschoolswerenotselectedasarepresentativesample
(suchaslowincome5orAfricanAmerican,amongoth-
of the nation’s population. Instead, we selected the
ers)thatmustreachtheproficientlevelinorderforthe
schoolsbecausetheyexhibitedarangeofcharacteristics
schooltomakeAYPinagivenyear.TheAMOsvaryby
on measures such as academic performance, academic
state (as do, of course, the difficulty of the proficiency
standards). growth, and socioeconomic status (the latter calculated
bythepercentageofstudentsreceivingfreeorreduced-
Statesalsodeterminetheminimumnumberofstudents pricelunches).Appendix1containsacompletediscus-
thatmustconstituteasubgroupinorderforitsscoresto sionofthemethodologyforthisprojectalongwiththe
beanalyzedseparately(alsocalledtheminimumn[num- characteristicsoftheschoolsample.7
5Low-incomestudentsarethosewhoreceiveafreeorreduced-pricelunch.
6Notethatweuse“LEPstudents”and“Englishlanguagelearners”interchangeablytorefertostudentsinthesamesubgroup.
7Wegaveallschoolsinoursamplepseudonymsinthisreport.
3 THOMASB.FORDHAMINSTITUTE
45
40
a
n 35
o g
n 30
z ki
n
i a 2255
r RR
e Reading
A l 20
n
e Math
erc 15
P
10
5
0
Grade3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8
Figure2.Arizonareadingandmathcutscoreestimates,expressedaspercentileranks(2006)
Note:ThisfigureillustratesthedifficultyofArizona’scutscores(orproficiencypassingscores)foritsreadingandmathtests,aspercentilesoftheNWEAnorm,ingrades
threethrougheight.Higherpercentileranksaremoredifficulttoachieve.AllofArizona’scutscoresarebelowthe45thpercentile.
ProficiencycutscoreestimatesforArizona’sInstrument Furthermore,althoughmoststatesapplyconfidenceinter-
to Measure Standards (AIMS) are taken from The Pro- vals(ormarginsofstatisticalerror)totheirmeasurement
ficiencyIllusion(asshowninFigure2),whichfoundthat ofstudentproficiencyrates,Arizona’s99%confidencein-
Arizona’sdefinitionsofproficiencyinreadingandmath terval gives schools greater leniency than the 95% confi-
were below-average to average in terms of difficulty, denceintervalusedbymostotherstates.So,forinstance,
compared to the other states in the study. These cut althoughschoolsaresupposedtoget38%oftheireighth
grade students to the proficient level on the state math
scores were used to estimate whether students would
test—and 38% of their students in each subgroup—ap-
have scored as proficient or better on the Arizona test,
plying the confidence interval means that the real target
giventheirperformanceonMAP. Studenttestdataand
canactuallybelower,particularlywithsmallergroups.
subgroupdesignationswerethenusedtodeterminehow
these18elementaryand18middleschoolswouldhave
Note that we were unable to examine the effect of
faredunderArizonaAYPrulesfor2008.Inotherwords,
NCLB’s “safe harbor” provision. This provision per-
the school data and our proficiency cut score estimates
mitsaschooltomakeAYPevenifsomeofitssubgroups
arefromacademicyear2005–2006,butweareapplying
fail, as long as it reduces the number of nonproficient
themagainstArizona’s2008AYPrules.
students within any failing subgroup by at least 10%
relativetothepreviousyear’sperformance.Becausewe
Table1showsthepertinentArizonaAYPrulesthatwere
had access to only a single academic year’s data
appliedtoelementaryandmiddleschoolsinthisstudy.
(2005–2006), we were not able to include this in our
Arizona’s minimum subgroup size is 40, which is com-
analysis.Asaresult,it’spossiblethatsomeoftheschools
parabletomostotherstatesweexamined.8However,the inoursamplethatfailedtomakeAYPaccordingtoour
sizeisgrade-based,meaningaschoolmusthaveatleast estimateswouldhavemadeAYPunderrealconditions.
40 individuals within a grade for that subgroup to be
evaluated.Annualtargetsalsochangeaccordingtograde Furthermore,attendanceandtestparticipationratesare
andsubjectarea.Theannualtargetforgrade3reading, beyondthescopeofthestudy.Notethatmoststatesin-
for example, is 62% of students reaching proficiency; cludeattendanceratesasanadditionalindicatorintheir
thatnumberchangesto38%forgrade8math. NCLBaccountabilitysystemforelementaryandmiddle
8Keepinmindthatschoolsizeandnsizearerelated(e.g.,smallnsizesmakesenseforsmallschools).
TheAccountabilityIllusion 4
Table1.ArizonaAYPrulesfor2008
Subgroupminimumn Race/ethnicity:40
A
SWDs:40 r
i
z
Low-incomestudents:40
o
LEPstudents:40 n
a
CI Appliedtoproficiencyratecalcula-ons?
Yes;99%CIused
AMOs Baselineproficiencylevelsasof2002(%) 2008targets(%)
READING/LANGUAGEARTS
Grade3 44.0 62.6
Grade4 45.0 56.0
Grade5 32.0 54.6
Grade6 45.0 56.0
Grade7 49.0 59.2
Grade8 31.0 54.0
MATH
Grade3 32.0 54.6
Grade4 54.0 63.2
Grade5 20.0 46.6
Grade6 43.0 54.4
Grade7 48.0 58.4
Grade8 7.0 38.0
Sources:U.S.DepartmentofEducation(2008);CouncilofChiefStateSchoolOfficers(2008).
Abbreviations:SWDs=studentswithdisabilities;LEP=limitedEnglishproficiency;CI=confidenceinterval;AMOs=annualmeasurableobjectives
schools. In addition, federal law requires 95% of each How Did the Sample Schools
school’s students—and 95% of the students in each Fare Under Arizona’s AYP Rules?
school’ssubgroup—toparticipateintesting.
Figure 3 illustrates the AYP performance of the sample
Toreiterate,then,AYPdecisionsinthecurrentstudyare elementary schools under Arizona’s 2008 AYP rules.
modeledsolelyontestperformancedataforasingleac- Only 3 of the 18 elementary schools failed to make
ademicyear.Foreachschool,wecalculatedreadingand AYPundertheArizonarules.ThetrianglesinFigure3
mathproficiencyrates(alongwithanyconfidenceinter- show the average academic performance of students
vals)todeterminewhethertheoverallschoolpopulation withintheschool,withnegativevaluesindicatingbelow-
and any qualifying subgroups achieved the AMOs.We grade-level performance for the average student, and
deemed that a school made AYP if its overall student positivevaluesindicatingabove-grade-levelperformance.
body and all its qualifying subgroups met or exceeded The two schools with lowest average student perform-
itsAMOs.Again,Appendix1suppliesfurthermethod- ance(ClarksonandMaryweather)bothfailtomakeAYP,
ologicaldetail. as does one of the schools with higher average student
5 THOMASB.FORDHAMINSTITUTE
35 15
ol
o
a 30 h
10 c
n S
n
o ets 25 5 cei
z rg an
i Ta 20 m
Ar erof 15 0 erfor
b P
mm -5 tnt
Nu 10 de
u
-10 St
5 e
g
a
r
0 -15 ve
A
1) 1) 1) 7) 5) 3) 7) 7) 5) 9) 1) 2) 3) 7) 8) 0) 8) 4)
( ( ( ( ( ( ( ( ( ( 2 2 ( ( ( 1 2 1
Clarkson Maryweather Few Nemo IslandGrove ohnF.Kennedy Scholls Hissmore WolfCreek Mayberry ayneFineArts( Winchester( Coastal Paramount ForestLake Marigold( Roosevelt( KingRichard(
J W
TargetsPassed TargetsFailed Average Student Performance
Figure3.AYPPerformanceoftheelementaryschoolsampleunderArizona’s2008AYPrules
Note:ThisfigureindicateshoweachelementaryschoolwithinthesamplefaredunderArizona’sAYPrules(asdescribedinTable1).Thebarsshowthenumberoftargets
thateachschoolhastomeetinordertomakeAYPunderthestate’sNCLBrules,andwhethertheymetthem(darkblue)ordidnotmeetthem(lightblue).Themore
subgroupsinaschool,themoretargetsitmustmeet.Underthestudyconditions,aschoolthatfailedtomeettheAMOsforevenasinglesubgroupdidn’tmakeAYP,so
anylightbluemeanstheschoolfailed.CoastalElementary,forexample,met25ofits26targets,butbecauseitdidn’tmeetthemall,itdidn’tmakeAYP.Schoolsare
orderedfromlowesttohighestaveragestudentperformance(shownbytheorangetriangles)whichismeasuredbytheaverageMAPperformanceofstudentswithin
theschool;itsscaleisshownontherightsideofthefigure.Scoresbelowzero(whichisthegradelevelmedian)denotebelow-grade-levelperformanceandscores
abovezerodenoteabove-grade-levelperformance.Oneunitdoesnotequalagradelevel;however,thehigherthenumber,thebettertheaverageperformanceandthe
lowerthenumber,theworsetheaverageperformance.Thenumberinparenthesesaftereachschoolnameindicatesthenumberofstates,outof28,inwhichthat
schoolwouldhavemadeAYP.
performance (Coastal). All three schools that failed to mathproficiencyratesareaidedbytheconfidenceinter-
make it, however, have between 24 and 28 targets to val.Onthisfigure,thedarkerportionsofthebarsshow
meet, as opposed to the schools that made AYP, which theactualproficiencyratesateachschool,andthelighter
have,onaverage,only20targetstomeet.9 portionsofthebarsshowthedegreetowhichthesepro-
ficiency rates were “increased” by the application of the
Figure 4 illustrates the AYP performance of the sample confidence interval. The orange lines show the annual
middleschoolsunderthe2008ArizonaAYPrules.Out measurable objective needed to meet AYP. The figure
of 18 middle schools in our sample, 8 made AYP – showsthatnoneofthesampleelementaryschoolswasas-
three low-performance schools (Pogesto, Chesterfield, sistedbytheconfidenceintervals,becausethemathtargets
andFilmore),andfivehigh-performanceschools(Lake inArizonaarelowrelativetotheschools’overallperform-
Joseph, Ocean View, Walter Jones, Artemus, and ance.Althoughnotshown,thissametrendheldtruefor
Chaucer). As with the sample elementary schools, middle school math and reading proficiency rates at the
schools that made AYP tended to have fewer targets to middle and elementary school levels as well. Because of
meetthanschoolsthatdidn’tmakeAYP. therelativelyeasytargetsestablishedbyArizona’sannual
measurable objectives, confidence intervals have little
Figure5indicatesthedegreetowhichelementaryschools’ impactonwhetherschoolsmakeAYP.10
9RecallthatArizonahasmoretargetsbecauseeachgradelevelisconsideredagroupuntoitself. Forinstance,amiddleschoolinArizonawith
threegradesandfoursubgroupshas3×4×2(subjects)or24targets.
10Inthecurrentanalyses,confidenceintervalswereappliedtoboththeoverallschoolpopulationandtoalleligiblesubgroupsinoursample
schools.Thus,theultimateimpactoftheconfidenceintervalmaybelargerthantheimpactdepictedinFigure5.However,wechosenotto
showhowtheconfidenceintervalimpactedsubgroupperformancebecauseitwouldhaveaddedgreatlytothisreport’slengthandcomplexity.
TheAccountabilityIllusion 6
50 12
ol
45 10 o
h A
c
40 8 S
-groupsb 333500 464 anceinm rizo
Su 25 2 or n
erof 20 0 Perf a
mb 15 -2 ent
u d
N 10 -4 u
St
5 -6 e
g
a
0 -8 er
v
0) 0) 0) 5) 0) 0) 1) 1) 0) 0) 2) 0) 2) 1) 2) 0) 3) 5) A
( ( ( 1 ( ( ( ( ( ( ( ( ( ( ( 2 ( (
McBeal erCharterarringe LAndrewM Pogesto( McCord TigerbearT esterfieldChe Filmore Barban Kekata Hoyt BlackLakeB keJosephLak Zeus eanViewOc erJones(eWalt Artemus Chaucer
B
TargetsPassed TargetsFailed AverageStudentPerformance
Figure4.AYPperformanceofthemiddleschoolsampleunderArizona’s2008AYPrules
Note:ThisfigureshowshoweachmiddleschoolwouldhavefairedunderArizona’sAYPrules(asdescribedinTable1).Thebarsshowthenumberoftargetsthateachschool
hadtomeetinordertomakeAYPunderthestate’sNCLBrules,andwhethertheymetthem(darkblue)ordidnotmeetthem(lightblue).Themoresubgroupsinaschool,
themoretargetsitmustmeet.Underthestudyconditions,aschoolthatfailedtomeettheAMOforevenasinglesubgroupdidnotmakeAYP,soanylightbluemeansthe
schoolfailed.ZeusMiddleSchool,forexample,met29ofits30targets,butbecauseitdidn’tmeetthemall,itdidn’tmakeAYP.Schoolsareorderedfromlowesttohighest
averagestudentperformance(shownbytheorangetriangles)whichismeasuredbyaverageMAPperformanceofstudentswithintheschool;itsscaleisshownonthe
rightsideofthefigure.Scoresbelowzero(whichisthegradelevelmedian)denotebelow-grade-levelperformanceandscoresabovezerodenoteabove-grade-level
performance.Oneunitdoesnotequalagradelevel;however,thehigherthenumber,thebettertheaverageperformanceandthelowerthenumber,theworsethe
averageperformance.Thenumberinparenthesesaftereachschoolnameindicatesthenumberofstates,outof28,inwhichthatschoolwouldmakeAYP.
1
0.9
0.8
0.7
e
Rat 0.6
y
nc 0.5
e
fici 0.4
o
r
P 0.3
0.2
0.1
0
arksonCla eatherMarywe Few Nemo GroveIsland ennedyehnF.K SchollsS ssmoreHis fCreekWolf yberryMa ayneFineArtsn Winchester CoastalC mountParam stLakeFores arigoldMa oseveltRoo RichardKingR
o W
J
MathProficiencyRate MathProficiencyRatewithCI MathTarget
Figure5.Impactoftheconfidenceintervalonelementaryschoolmathproficiencyrates
Note:Thisfigureshowsthereportedproficiencyrateforthestudentpopulationasawholeandtheimpactoftheconfidenceintervalonmeetingannualtargets.The
darkerportionsofthebarsshowtheactualproficiencyrateachieved,whilethelighter(upper)portionsofthebarsshowthemarginoferrorascomputedbythe
confidenceinterval.Thefigureshowsthatnoneofthesampleelementaryschoolswasassistedbytheconfidenceinterval.Annualtargets(theorangelines)are
consideredtobemetbytheconfidenceintervaliftheyfallwithinthelightblueportion.
7
THOMASB.FORDHAMINSTITUTE
Table2.Elementaryschoolsubgroupperformanceofsampleschoolsunderthe2008ArizonaAYPrules
Arizona SPCSHEUODOOLNYM MatOverallh ProficiencyRRateeading MOverallR MSWDsR MLEPStudentsR MLow-incomeStudentsR MAAR MAsianR MHispanicR MAI/ANR MWhiteR AYPTargetsRequired TargetsMET %ofTargetsMet SchoolMetAYP? NumberofstatesinwhichschoolmetAYP?
Clarkson 70.6% 58.1% Y N Y N Y N 24 18 75% N 1
Maryweather 76.6% 68.2% Y Y Y N Y Y Y N 28 24 86% N 1
Few 81.3% 70.6% Y Y Y Y Y Y 24 24 100% Y 1
Nemo 85.5% 85.4% Y Y Y Y 18 18 100% Y 7
IslandGrove 87.0% 83.1% Y Y Y Y 16 16 100% Y 5
JFK 89.5% 78.7% Y Y Y Y Y Y 24 24 100% Y 3
Scholls 94.2% 84.7% Y Y Y Y Y Y Y Y 28 28 100% Y 7
Hissmore 94.0% 86.7% Y Y Y Y Y Y 24 24 100% Y 7
WolfCreek 87.7% 85.3% Y Y Y Y Y Y 22 22 100% Y 5
AliceMayberry 92.5% 88.7% Y Y Y Y Y Y Y Y Y Y 32 32 100% Y 9
WayneFineArts 95.9% 96.4% Y Y Y Y 14 14 100% Y 21
Winchester 93.3% 94.2% Y Y Y Y 16 16 100% Y 22
Coastal 91.2% 85.3% Y Y Y N Y Y Y Y Y Y 26 25 96% N 3
Paramount 93.2% 88.6% Y Y Y Y 18 18 100% Y 7
ForestLake 97.3% 94.7% Y Y Y Y Y Y 20 20 100% Y 8
Marigold 98.1% 94.7% Y Y Y Y 16 16 100% Y 10
Roosevelt 100.4% 99.7% Y Y Y Y 18 18 100% Y 28
KingRichard 98.1% 96.3% Y Y Y Y 16 16 100% Y 14
Abbreviations:M=math;R=reading;N=no;Y=yes;SWDs=studentswithdisabilities;AA=AfricanAmerican;Asian/PacificIslander=Asian;Hispanic/Latino=
Hispanic;AmericanIndian/AlaskaNative=AI/AN.
Note:Schoolsareorderedfromlowest(Clarkson)tohighest(KingRichard)averagestudentperformanceasmeasuredbycombinedandweightedmathandreading
performanceontheMAPassessment(notshownintable).Ablankspaceunderneathasubgroupmeansthatsubgroupcontainedfewerthantheminimumnumberof
studentsrequiredforevaluation,soitwasn’tcounted.A“Y”inbluemeansthatthegroupmettheAMOsandan“N”inpeachmeansthatthegroupdidnotmeettheAMOs.
Thetworightmostcolumnsshow(1)whetherthatschoolmetAYP(i.e.,itmetthetargetsforitsoverallpopulationandallrequiredsubgroups);and(2)thetotalnumber
ofstatesinthestudyforwhichthatschoolmetAYP.Unlikemoststates,Arizonaschoolsconsidereachgradeseparatelywhendeterminingwhethertheminimumnsize
isexceededforaparticularsubgroup.ThismeansthatArizonaschoolsmayberequiredtomeetupto18targetsforeachgrade(2targetseach—mathandreading—for
theoverallpopulation,SWDs,LEP,lowincome,AfricanAmerican,Asian,Hispanic,AmericanIndian,andwhite).Thisis,ofcourse,providedthattherearesufficient
numbersofstudentswithinthegradetoexceedthestate'sminimumnsizeof40ineverysubgroup.(Inactuality,it'smuchhardertoexceedtheminimumnsizewhen
individualgradelevelsareconsideredversustheschoolasawhole.)Inthistable,forexample,weseethatClarksonElementarymettheminimumnsizeforitsoverall,
Hispanic,andlowincomesubgroups.However,topreservespace,eachgradeisnotdisplayedseparately.Consequently,thenumberofAYPtargetsrequiredatClarkson
(24)andthenumberoftargetsmet(18),letusknowthattheschoolfailedtomeetallofitsrequiredsubgrouptargets,butwedon’tknowinwhichgrades.
Where Do Schools Fail? mentaryandmiddleschools,respectively.
Figures 3 and 4 illustrate that schools with low average Tables2and3showwhichsubgroupsqualifiedforeval-
studentperformancecanstillmakeAYPwhentheschool uation at each school (i.e., whether the number of stu-
has relatively few targets to meet because it has fewer dents within that subgroup exceeded the state’s
subgroups. These figures do not, however, indicate minimum n), and whether that subgroup passed or
whichsubgroupsfailedorpassedinwhichschool.Tables failed. Although all schools are evaluated on the profi-
2and3listinformationonindividualsubgroupforele- ciency rate of their overall population, potential sub-
TheAccountabilityIllusion 8
Table3.Middleschoolsubgroupperformanceofsampleschoolsunderthe2008ArizonaAYPrules
SPCSHEUODOOLNYM MatOverallh ProficiencyRRateeading MOverallR MSWDsR MLEPStudentsR MLow-incomeStudentsR MAAR MAsianR MHispanicR MAI/ANR MWhiteR AYPTargetsRequired TargetsMET %ofTargetsMet SchoolMetAYP? NumberofstatesinwhichschoolmetAYP? Arizona
McBeal 62.9% 66.0% Y Y N N N N N N N N Y Y 40 27 68% N 0
BarringerCharter 66.9% 69.4% Y Y Y Y Y N Y Y 48 47 98% N 0
MLAndrew 63.9% 71.6% Y Y N N N Y N N Y Y Y Y 32 24 75% N 0
Pogesto 77.7% 92.1% Y Y 12 12 100% Y 15
McCordCharter 65.8% 72.9% Y Y Y Y N N N Y Y Y Y Y 35 30 86% N 0
Tigerbear 73.2% 71.5% Y Y N N Y Y Y Y Y Y 36 31 86% N 0
Chesterfield 78.4% 75.1% Y Y Y Y Y Y Y Y 30 30 100% Y 1
Filmore 76.4% 82.2% Y Y Y Y Y Y Y Y 30 30 100% Y 1
Barban- 69.6% 75.0% Y Y N N N N N Y Y Y Y 37 27 73% N 0
Kekata 80.4% 77.9% Y Y N Y Y Y Y Y Y Y 32 31 97% N 0
Hoyt 81.7% 80.9% Y Y N N Y Y Y Y Y Y 36 33 92% N 2
BlackLake 83.5% 80.3% Y Y N N Y Y Y Y Y Y 36 31 86% N 0
LakeJoseph 82.1% 86.5% Y Y Y Y Y Y Y Y 30 30 100% Y 2
Zeus 83.7% 82.2% Y Y N Y Y Y Y 30 29 97% N 1
OceanView 86.4% 91.4% Y Y Y Y Y Y Y Y 26 26 100% Y 2
WalterJones 100.0% 99.9% Y Y 12 12 100% Y 20
Artemus 90.3% 92.5% Y Y Y Y Y Y 18 18 100% Y 3
Chaucer 91.4% 93.1% Y Y Y Y Y Y Y Y Y 28 28 100% Y 5
Abbreviations:M=math;R=reading;N=no;Y=yes;SWDs=studentswithdisabilities;AA=AfricanAmerican;Asian/PacificIslander=Asian;Hispanic/Latino=
Hispanic;AmericanIndian/AlaskaNative=AI/AN.
Note:Schoolsareorderedfromlowest(McBeal)tohighest(Chaucer)averagestudentperformanceasmeasuredbycombinedandweightedmathandreadingperformance
ontheMAPassessment(notshownintable).Ablankspaceunderneathasubgroupmeansthatsubgroupcontainedfewerthantheminimumnumberofstudentsrequired
forevaluation,soitwasn’tcounted.A“Y”inbluemeansthatthegroupmettheAMOsandan“N”inpeachmeansthatthegroupdidnotmeettheAMOs.Thetworightmost
columnsshow(1)whetherthatschoolmetAYP(i.e.,itmetthetargetsforitsoverallpopulationandallrequiredsubgroups);and(2)thetotalnumberofstatesinthe
studyforwhichthatschoolmetAYP.Unlikemoststates,Arizonaschoolsconsidereachgradeseparatelywhendeterminingwhethertheminimumnsizeisexceededfor
aparticularsubgroup.ThismeansthatArizonaschoolsmayberequiredtomeetupto18targetsforeachgrade(2targetseach—mathandreading—fortheoverall
population,SWDs,LEP,lowincome,AfricanAmerican,Asian,Hispanic,AmericanIndian,andwhite).Thisis,ofcourse,providedthattherearesufficientnumbersof
studentswithinthegradetoexceedthestate'sminimumnsizeof40ineverysubgroup.(Inactuality,it'smuchhardertoexceedtheminimumnsizewhenindividual
gradelevelsareconsideredversustheschoolasawhole.)Inthistable,forexample,weseethatBarringerChartermettheminimumnsizeforitsoverall,AfricanAmerican,
Hispanic,andlowincomesubgroups.However,topreservespace,eachgradeisnotdisplayedseparately.Consequently,thenumberofAYPtargetsrequiredatBarringer
Charter(48)andthenumberoftargetsmet(47),letusknowthattheschoolfailedtomeetallofitsrequiredsubgrouptargets,butwedon’tknowinwhichgrades.
groups that are separately evaluated for AYP include Theschool-by-schoolfindingsinTables2and3showthat:
SWDs,studentswithLEP,low-incomestudents,andthe
(cid:1) No elementary schools failed to meet their overall
following race/ethnic categories: African American,
targetsformath.
Asian/Pacific Islander, Hispanic/Latino, American In-
dian/AlaskaNative,andWhite.Tables2and3alsoshow (cid:1)Oneelementaryschool(Clarkson)failedtomeetthe
overalltargetforreading.
whetheraschoolmetAYPunderthe2008Arizonarules,
andthetotalnumberofstateswithinthestudyinwhich (cid:1) All middle schools met overall targets for reading
thatschoolmetAYP. andmath.
9 THOMASB.FORDHAMINSTITUTE
Table4.Summaryofsubgroupperformanceofsampleelementaryschoolsunderthe2008ArizonaAYPrules
Numberofschoolswith Numberofschoolswhere Numberofschoolswhere
a SUBGROUP qualifyingsubgroups subgroupfailedtomeetmath subgroupfailedtomeetreading
n target target
o
z Studentswithdisabili"es 1 0 1
i
r StudentswithlimitedEnglish
A proficiency 4 0 1
Low-incomestudents 9 0 1
African-Americanstudents 2 0 0
Asian/PacificIslanderstudents 0 0 0
Hispanicstudents 3 0 2
AmericanIndian/AlaskaNa"ve
0 0 0
students
Whitestudents 15 0 0
Table5.Summaryofsubgroupperformanceofsamplemiddleschoolsunderthe2008ArizonaAYPrules
Numberofschoolswith Numberofschoolswhere Numberofschoolswhere
SUBGROUP qualifyingsubgroups subgroupfailedtomeetmath subgroupfailedtomeetreading
target target
Studentswithdisabili"es 8 7 7
StudentswithlimitedEnglish
3 1 2
proficiency
Low-incomestudents 16 4 3
African-Americanstudents 8 2 2
Asian/PacificIslanderstudents 1 0 0
Hispanicstudents 9 1 1
AmericanIndian/AlaskaNa"ve
0 0 0
students
Whitestudents 15 0 0
(cid:1) One elementary school (Coastal) met every target mentaryandmiddleschools,respectively.Asshown,the
exceptforthereadingtargetforitsSWDs. performance of SWDs is proving most challenging for
schools under Arizona’s system, particularly in middle
(cid:1)Fivemiddleschools(Tigerbear,Kekata,Hoyt,Black schools, where this subgroup tends to have enough stu-
Lake,andZeus)metalltargetsexceptforSWDs.
dentstomeetthestate’sminimumnof40.Infact,every
schoolwithinthesamplewithqualifyingSWDsfailedto
(cid:1) One middle school (Barringer Charter) met every
makeAYP.(However,it’swellworthnotingthatonlyone
targetexceptforoneethnicminoritygroup.
school met the minimum n size for SWD subgroups at
Tables4and5summarizesubgroupperformanceforele- theelementarylevel.)
TheAccountabilityIllusion 10