Table Of Content

Wisconsin Executive Summary has the second highest number of middle schools W makingAYPinthesample(onlyArizonahasmore) i TheintentoftheNoChildLeftBehind(NCLB)Actof s (SeeFigure1). c 2001 is to hold schools accountable for ensuring that o alltheirstudentsachievemasteryinreadingandmath, (cid:1)The high number of schools making AYP inWis- n s withaparticularfocusongroupsthathavetraditionally consinislikelyduetothefactthatWisconsin’spro- i beenleftbehind.UnderNCLB,statessubmitaccount- n ficiency standards are extremely easy compared to ability plans to the U.S. Department of Education de- otherstates,plusitusesaproficiencyindex,which tailing the rules and policies to be used in tracking the meansitgives“partialcredit”tostudentsperform- adequateyearlyprogress(AYP)ofschoolstowardsthese ingbelowproficient. goals. (cid:1)ThefewschoolsinoursamplethatfailtomakeAYP ThisreportexaminesWisconsin’sNCLBaccountability in Wisconsin are meeting expected targets for their system—particularlyhowitsvariousrules,criteria,and overallpopulations2butfailingbecauseoftheperform- practicesresultinschoolseither“makingAYP”—ornot ance of individual subgroups, particularly students making AYP. It also gauges how toughWisconsin’s sys- temiscomparedwithotherstates.Forthisstudy,wese- lected 36 schools from around the nation, schools that vary by size, achievement, and diversity, among other factors,anddeterminedwhethereachwouldmakeAYP underWisconsin’ssystemaswellasunderthesystemsof MoreschoolsmakeAYPin2008underWisconsin’s 27otherstates.Weusedschooldataandproficiencycut accountabilitysystemthaninanyotherstateinour score1estimatesfromacademicyear2005–2006,butap- sample.ThisislikelyduetothefactthatWisconsin’s pliedthemagainstWisconsin’sAYPrules. proficiencystandards(orcutscores)arerelatively easycomparedtootherstates(allofthemarebelow Herearesomekeyfindings: the30thpercentile).Second,Wisconsin’sminimum (cid:1) We estimate that just 1 of 18 elementary schools subgroupsizeforstudentswithdisabilitiesis50, and11of18middleschoolsinoursamplefailedto whichisabitlargerthanmostotherstates(thesize makeAYPin2008underWisconsin’saccountabil- fortheirothersubgroupsiscomparabletoother itysystem. states’).ThismeansthatWisconsinschoolsmust (cid:1)Lookingacrossthe28stateaccountabilitysystems havemorestudentswithdisabilitiesinorderforthat examinedinthestudy,wefindthatWisconsinhas grouptobeheldseparatelyaccountable.Third, thegreatestnumberofelementaryschoolsmaking Wisconsin’s99percentconfidenceintervalprovides AYPinoursample.Inaddition,atseven,Wisconsin schoolswithgreaterleniencythanthemore commonlyused95percentconfidenceinterval.Last, unlikemoststates,Wisconsinmeasuresitsstudent 1Acutscoreistheminimumscoreonastudentmustreceiveon performancewithaproficiencyindex,whichgives NWEA’sMeasuresofAcademicProgress(MAP)thatisequivalentto performingproficientontheWisconsinKnowledgeandConcepts partialcreditforstudentsachieving“partial Examinations-CriterionReferencedTest(WKCE-CRT). proficiency.”Allofthesefactorsworktogetherso 2It’simportanttonotethatstudentsinsubgroupsnotmeetingthe minimumnsizesarestillincludedforaccountabilitypurposesinthe that17outof18elementaryschoolsmakeAYPin overallstudentcalculations;theyaresimplynottreatedastheirown subgroup. Wisconsin,morethananyotherstateinthestudy. 1 THOMASB.FORDHAMINSTITUTE 18 n 16 i YP s A 14 g n n ki o Ma 12 c ols s ho 10 i Sc W ple 8 m a ofS 6 er mb 4 u N 2 0 Massachuse s Nevada Idaho NorthDakota Kansas Washington Wyoming Indiana SouthCarolina Montana Florida Vermont NewJersey NewHampshire Maine NewMexico Delaware Colorado RhodeIsland Georgia Illinois Ohio Minnesota Michigan California Texas Arizona Wisconsin Elementary Schools Middle Schools Figure1.NumberofsampleschoolsmakingAYPbystate Note:MiddleschoolswerenotincludedforTexasandNewJersey;absenceofamiddleschoolbarinthosestatesmeans“notapplicable”asopposedtozero.Stateslike IdahoandNorthDakota,however,havezeropassingmiddleschools. with disabilities (SWDs)3 and students with limited (cid:1)Asinotherstates,middleschoolshavegreaterdiffi- Englishproficiency(LEP).4 cultyreachingAYPinWisconsinthandoelementary schools,primarilybecausetheirstudentpopulations (cid:1)TwosampleschoolsthatfailedtomakeAYPinany are larger and therefore have more qualifying sub- other state made AYP in Wisconsin. Again, this is groups—not because their student achievement is likely due to the fact that Wisconsin’s proficiency anylowerthanintheelementaryschools. standardsarerelativelyeasycomparedtootherstates, alongwiththefactthatthesetwoschoolshadfewer (cid:1) A strong predictor of a school making AYP under accountablesubgroups. Wisconsin’ssystemiswhetherithasenoughSWDs toqualifyasaseparatesubgroup.Almostallschools (cid:1) In Wisconsin, as is the case in most states, schools withqualifyingsubgroupsinthiscategoryfailedto with fewer subgroups attain AYP more easily than meet their AYP targets, particularly at the middle schoolswithmoresubgroups,evenwhentheiraver- school level.5 Ironically, Wisconsin has one of the age student performance is much lower. In other largestminimumnsizesforSWDsinoursample; words, schools with greater diversity and size face still,whenenoughSWDsexisttocompriseasub- greaterchallengesinmakingAYP. group,theydonotperformwell. 3SWDsaredefinedasthosestudentsfollowingindividualizededucationplans. 4Notethatweuse“LEPstudents”and“Englishlanguagelearners”interchangeablytorefertostudentsinthesamesubgroup. 5ItshouldbenotedthatoursubgroupfindingsforLimitedEnglishProficient(LEP)andstudentswithdisabilities(SWDs)maybeslightly morenegativethanwouldbeseenunderrealworldconditions. ThisismostlyduetothedifferencesintestingpracticesbetweenhowLEP studentsandSWDsaretreatedintheWisconsinKnowledgeandConceptsExaminations-CriterionReferencedTest(WKCE-CRT)stateas- sessmentandNWEA’sMeasuresofAcademicProgress(MAP),theassessmentusedinthisstudy. Specifically,theU.S.DepartmentofEducation hasissuedNCLBguidelinespermittingschoolstoexcludesmallpercentagesofLEPordisabledstudentsfromtakingstatetests,orproviding themalternateassessments. Inthecurrentstudy,however,novalidMAPscoreswereomittedfromconsideration. TheAccountabilityIllusion 2 Introduction bystate(asdo,ofcourse,thedifficultyoftheproficiency standards). TheProficiencyIllusion(Croninetal.2007a)linkedstu- W dent performance on Wisconsin’s tests and those of 25 States also determine the minimum number of students i s other states to the Northwest Evaluation Association’s thatmustconstituteasubgroupinorderforitsscorestobe c (NWEA’s) Measures of Academic Progress (MAP), a o analyzedseparately(alsocalledtheminimumn[numberof n computerizedadaptivetestusedinschoolsnationwide. students in sample] size).The rationale is that reporting s Thissinglecommonscalepermittedcross-statecompar- i theresultsofverysmallsubgroups—fewerthan10pupils, n isons of each state’s reading and math proficiency stan- for example—could jeopardize students’ confidentiality dards to measure school performance under the No and risk presenting inaccurate results. (With such small ChildLeftBehind(NCLB)of2001.Thatstudyrevealed groups,randomevents,likeonestudentbeingoutsickon profounddifferencesinstates’proficiencystandards(i.e., testday,couldskewtheoutcome.)Becauseofthisflexibil- howdifficultitistoachieveproficiencyonthestatetest), ity,stateshavesetwidelyvaryingnsizesfortheirsubgroups, andevenacrossgradeswithinasinglestate. fromasfewastenyoungsterstoasmanyas100. Our study expandsonTheProficiencyIllusion by exam- Manystateshavealsoadoptedconfidenceintervals—ba- ining other key factors of state NCLB accountability sicallymarginsofstatisticalerror—toaccountforpoten- plansandhowtheyinteractwithstateproficiencystan- tial measurement error within the state test. In some dards to determine whether the schools in our sample states,thesemarginsarequitewide,whichhastheeffect made adequate yearly progress (AYP) in 2008. Specifi- ofmakingiteasiertoachieveanannualtarget. cally, we estimated how a single set of schools, drawn fromaroundthecountry,wouldfareunderthediffering AlloftheseAYPrulesvarybystate,whichmeansthata rulesfordeterminingAYPin28states(theoriginal25in schoolthatmakesAYPinArizonaorOhio,forexample, The Proficiency Illusion plus 3 others for which we now might not make it under South Carolina’s or Idaho’s have cut score estimates). In other words, if we could rules(U.S.DepartmentofEducation2008). somehow move these entire schools—with their same mix of characteristics—from state to state, how would they fare in terms of making AYP? Will schools with What We Studied high-performing students consistently make AYP? Will Wecollectedstudents’MAPtestscoresfromthe2005– schoolswithlow-performingstudentsconsistentlyfailto 2006academicyearfrom18elementaryand18middle make AYP? If AYP determinations for schools are not schoolsaroundthecountry.WealsocollectedtheNCLB consistentacrossstates,whatleadstotheinconsistencies? subgroupdesignationsforallstudentsinthoseschools— NCLB requires every state, as a condition of receiving inotherwords,whethertheyhadbeenclassifiedasmem- TitleIfunding,toimplementanaccountabilitysystem bers of a minority group, such as students with limited that aims to get 100% of its students to the proficient Englishproficiency(LEP),amongothersubgroups. level on the state test by academic year 2013–2014. In the intervening years, states set annual measurable ob- Theschoolswerenotselectedasarepresentativesample jectives (AMOs).This is the percentage of students in of the nation’s population. Instead, we selected the each school, and in each subgroup within the school schoolsbecausetheyexhibitedarangeofcharacteristics (suchaslowincome6orAfricanAmerican,amongoth- on measures such as academic performance, academic ers),thatmustreachtheproficientlevelinorderforthe growth, and socioeconomic status (the latter calculated school to make AYP in a given year.These AMOs vary bythepercentageofstudentsreceivingfreeorreduced- 6Low-incomestudentsarethosewhoreceiveafreeorreduced-pricelunch. 3 THOMASB.FORDHAMINSTITUTE 35 n 30 i s g 25 n n ki o an 200 c RR e Reading s n l 15 i e Math W Perc 10 5 0 Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Figure2.Wisconsinreadingandmathcutscoreestimates,expressedaspercentileranks(2006) Note:ThisfigureillustratesthedifficultyofWisconsin’scutscores(orproficiencypassingscores)foritsreadingandmathtests,aspercentilesoftheNWEAnorm,in gradesthreethrougheight.Higherpercentileranksaremoredifficulttoachieve.AllofWisconsin’smathcutscoresarebelowthe30thpercentileandallitsreadingcut scoresarebelowthe18thpercentile. pricelunches).Appendix1containsacompletediscus- study.Wisconsin’sminimumsubgroupsizeformostsub- sionofthemethodologyforthisprojectalongwiththe groups is 40, which is comparable to most other states characteristicsoftheschoolsample.7 we examined.8 However, for students with disabilities (SWDs) the minimum is 50, which is a bit larger than ProficiencycutscoreestimatesfortheWisconsinKnowl- mostotherstates. edgeandConceptsExaminations-CriterionReferenced Test(WKCE-CRT)aretakenfromTheProficiencyIllu- Furthermore, although most states examined in the sion (as shown in Figure 2), which found thatWiscon- studyapplyconfidenceintervals(ormarginsofstatistical sin’s definitions of proficiency were generally below error)totheirmeasurementsofstudentproficiencyrates, averagecomparedwiththestandardssetbytheother25 Wisconsin’s 99% confidence interval gives schools states in that study (especially in reading). These cut greater leniency than the more commonly used 95% scores were used to estimate whether students would confidenceintervalusedbymostotherstates.So,forin- havescoredasproficientorbetterontheWisconsintest, stance, although schools are supposed to get 74% of giventheirperformanceonMAP.Studenttestdataand theirgrade3-8students(aswellas74%oftheirgrade3- subgroup designations are then used to determine how 8 students in each subgroup) to the proficient level on these18elementaryand18middleschoolswouldhave the state reading test, applying the confidence interval fared under Wisconsin AYP rules for 2008. In other meansthattherealtargetcanactuallybelower,particu- words,theschooldataandourproficiencycutscorees- larlywithsmallergroups.9 timatesarefromacademicyear2005–2006,butweare applyingthemagainstWisconsin’s2008AYPrules. Unlikemoststates,Wisconsinmeasuresitsstudentper- formance with a proficiency index, which gives partial Table 1 shows the pertinent Wisconsin AYP rules that creditforstudentsachieving“partialproficiency.”Inthe were applied to elementary and middle schools in this short term, the index makes it easier for Wisconsin 7Wegaveallschoolsinoursamplepseudonymsinthisreport. 8Keepinmind,however,thatschoolsizeandnsizearerelated(e.g.,smallnsizesmakesenseforsmallschools). 9Wealsoconductedananalysistoshowtheeffectofconfidenceintervalsonthereadingandmathproficiencyratesforelementaryandmiddle schools.Wedescribethoseresultslaterinthereport. TheAccountabilityIllusion 4 Table1.WisconsinAYPrulesfor2008 Subgroupminimumn Race/ethnicity:40 W SWDs:50 i s Low-incomestudents:40 c o LEPstudents:40 n CI Appliedtoproficiencyratecalcula-ons? s i n Yes;99%CIused AMOs Baselineproficiencylevelsasof2002(index) 2008targets(index) READING/LANGUAGEARTS Grade3 61 74 Grade4 61 74 Grade5 61 74 Grade6 61 74 Grade7 61 74 Grade8 61 74 MATH Grade3 37 58 Grade4 37 58 Grade5 37 58 Grade6 37 58 Grade7 37 58 Grade8 37 58 Sources:U.S.DepartmentofEducation(2008);CouncilofChiefStateSchoolOfficers(2008). Abbreviations:SWDs=studentswithdisabilities;LEP=limitedEnglishproficiency;CI=confidenceinterval;AMOs=annualmeasurableobjectives schools to meet their targets, although the effect of the accesstoonlyasingleacademicyear’sdata(2005–2006), index diminishes as the targets approach 100% profi- wewerenotabletoincludethisinouranalysis.Asare- ciencyrequirementdictatedunderNCLBfor2014.10 sult,itispossiblethatsomeoftheschoolsinoursample that failed to make AYP according to our estimates Note that we were unable to examine the effect of wouldhavemadeAYPunderrealconditions. NCLB’s“safeharbor”provision.Thisprovisionpermits aschooltomakeAYPevenifsomeofitssubgroupsfail Furthermore,attendanceandtestparticipationratesare as long as it reduces the number of nonproficient stu- beyondthescopeofthestudy.Notethatmoststatesin- dentswithinanyfailingsubgroup by atleast10%rela- cludeattendanceratesasanadditionalindicatorintheir tivetothepreviousyear’sperformance.Becausewehad NCLBaccountabilitysystemforelementaryandmiddle 10Insixofthestatesstudied(Massachusetts,Minnesota,RhodeIsland,VermontandNewHampshire,aswellasWisconsin),anindexisused thatgivesfullcredittostudentswhoachieveproficient(orbetter)andpartialcredittostudentsperformingatlowerlevels.Consequently,the resultantscoreinstatesusingthis“hybrid”modelisalwayshigherthantheactualproficiencypercentage(givingstudentspartialcreditforachiev- inglowerproficiencylevelsisobviouslybetterthannocredit,atleastfortheschools’ratings).Theindexprovidesafairamountofhelpwhen annualtargetsarebelow50%;however,oncetargetsriseabove75%,theindexhasfarlessimpact. 5 THOMASB.FORDHAMINSTITUTE 18 15 ol 16 o n h 10 c S i 14 n ns gets 12 5 ncei r a o Ta 10 m sc erof 8 0 erfor b P Wi mum 6 -5 entt N d 4 u -10 St 2 ge a r 0 -15 ve A 1) 1) 1) 7) 5) 3) 7) 7) 5) 9) 1) 2) 3) 7) 8) 0) 8) 4) ( ( ( ( ( ( ( ( ( ( 2 2 ( ( ( 1 2 1 Clarkson Maryweather Few Nemo IslandGrove ohnF.Kennedy Scholls Hissmore WolfCreek Mayberry ayneFineArts( Winchester( Coastal Paramount ForestLake Marigold( Roosevelt( KingRichard( J W TargetsPassed TargetsFailed Average Student Performance Figure3.AYPperformanceoftheelementaryschoolsampleundertheWisconsin2008AYPrules Note:ThisfigureindicateshoweachelementaryschoolwithinthesamplefaredunderWisconsin’sAYPrules(asdescribedinTable1).Thebarsshowthenumberof targetsthateachschoolhastomeettomakeAYPunderthestate’sNCLBrules,andwhethertheymetthem(darkblue)ordidnotmeetthem(lightblue).Themore subgroupsinaschool,themoretargetsitmustmeet.Underthestudyconditions,aschoolthatfailedtomeettheAMOsforevenasinglesubgroupdidn’tmakeAYP,so anylightbluemeansthattheschoolfailed.FewElementary,forexample,meteightofitstentargets,butbecauseitdidn’tmeetthemall,itdidn’tmakeAYP.Schools areorderedfromlowesttohighestaveragestudentperformance(shownbytheorangetriangles),whichismeasuredbytheaverageMAPperformanceofstudents withintheschool;itsscaleisshownontherightsideofthefigure.Scoresbelowzero(whichisthegradelevelmedian)denotebelow-grade-levelperformanceand scoresabovezerodenoteabove-grade-levelperformance.Oneunitdoesnotequalagradelevel;however,thehigherthenumber,thebettertheaverageperformance andthelowerthenumber,theworsetheaverageperformance.Thenumberinparenthesesaftereachschoolnameindicatesthenumberofstates—outof28—inwhich thatschoolwouldhaveAYPinthestudy. schools. In addition, federal law requires 95% of each SeventeenelementaryschoolsmadeAYP,whileonlyone school’sstudents—and95%ofstudentsineachschool’s (FewElementary)failedtomakeit.ThetrianglesinFig- subgroups—toparticipateintesting. ure3showtheaverageacademicperformanceofstudents withintheschool,withnegativevaluesindicatingbelow- Toreiterate,then,AYPdecisionsinthecurrentstudyare grade-levelperformancefortheaveragestudent,andpos- modeledsolelyontestperformancedataforasingleac- itivevaluesindicatingabove-grade-levelperformance. ademicyear.Foreachschool,wecalculatedreadingand mathproficiencyrates(alongwithanyconfidenceinter- Figure 4 illustrates the AYP performance of the sample vals)todeterminewhethertheoverallschoolpopulation middle schools under the 2008 Wisconsin AYP rules. and any qualifying subgroups achieved the AMOs.We Out of 18 middle schools in our sample, 7 made deemed that a school made AYP if its overall student AYP—onelow-performanceschool(Pogesto),whichhas body and all its qualifying subgroups met or exceeded relatively few qualifying subgroups and six higher per- its annual AMOs. Again, Appendix 1 supplies further formingschools(Hoyt,Zeus,OceanView,WalterJones, methodologicaldetail. Artemus,andChaucer). How Did the Sample Schools Fare Figures 5 and 6 indicate the degree to which schools’ math proficiency rates are aided by the confidence in- under Wisconsin’s AYP Rules? terval for elementary and middle schools, respectively. Figure 3 illustrates the AYP performance of the sample On these figures, the darker portions of the bars show elementary schools under Wisconsin’s 2008 AYP rules. theactualproficiencyratesateachschool,andthelighter TheAccountabilityIllusion 6 18 12 ol 16 10 o h W c 8 S ets 1124 6 cein is g n c Tar 10 4 ma o berof 8 02 Perfor ns mm 6 ntt i Nu -2 de n 4 u -4 St 2 -6 ge a r 0 -8 ve A 0) 0) 0) 5) 0) 0) 1) 1) 0) 0) 2) 0) 2) 1) 2) 0) 3) 5) ( ( ( 1 ( ( ( ( ( ( ( ( ( ( ( 2 ( ( McBeal Charterarringer MLAndrew Pogesto( McCord Tigerbear Chesterfield Filmore Barban Kekata Hoyt BlackLake JosephLake Zeus OceanView WalterJones( Artemus Chaucer B Targets Passed Targets Failed Average Student Performance Figure4.AYPperformanceofthemiddleschoolsampleundertheWisconsin2008AYPrules Note:ThisfigureindicateshoweachmiddleschoolwithinthesamplewouldhavefaredunderWisconsin’sAYPrules(asdescribedinTable1).Thebarsshowthenumberof targetsthateachschoolhastomeettomakeAYPunderthestate’sNCLBrules,andwhethertheymetthem(darkblue)ordidnotmeetthem(lightblue).Themoresubgroups inaschool,themoretargetsitmustmeet.Underthestudyconditions,aschoolthatfailedtomeettheAMOforevenasinglesubgroupdoesnotmakeAYP,soanylightblue meansthattheschoolfailed.BlackLake,forexample,met11ofits12targets,butbecauseitdidn’tmeetthemall,itdidn’tmakeAYP.Schoolsareorderedfromlowestto highestaveragestudentperformance(shownbytheorangetriangles),whichismeasuredbyaverageMAPperformanceofstudentswithintheschool;itsscaleisshown ontherightsideofthefigure.Scoresbelowzero(whichisthegradelevelmedian)denotebelow-grade-levelperformanceandscoresabovezerodenoteabove-grade- levelperformance.Oneunitdoesnotequalagradelevel;however,thehigherthenumber,thebettertheaverageperformanceandthelowerthenumber,theworsethe averageperformance.Thenumberinparenthesesaftereachschoolnameindicatesthenumberofstates,outof28,inwhichthatschoolwouldhavemadeAYP. 1 0.9 0.8 0.7 e Rat 0.6 y nc 0.5 e cifi 00.44 o r P 0.3 0.2 0.1 0 Clarkson Maryweather Few Nemo GroveIsland hnF.Kennedy Scholls Hissmore CreekWolf Mayberry ayneFineArts Winchester Coastal Paramount LakeForest Marigold Roosevelt KingRichard o W J MathProficiency Rate MathProficiency RatewithCI MathTarget Figure5.ImpactoftheconfidenceintervalonelementaryschoolmathproficiencyratesundertheWisconsin2008AYPrules Note:Thisfigureshowsthereportedproficiencyrateforthestudentpopulationasawholeandtheimpactoftheconfidenceintervalonmeetingannualtargets.The darkerportionsofthebarsshowtheactualproficiencyrateachieved,whilethelighter(upper)portionsofthebarsshowthemarginoferrorascomputedbythe confidenceinterval. Thefigureshowsthatnoneofthesampleelementaryschoolswasassistedbytheconfidenceinterval.Annualtargets(theorangelines)are consideredtobemetbytheconfidenceintervaliftheyfallwithinthelightblueportion. 7 THOMASB.FORDHAMINSTITUTE 1 0.9 n i 0.8 s 0.7 n e o Rat 0.6 c ncy 0.5 s e Wi icirof 00.44 P 0.3 0.2 0.1 0 McBeal ngerCharter MLAndrew Pogesto McCord Tigerbear Chesterfield Filmore Barban Kekata Hoyt BlackLake LakeJoseph Zeus OceanView WalterJones Artemus Chaucer arri B Math ProficiencyRate MathProficiency RatewithCI MathTarget Figure6.ImpactoftheconfidenceintervalonmiddleschoolmathproficiencyratesundertheWisconsin2008AYPrules Note:Thisfigureshowsthereportedproficiencyrateforthestudentpopulationasawholeandtheimpactoftheconfidenceintervalonmeetingannualtargets.The darkerportionsofthebarsshowtheactualproficiencyrateachieved,whilethelighter(upper)portionsofthebarsshowthemarginoferrorascomputedbythe confidenceinterval.Thefigureshowsthatnoneofthesamplemiddleschoolswasassistedbytheconfidenceinterval.Annualtargets(theorangelines)areconsidered tobemetbytheconfidenceintervaliftheyfallwithinthelightblueportion. portionsofthebarsshowthedegreetowhichthesepro- notindicatewhichsubgroupsfailedorpassedinwhich ficiencyrateswereincreasedbyapplyingtheconfidence school. Tables 2 and 3 list information on individual interval.Theorangelinesshowtheannualtargetneeded subgroup performance for elementary and middle tomeetAYP.Thesefiguresshowthatnoneofthesample schools,respectively. elementaryormiddleschoolswasassistedbythecon- fidenceintervals,becausethemathtargetsinWiscon- Tables2and3showwhichsubgroupsqualifiedforeval- sin are so low, relative to the schools’ overall uation at each school (i.e., whether the number of stu- performance.Thepictureismuchthesameforreading dentswithinthatsubgroupexceededthestate’sminimum proficiency rates at the elementary and middle school n),andwhetherthatsubgrouppassedorfailed.Although levels (not shown). No school is assisted by the confi- all schools are evaluated on the proficiency rate of their denceintervalbecausethereadingtargetsaresolow.In overallpopulation,potentialsubgroupsthatareseparately short,applyingtheconfidenceinterval,eventhoughit evaluated for AYP include SWDs, students with LEP, isalenientone,hasnoeffectonwhetherornotsample low-incomestudents,andthefollowingrace/ethniccat- schoolsmeettheiroverallreadingandmathtargets.11 egories: African American (AA), Asian/Pacific Islander (Asian), Hispanic/Latino (Hispanic), American Where do schools fail? Indian/AlaskaNative(AI/AN),andwhite.Tables2and Figures 3 and 4 illustrate how many subgroup targets 3 also show whether a school met AYP under the 2008 eachsampleschoolisheldaccountable,andwhetheror Wisconsin rules, and the total number of states within not each school made AYP. However, these figures do thestudyinwhichthatschoolmetAYP. 11Inthecurrentanalyses,confidenceintervalswereappliedtoboththeoverallschoolpopulationandtoalleligiblesubgroupsinoursample schools.Thus,theultimateimpactoftheconfidenceintervalmaybelargerthantheimpactdepictedinFigures5and6.However,wechosenot toshowhowtheconfidenceintervalimpactedsubgroupperformancebecauseitwouldhaveaddedgreatlytothereport’slengthandcomplexity. TheAccountabilityIllusion 8 Table2.Elementarysubgroupperformanceofsampleschoolsunderthe2008WisconsinAYPrules SPCSHEUODOOLNYM MatOverallh ProficiencyRRateeading MOverallR MSWDsR MLEPStudentsR MLow-incomeStudentsR MAAR MAsianR MHispanicR MAI/ANR MWhiteR AYPTargetsRequired TargetsMET %ofTargetsMet SchoolMetAYP? NumberofstatesinwhichschoolmetAYP? Wiscons i Clarkson 64.5% 79.6% Y Y Y Y Y Y Y Y 8 8 100% Y 1 n Maryweather 67.4% 79.0% Y Y Y Y Y Y Y Y Y Y 10 10 100% Y 1 Few 73.7% 80.6% Y Y Y N Y N Y Y Y Y 10 8 80% N 1 Nemo 76.0% 91.2% Y Y Y Y Y Y 6 6 100% Y 7 IslandGrove 79.0% 89.3% Y Y Y Y Y Y Y Y 8 8 100% Y 4 JFK 81.8% 88.6% Y Y Y Y Y Y Y Y 8 8 100% Y 3 Scholls 86.0% 91.4% Y Y Y Y Y Y Y Y 8 8 100% Y 7 Hissmore 85.9% 92.2% Y Y Y Y Y Y Y Y 8 8 100% Y 7 WolfCreek 77.8% 90.0% Y Y Y Y Y Y Y Y 8 8 100% Y 5 AliceMayberry 86.6% 93.4% Y Y Y Y Y Y Y Y 8 8 100% Y 9 WayneFineArts 87.6% 97.7% Y Y Y Y 4 4 100% Y 21 Winchester 83.5% 95.0% Y Y Y Y 4 4 100% Y 22 Coastal 88.1% 90.8% Y Y Y Y Y Y Y Y Y Y Y Y Y Y 14 14 100% Y 3 Paramount 84.9% 90.7% Y Y Y Y Y Y Y Y 8 8 100% Y 7 ForestLake 93.3% 96.6% Y Y Y Y Y Y 6 6 100% Y 8 Marigold 94.6% 96.2% Y Y Y Y Y Y 6 6 100% Y 10 Roosevelt 96.6% 99.2% Y Y Y Y 4 4 100% Y 28 KingRichard 93.1% 97.3% Y Y Y Y Y 5 5 100% Y 14 Abbreviations:M=math;R=reading;N=no;Y=yes;SWDs=studentswithdisabilities;AA=AfricanAmerican;Asian/PacificIslander=Asian;Hispanic/Latino= Hispanic;AmericanIndian/AlaskaNative=AI/AN. Note:Schoolsareorderedfromlowest(Clarkson)tohighest(KingRichard)averagestudentperformanceasmeasuredbycombinedandweightedmathandreading performanceontheMAPassessment(notshownintable).Ablankspaceunderneathasubgroupmeansthatsubgroupcontainedfewerthantheminimumnumberof studentsrequiredforevaluation,soitwasn’tcounted.A“Y”inbluemeansthatthegroupmettheAMOsandan“N”inpeachmeansthatthegroupdidnotmeettheAMOs. Thetworightmostcolumnsshow(1)whetherthatschoolmetAYP(i.e.,itmetthetargetsforitsoverallpopulationandallrequiredsubgroups);and(2)thetotalnumber ofstatesinthestudyforwhichthatschoolmetAYP. Theschool-by-schoolfindingsinTables2and3showthat: Indian,andwhitesubgroupsmetboththeirreading andmathtargets. (cid:1)Allschoolsmetboththeirmathandreadingtargets Tables4and5summarizesubgroupperformanceforel- fortheiroverallstudentpopulations. ementary and middle schools, respectively. First, there areveryfewqualifyingSWDandLEPsubgroupsatthe (cid:1) Nine of the 11 failing middle schools only missed elementarylevel(keepinmindthattheminimumnsize targetsforthestudentswithdisabilitiessubgroup. forSWDsisratherlargeat50).Butwhentherearelarge (cid:1) One middle school (Kekata) failed to make AYP enough numbers of these students to comprise sub- onlybecauseofitsLEPsubgroup. groupsatthemiddleschoollevel,theytendtostruggle. In fact, one of the two elementary schools and most of (cid:1)Unlikeanyotherstateinthestudy,allofthelow-in- the middle schools in the study that have qualifying come,AfricanAmerican,Hispanic,Asian,American SWD subgroups fail to make AYP. Students with LEP 9 THOMASB.FORDHAMINSTITUTE Table3.Middleschoolsubgroupperformanceofsampleschoolsunderthe2008WisconsinAYPrules isconsin SPCSHEUODOOLNYM MatOverallh ProficiencyRRateeading MOverallR MSWDsR MLEPStudentsR MLow-incomeStudentsR MAAR MAsianR MHispanicR MAI/ANR MWhiteR AYPTargetsRequired TargetsMET %ofTargetsMet SchoolMetAYP? NumberofstatesinwhichschoolmetAYP? W McBeal 74.9% 80.2% Y Y N N N N Y Y Y Y Y Y Y Y Y Y 16 12 75% N 0 BarringerCharter 74.7% 88.5% Y Y N N Y Y Y Y Y Y 10 8 80% N 0 MLAndrew 78.0% 89.1% Y Y N N Y Y Y Y Y Y Y Y 12 10 83% N 0 Pogesto 85.2% 92.6% Y Y Y Y 4 4 100% Y 15 McCordCharter 79.1% 89.5% Y Y N Y Y Y Y Y Y Y Y Y 12 11 92% N 0 Tigerbear 83.7% 86.4% Y Y Y N Y Y Y Y Y Y 10 9 90% N 0 Chesterfield 86.4% 89.9% Y Y Y N Y Y Y Y Y Y 10 9 90% N 1 Filmore 87.2% 92.3% Y Y N Y Y Y Y Y Y 9 8 89% N 1 Barban- 82.0% 87.7% Y Y N N Y Y Y Y Y Y Y Y 12 10 83% N 0 Kekata 88.7% 90.1% Y Y Y Y Y N Y Y Y Y Y Y Y Y 14 13 93% N 0 Hoyt 90.3% 91.5% Y Y Y Y Y Y Y Y Y Y 10 10 100% Y 2 BlackLake 91.1% 91.6% Y Y Y N Y Y Y Y Y Y Y Y 12 11 92% N 0 LakeJoseph 89.2% 92.8% Y Y Y N Y Y Y Y Y Y Y Y 12 11 92% N 2 Zeus 91.7% 91.9% Y Y Y Y Y Y Y Y Y Y Y Y Y Y 14 14 100% Y 1 OceanView 92.0% 96.0% Y Y Y Y Y Y Y Y Y Y 10 10 100% Y 2 WalterJones 90.7% 94.3% Y Y Y Y Y Y 6 6 100% Y 20 Artemus 93.9% 93.9% Y Y Y Y Y Y Y Y 8 8 100% Y 3 Chaucer 95.3% 97.2% Y Y Y Y Y Y Y Y Y Y Y Y Y Y 14 14 100% Y 5 Abbreviations:M=math;R=reading;N=no;Y=yes;SWDs=studentswithdisabilities;AA=AfricanAmerican;Asian/PacificIslander=Asian;Hispanic/Latino= Hispanic;AmericanIndian/AlaskaNative=AI/AN. Note:Schoolsareorderedfromlowest(McBeal)tohighest(Chaucer)averagestudentperformanceasmeasuredbycombinedandweightedmathandreading performanceontheMAPassessment(notshownintable).Ablankspaceunderneathasubgroupmeansthatsubgroupcontainedfewerthantheminimumnumberof studentsrequiredforevaluation,soitwasn’tcounted.A“Y”inbluemeansthatthegroupmettheAMOsandan“N”inpeachmeansthatthegroupdidnotmeettheAMOs. Thetworightmostcolumnsshow(1)whetherthatschoolmetAYP(i.e.,itmetthetargetsforitsoverallpopulationandallrequiredsubgroups);and(2)thetotalnumber ofstatesinthestudyforwhichthatschoolmetAYP. arealsostrugglingsomewhattomeetthestate’stargets; states—21and20,respectively.Andtheseschoolsmade three schools with a large enough LEP population to AYPinWisconsin,too. qualify as a separate subgroup fail to meet targets for thesestudents. But Wisconsin is also home to a few anomalies. First, considerClarksonandMaryweatherelementaryschools Characteristics of Schools (seeTable2).TheyeachfailedtomakeAYPin27ofthe 28statesinoursample,yetmadeAYPinWisconsin.In that Did and Didn’t Make AYP examiningTable2,wecanseethatClarksonandMary- AcloselookatFigures3and4indicatesthatWisconsin’s weather didn’t meet the minimum numbers for the NCLB accountability system is, at least somewhat, be- SWD subgroup, which create difficulty for so many having like those in other states. For example, Wayne other schools in the study. Without fewer accountable Fine Arts and Walter Jones made AYP in many of the subgroupsandeasyproficiencystandards(seeFigure2), TheAccountabilityIllusion 10