Table Of Content

G R E A T P L A I N S I N S T I T U T E O F R E A D I N G A N D W R I T I N G C O L L E G E O F E D U C A T I O N A N D H U M A N S C I E N C E S U N I V E R S I T Y O F N E B R A S K A L I N C O L N 2 0 0 7 - 0 8 A N N UA L R E P O R T NEBRASKA READING FIRST Emily Hayden M.S. Guy Trainin Ph.D. Kristin Javorsky B.A. Malinda Murphy-Yagil Ph.D. Kathryn Cook B.A NEBRASKA READING FIRST- ANNUAL REPORT 2007-08 OVERVIEW The 2007-2008 Annual Progress Report provides an overview of student achievement and teacher implementation in Nebraska Reading First schools. The report begins with a description of the demographic characteristics in Reading First classrooms, and compares them to the state population. Next, in an effort to understand the impact of student mobility on academic achievement, we examine spring and fall scores for mobile and stable students in 2007-08. A description of cluster characteristics concludes Section One of the report. Section Two presents year-end academic achievement. Performance on significant tests and student risk level changes are reported. Achievement gap information compares the progress of specific categories of students. Vocabulary and comprehension for first, second, and third grade students are included in this section as well. Section Three explores the achievement of special education students. Performance and growth for these students over the years of Reading First implementation is explored, and we discuss the unique challenges of measuring this population. Section Four provides a closer look at the grade level performance of students in Reading First schools. The impact of Reading First on overall achievement is explored in two individual districts by looking at performance on district norm referenced STARS assessments, from the year prior to Reading First implementation through spring of 2007-08. Teacher surveys and teacher log results are presented. These provide insight into the instructional procedures, focus, and teacher experiences in Reading First classrooms at every grade level. Finally, to understand the experience of students in Reading First, we track the assessment achievements of three students who have spent all four of their school years in Reading First classrooms. 2 NEBRASKA READING FIRST- ANNUAL REPORT 2007-08 SECTION 1 STUDENT CHARACTERISTICS STUDENT DEMOGRAPHIC CHARACTERISTICS Some demographic groups experienced noticeable changes during 2007-2008. Round one schools displayed an increase of more than 10% in students qualifying for free/reduced lunch, while Round two schools increased in the number of English Language Learner students by more than 8%. All Nebraska Reading First schools report much larger percentages of students qualifying for free/ reduced lunch than the state average of 36.4%, and mostly larger percentages of students identifying with the three non-white minority categories. Nebraska Reading First schools are educating students that are more culturally diverse and economically disadvantaged than the rest of the state. State Round I Round II 2006- 2006- 2007- 2006- 2007- 2007 2007 2008 Change 2007 2008 Change English Learners 6.5% 4.7% 5.3% 0.6% 14.3% 22.9% 8.6% Special Education 15.0% 7.1% 12.8% 5.7% 3.9% 10.8% 6.9% Free/Reduced Lunch 36.4% 48.3% 58.5% 10.2% 56.6% 56.1% -0.5% African American 7.7% 23.0% 25.7% 2.7% 27.5% 28.0% 0.5% Hispanic 12.2% 14.3% 15.7% 1.4% 27.8% 28.0% 0.2% Native American 1.7% 2.2% 3.1% 0.9% 1.3% 1.3% 0.0% White (non- Hispanic) 77.0% 59.8% 54.3% -5.5% 42.6% 42.0% -0.6% *Numbers may not add to 100% due to rounding and overlapping categories **State percentages were taken from the 2006-07 NDE report MOBILITY For our evaluation of spring assessment data we take into account the mobility of students during the school year. Students are considered stable for the year end report if they were present for spring assessment and at least one of the previous assessment rounds. Those students who missed two assessment rounds during the year are considered mobile and are dropped from the year end reporting database. We include only the achievement data for the students who have been stable in our aggregate reporting of year-end results. For 2007-08 spring assessment 93% of students at all grade levels were stable. We examined the link between mobility of Reading First students and the achievement data. We examined the scores of students who were present for fall 2007-08 assessment but left Reading First before spring assessment and compared them to the fall scores of students who were stable in the spring. At all grade levels, 7% to 9% of students present in the fall left before spring assessment. 3 NEBRASKA READING FIRST- ANNUAL REPORT 2007-08 These fall mobile students scored significantly lower on the fall assessments than their peers who remained for the rest of the year. The differences were most pronounced at second grade, where students who were present only for the fall round achieved fall oral reading fluency scores that were nearly .5 standard deviation lower than the scores of their peers who were stable at the end of the year. Students who became mobile before the end of the year started the year at significantly lower achievement levels than their peers who did not move. In all grades, the mobile students who were present for only spring assessments during the 2007-08 year scored significantly lower (.6 to .7 standard deviation lower—a moderate effect size) than those stable students who were present for spring and at least one of the previous assessment rounds. High mobility has a significant effect on spring academic performance, and the pattern of mobility is linked to student achievement in Reading First schools that is apparent even at the beginning of the year. While the data on mobile students is not included in the reports, schools should go to extra lengths to insure that students coming in are supported as quickly as possible, and these students should be supported in following years so they can catch up to their peers. STUDENT POPULATION BY CLUSTER As in previous years, Reading First schools were grouped into clusters for analysis. Cluster analysis allows comparisons among schools that face similar challenges. Nebraska Reading First schools are clustered according to school size, minority group proportion, percentage of English Language Learners, percentage of students eligible for free/reduced lunch, and percentage of students qualified for special education. Cluster 1 includes the smallest schools, where approximately 55% Cluster 1: smallest schools, 55% F/RL, 23% minority, of students are eligible for free/reduced lunch, 25.6% belong to 7% ELL, 16% special minority populations, 7.8% are English Learners (ELL), and 12% education qualify for special education services. Cluster 2 schools are larger, Cluster 2: somewhat larger with fewer students qualifying for free/reduced lunch, slightly schools, slightly smaller smaller percentages of minority and English Learners (ELL) and percentage of F/RL, minority, ELL, special slightly more special education students. Cluster 3 has the largest education schools, with much higher percentages of minority students (89%) Cluster 3: largest schools, and more students qualifying for free/reduced lunch (67%). In higher percentages of all this cluster 20% of the students qualify for ELL services and 9.8% diversity groups qualify for special education. 4 NEBRASKA READING FIRST- ANNUAL REPORT 2007-08 The following figure shows current population characteristics for each cluster. As noted earlier, the number of minority students varies dramatically across clusters. There is also noticeable variability in the percentages of English Learners (ELL) across the three clusters. All clusters report more than 50% of their students qualify for free/reduced lunch, a percentage that is well above the state average. Percentage of ELL, Minority, FRL and Special Education per Cluster 89.5% ELL 100% SpEd 67.6% 80% FRL 55.4% 60% 50.1% Minority 9.8% 40% 25.6% 22.4% 20.1% 12.2% 14.5% 20% 7.8% 1.5% 0% Cluster 1 Cluster 2 Cluster 3 5 NEBRASKA READING FIRST- ANNUAL REPORT 2007-08 SECTION 2 STUDENT ACHIEVEMENT The figure below presents percentages of students on grade level for each grade over the last two years. While each academic year represents a substantially different cohort of students a pattern of continuing improvement is apparent. The black line indicates the national average. Percentage of Students at Grade Level 2007-08 compared to last year 2006-07 100% 2007-08 90.1% 90% 84.1% 80% 72.1% 71.2% 69.6% 70% 64.8% 62.7% 62.9% National 60% Average 50% Kindergarten First Grade Second Grade Third Grade (NWF) (ORF) (ORF) (ORF) Significant Test The significant test for the end of kindergarten, nonsense word fluency (NWF), is a measure of students’ ability to apply phonics rules and blend sounds within word forms. Because the task uses pseudo words and there is no element of context to support the word reading, decoding is a relatively pure measure of skill in phonological processing. A higher percentage of students who completed kindergarten in 2007-08 scored on grade level for this measure than in the previous year, and classes in both years scored well above the national average for this skill. This is a strong predictor for of success in first grade, when students will need to use phonics to decode words quickly during text reading. Once students become proficient at decoding words in isolation, first grade teachers shift their instructional emphasis to developing oral reading fluency (ORF). The ability to read connected text smoothly, accurately, and at an appropriate pace significantly predicts success in reading comprehension. For the last two years more than 70% of first graders were at grade level for this measure: a figure considerably higher than the national average. 6 NEBRASKA READING FIRST- ANNUAL REPORT 2007-08 Developing oral reading fluency remains a challenge as students move through grade levels and encounter increasingly complex texts. This challenge is evident in the percentage of second and third graders who are at grade level. More Reading First students achieve grade level than the national average, and there is a small but consistent growth in oral reading fluency across first, second, and third grade since last year. Establishing strong oral reading fluency skills is a challenge that Reading First teachers across the state continue to face. The following sections present a detailed look at the progress in each grade level in 2007-08. KINDERGARTEN A significant task during the kindergarten year is the mastery of foundational skills for later word decoding. One of the earliest of these skills is letter knowledge (measured by LNF). Proficiency in letter naming facilitates letter-sound match skills Kindergarten Achievement (LNF) Comparison that contribute to fast and between Round I and Round II Schools accurate blending of sounds 100 within words. The figure to the left shows 80 the progress of kindergarten e students in Reading First ro 60 c S schools on Letter Naming n a 40 Fluency. A score at or e M Round 1 above 40 on letter 20 knowledge in the spring Round 2 indicates a low level of risk 0 Fall Winter Spring for difficulty, and kindergarten groups in both Rounds 1 and 2 achieved average scores above this cutoff. On average students in both rounds advanced at the same rate. Once students master letter naming they move on to word based skills such as Phoneme Segmentation Fluency, a measure of the ability to isolate and manipulate individual sounds within short words, and Nonsense Word Fluency (NWF), a task that requires students to apply phonics rules and blending to nonsense words At risk Some risk Low risk without the benefit of context. It is this measure (NWF) that is used to Nonsense Word 0-14 15-24 25 or assess the level of student risk for Fluency score at the greater reading difficulty at the end of the end of Kindergarten kindergarten year. 7 NEBRASKA READING FIRST- ANNUAL REPORT 2007-08 Scores for each level of risk on Nonsense Word Fluency are established in the Dynamic Indicators of Basic Early Literacy Skills (DIBELS) framework. These scores are listed in the table above. Since skill in decoding (NWF) provides a basis for decoding words within connected text, developing this skill is very important for success in first grade and beyond. Kindergarten teachers in Reading First classrooms make this skill a priority as Changes in Kindergarten Risk Levels the year progresses. In the fall of 2007-08, 125% nearly half of all kindergarten students were 100% Low either at risk or at some risk Risk for reading difficulty, as 51.8% 75% Some indicated by the red and 73.9% Risk 90.1% yellow sections of the bars 50% At Risk 20.3% in the figure at right. 25% 15.4% By the spring of this year, 27.9% 10.7% 6.8% 90% of students in Reading 3.2% 0% First kindergarten Fall LNF Winter PSF Spring NWF classrooms were on grade Significant Test level for Nonsense Word Fluency. This indicates remarkable skill growth during 2007-08, and should provide a sound basis for further word decoding skills. KINDERGARTEN ACHIEVEMENT GAPS At the end of Kindergarten Performance By Subcategory the 2007-08 ELL l100% school year e v e 80% SpEd only 10% of L kindergarten e d 60% F/RL a students r G 40% remained at Minority t a some level of % 20% No risk for Subcategory 0% difficulty with Fall (LNF) Winter (PSF) Spring (NWF) reading. To Significant Test further examine these results, 8 NEBRASKA READING FIRST- ANNUAL REPORT 2007-08 performance is broken down by categories of students that have a higher risk for reading difficulties: students who are English language learners, those qualifying for special education services or for free/reduced lunch, and students from minority backgrounds. For purposes of comparison we added group comprised of mainstream students who do not fall into any of these categories. The previous figure shows the percent of students within each of these groups who are performing at grade level at the end of their 2007-08 kindergarten year. Students qualifying as English Language Learners started out with the smallest percentage on grade level in the fall. This group struggled in fall and winter of this year but posted a positive and encouraging gain in grade level skill by the end of the year, ending with a slightly higher percentage of students on grade level than the non-category student group. Minority students and those qualifying for free/reduced lunch also finished near the 90% mark. The category with the lowest percentage of grade level achievement is the group of students who qualify for special education services. However, these students improved dramatically during the course of the year, from 39% performing on grade level in the fall to 73% in the spring. The following figures look at each of these category groups individually in comparison to all other students for each of the four years of Reading First implementation. Caution in interpreting these results is needed since each year of implementation presents a different group of students. Because some of the groups are fairly small, fluctuations are not Kindergarten ELL Gaps uncommon. Overall trends in 100% achievement are le v represented in these e 75% L figures. The lower e d a line of the fill area rG 50% represents the t a t achievement of the n ec 25% subcategory group. r e P The solid line above 0% the colored fill area Fall 04-05 Spring 04-05 Spring 05-06 Spring 06-07 Spring 07-08 represents the average percentage of all other students on grade level, and the colored fill area illustrates the gap between the two groups of students. In the figure above it can be seen that English Learners (ELL) performed at a markedly lower level of achievement in the baseline assessment in 2004-05. That gap has progressively tapered in the following years, with English Learners (ELL) surpassing their grade level peers from all other 9 NEBRASKA READING FIRST- ANNUAL REPORT 2007-08 categories in the spring of 04-05, and performing at a slightly higher level than all other categories at the end of this 2007-08 school year. The gap between students qualifying Kindergarten FRL Gaps for free/reduced lunch and all other students has also 100% narrowed over the le v years of Reading e 75% L e First d a implementation, rG 50% t although these gaps a t were not as n e 25% c pronounced in the r e P beginning as those 0% for English Fall 04-05 Spring 04-05 Spring 05-06 Spring 06-07 Spring 07-08 Learners (ELL). In 2007-08 students who receive F/RL achieved at a rate nearly equal to that of their non-F/RL peers, finishing the year with only a 4% difference in percentage of students on grade Kindergarten Ethnicity Gaps level. In the last two 100% years of Reading le v First e 75% L implementation, ed a the gap between rG 50% kindergarten ta t minority students n e 25% c and their peers re P has closed. 0% Children in the Fall 04-05 Spring 04-05 Spring 05-06 Spring 06-07 Spring 07-08 minority categories outperformed the combined group of all other kindergarten children at the end of the 2006-07 year and at the end of 2007-08. 1 0