ebook img

ERIC ED610748: An Evaluation of the Teaching English Learners Early Mathematics (TEEM) Program; an Investing in Innovations (i3) Development Grant PDF

2021·1.5 MB·English
by  ERIC
Save to my drive
Quick download
Download
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.

Preview ERIC ED610748: An Evaluation of the Teaching English Learners Early Mathematics (TEEM) Program; an Investing in Innovations (i3) Development Grant

2021 An Evaluation of the Teaching English Learners Early Mathematics (TEEM) Program; An Investing in Innovations (i3) Development Grant (U411C140073) IMPLEMENTATION YEARS 1-4 FINAL REPORT PREPARED BY: DR. LORIE A. SOUSA WITH DR. MADELEINE JETTER (CSUSB) Table of Contents Executive Summary 2 Introduction 6 Method 10 Results 18 Key Findings 52 Conclusions 54 References 56 Appendices 61 Logic Model Fidelity of Implementation Table Impact Contrast Table 1 Executive Summary California State University, San Bernardino (CSUSB) and the Riverside County Office of Education (RCOE) were awarded an Investing in Innovation (i3) Development grant in 2015 to investigate the impact of an early education mathematics program on English Learner students at two rural school districts in Riverside, CA. COREducational, an evaluation and research consultancy, conducted the external evaluation of the findings over a four-year period from 2015-2016 to 2018-2019 school years. A no cost extension was approved for the final and fifth year (2019-2020). However, the COVID pandemic resulted in school closures that precluded data collection. Thus, the decision was made to report on the four complete years of data collected. Project Activities and their relationship to project goals: Teaching English Learners Early Mathematics (TEEM) had five major goals: to improve the academic achievement of English Learners (ELs) (Goal 1); to improve teacher knowledge and beliefs for teaching math to ELs (Goal 2); to improve the teaching of math to ELs and all students (Goal 3); to develop principals as instructional leaders in mathematics (Goal 4); and to build a learning partnership that develops, sustains and shares a model of instructional change supporting ELs. (Goal 5). In support of these goals, a combination of four mutually supportive interventions were implemented. Intervention 1. Intensive Teacher Professional Development (PD) Activities Summer Intensive Institutes were held every summer. The institutes addressed math content, best practices for teaching English Learners, and interactive notebooking. Over the course of the project, the mathematical focus of the Institutes progressed through the major themes of elementary mathematics, including whole numbers and algebraic thinking, multiplicative thinking, geometry, fractions, proportional reasoning and expressions and equations. The Common Core State Standards for Mathematical Practice were emphasized throughout the professional development. The use of multiple representations, to support understanding for English Learners and all students, was another focus of attention. Participants received curriculum units from the Contexts for Learning Mathematics curriculum as a resource to support them in teaching through problem solving. TEEM staff, including CSUSB faculty and Riverside County Office of Education (RCOE) specialists, presented the PD. Lesson Study experts were also brought in to cover special topics such as Teaching Through Problem Solving. Intervention 2. Lesson Study Each year, participants worked in grade-level groups of three to eight members each to complete two three-day cycles of lesson study in each academic year of the project. In a lesson study cycle, teams set shared instructional goals; studied relevant content, pedagogy and formative assessment data; and planned, taught, observed and reflected on a mathematics lesson. As a result of participation in the California Action Network for Mathematics Excellence and Equity (CANMEE), lesson study participants began crafting equity goals (in addition to mathematical goals) for the 2 classrooms in which lesson study is taking place. An example of an equity goal is “Students see themselves as powerful mathematical thinkers through small-group and whole-classroom discourse.” Intervention 3. Interactive Math Notebooks for participating students Each participating classroom teacher received a class set of composition books at each TEEM Summer Institute. Notebooks are intended to contain a record of the teacher’s teaching and the student’s learning: “input” pages on the right hand side include note-taking, practice, or other elements directed by the teacher, while “output” pages on the left provide opportunities for students to engage more actively with the material through problem solving, summarizing or reflection on learning. Project staff rated a sample of student notebooks each year using a rubric. Intervention 4. Professional Learning for school leaders Site principals participated in several activities designed to support them in understanding the curriculum shifts described in the Common Core State Standards for Mathematics. TEEM staff provided 30 to 45-minute presentations at monthly district-based principals’ meetings (six at each district). These presentations addressed the practices and strategies implemented in TEEM to aid principals in understanding what they would expect to see in observations of TEEM classrooms. Topics included Lesson Study, mathematics notebooking, and the importance of multiple representations in mathematics learning, among others. All principals participated in classroom walkthroughs with TEEM staff. In one of the districts, each site principal hosted one walkthrough attended by all district principals and TEEM staff. In the other district, the two principals walked classrooms at both sites twice a year. The 5x8 Card developed by the Strategic Education Research Partnership (SERP) was used to facilitate observation and discussion of mathematics classroom practice. This led to more productive conversations with principals about mathematics teaching and learning. Independent Evaluation and preliminary findings: The Independent Evaluation of the TEEM Project was conducted by TEEM’s independent evaluator COREducational, headed by Dr. Lorie Sousa. Below is a summary of results. TEEM Teachers demonstrated growth in mathematics content knowledge, as measured in the Learning Mathematics for Teaching (LMT) Assessment. They have showed improving self-efficacy beliefs relative to a comparison group. Participating teachers’ mathematics content knowledge improved relative to a comparison group of non-participating teachers in each project year (although due to sample size, the difference was not statistically significant in the last two project years). 3 The Mathematics Teaching Efficacy and Expectancy Beliefs Instrument (MTEEBI) was used to measure teachers’ beliefs about developing mathematical practices in a diverse student population. Teachers are asked to rate their level of agreement with statements like “I am able to help students from impoverished backgrounds excel in math.” Participating teachers’ beliefs improved relative to a comparison group to a statistically significant degree, showing that participation in TEEM had a measurable, positive impact on teachers’ beliefs -- both in their own teaching abilities and in their students’ ability to learn mathematics. Classroom observations were performed each project year using a project-specific rubric. These results were analyzed in the evaluation report and demonstrated some significant positive shifts in teacher classroom practices. For example, the percentage of classrooms in which “English Learners produce language that communicates ideas and reasoning, even when that language is imperfect” increased from 48% in Spring 2016 to 80% in Spring 2018. Student outcomes were tracked using state-administered Smarter Balanced Assessment (SBA) for grades 3 and higher. Because students do not take these tests prior to grade 3, we used Assessing Math Concepts (AMC) interviews for grades PK-2. In grades 3 and above, students taught by a TEEM teacher for at least one year outperformed students in a comparison group in both the Mathematics and English Language Arts SBA tests. This was true both for English Learners and the general student population. Comparison data was not available for AMC interviews, so our evaluator compared participating students’ results at the beginning and end of each project year and found significant improvements in every section of the assessment. Extent to which objectives, expected outcomes and annual targets were achieved: Based upon the evaluation data, TEEM achieved many of its proposed outcomes for teachers and students. Due to participant attrition, participation levels were below the proposed target of 75 teachers participating for all four years. Some objectives for site principals could not be achieved as proposed: for example, principal attendance at the TEEM Summer Institute remained well below target in spite of an increase in Y4. Sustainability planning and dissemination: The TEEM Teacher Leadership Team consisted of eight teachers interested in developing themselves as math leaders and disseminating best practices learned in TEEM. Four bimonthly Leadership Team seminars were held in Y4. The Project director lead each meeting using the Learning to Lead Mathematics Professional Development curriculum by Carroll and Mumme, and participant feedback was very positive. Based on teacher interest, Leadership Team teachers provided activities and material kits to support Family Math nights at school sites. The leadership team also went on to plan and present an online dissemination workshop attended by approximately 30 local PK- TK-Kindergarten teachers at the close of the project. 4 Project dissemination goals were met. The Project Director gave two presentations at regional conferences discussing the math notebooking practice in TEEM classrooms. The first of these was done in collaboration with a project teacher at the Riverside County MaTHinK conference; the other was to the California Math Council Southern Section Conference. An article submission to NCTM on the same topic was not accepted for publication. Alternate publication venues are being considered to disseminate information regarding the notebooking practice and results of the study. Challenges, best practices and lessons learned: Attrition. Teacher attrition was a challenge. In the hope of compensating for attrition, TEEM received permission to invite new teachers to participate in lesson study beginning in Fall of 2018. As a result,10 new teachers were included in TEEM lesson study in September 2018. Substitute teacher availability. Occasionally, a school district partner was short on substitute teacher coverage on a TEEM lesson study day. In these situations, administrators did their best to enable teachers to keep their lesson study meeting (for example, by splitting classes into other teachers’ classrooms), however on some occasions district administrators cancelled a day of TEEM lesson study. This was difficult for teachers and facilitators alike. TEEM attempted to mitigate this problem by scheduling lesson study well in advance. Lesson study. Work with the CANMEE Lesson Study Support Hub provided insight into best practices for mathematics lesson study. The use of equity goals and an equity commentator also encouraged a deeper look into student learning, causing teachers to slow down and think specifically about targeted students including English learners or those with special needs. Impact beyond the partners and implementation sites: Lesson study. Through the CANMEE Lesson Study Support Hub, TEEM staff support lesson study across our region as a vehicle to promote equitable and rigorous mathematics instruction. Further, educators involved in the project have developed their own skills and capacity with respect to the lesson study practice. 5 Introduction The Teaching English Learners Early Mathematics (TEEM)program was designed by, California State University, San Bernardino (CSUSB) and the Riverside County Office of Education (RCOE), in conjunction with two Local Education Agency (LEA) partners, and the Head Start and State Preschool programs affiliated with one of the LEAs. The purpose was to build on a history of close collaboration to develop, implement and evaluate a replicable model for improving the academic achievement of English Learner (EL) students in pre-K through grade 6. TEEM program intentions were to increase the number and proportion of ELs completing challenging courses in core academic subjects through a novel combination of interventions designed to support communities of inquiry among students, teachers, and school leaders. Students engaged in structured inquiry through the use of interactive notebooks in mathematics; teachers sought to improve mathematics instruction through the formation of communities of inquiry supported by Summer Institutes and an adaptation of Japanese Lesson Study; and principals were developed as instructional leaders in mathematics through a variety of supports. The project intended to directly impact at least 4,000 students, 75 teachers and 7 principals. The primary goal was to increase the number and proportion of ELs successfully completing challenging courses in core academic subjects by implementing structured inquiry-based approaches to learning, teaching and school leadership. These approaches were intended to develop collective efficacy in schools and provide ELs access to challenging math content from pre-K through grade 6. Importance of math in early grades. To maximize long-term impact, math-focused interventions were developed for ELs in early grades. The risk of dropout is roughly twice as high for ELs as it is for non-ELs (Callahan, 2013), and in many cases the dropout problem is a math problem. In the Los Angeles Unified School District, students who passed Algebra I by grade 9 were twice as likely to graduate as those who had not (Silver, 2008). In order to improve academic outcomes through high school, ELs must succeed in challenging math courses, and they must begin early. Academic performance in early grades is a strong indicator of future dropout risk, and high-quality preschool decreases the risk (Rumberger & Lim, 2008). In California, achievement gaps existing in grade 2 largely persist through grade 7, a “pivot year” in which student achievement varies widely and determines future opportunities in math (Terry and Rosin, 2011). In California, ELs are a large percentage of early-grade students, and many struggle in math. In 2013, ELs made up 35.7% of CA grade 2 enrollments, and 47% of those students tested below proficient in math. The status of ELs statewide is mirrored in the partner, small districts serving the unincorporated communities in Riverside County. The percentage of students receiving free or reduced-price lunch is 76.1% and 65% at the two districts, while the state average is 58%. Per the 2010 Census, the largest ethnic group in both communities is Hispanic/Latino at 51.4% and 54.5%. In one district involved in the study, 45.4% of people aged 5 or more years speak a language other than English at home. Results from the 6 2013 California Standards Test (CST) showed the percentage of students classified as EL tapers through the elementary grades. While the proportion of ELs decreases, the math achievement gap for ELs grows with the complexity of the mathematics from grade to grade. In the seventh grade, 52% of English-Fluent or English Only seventh graders were proficient or advanced in mathematics while only 13% of ELs attained that level of achievement. A student classified as EL in the grade 7 pivot year faces restricted access to advanced mathematics, to success in high school and college, and to STEM careers. Mathematics interventions supporting ELs and their teachers are especially timely in light of the Common Core State Standards for Mathematics (CCSS-M). The CCSS-M call for students to demonstrate proficiency through eight Standards for Mathematical Practice (SMP). In order to give ELs access to the CCSS-M, Moschkovich (2013) recommends that teachers balance concepts with procedures; maintain high cognitive demand; develop productive student beliefs about math; and engage students in the SMP. At the same time, teachers in the 22 states using the new Smarter Balanced Assessment Consortium (SBAC) tests to assess student proficiency with respect to the CCSS-M are challenged to prepare ELs for the higher language demands of these tests. TEEM supported teachers in this shift by developing a replicable model of intensive professional learning. Improving outcomes in core academic subjects through attention to metacognitive and non-cognitive skills and mindsets. When students believe that intellectual ability is not fixed but grows through effort, they show greater persistence (Dweck, 2006). The National Mathematics Advisory Panel (2008) reported that interventions addressing affective and motivational factors can improve the achievement of traditionally underserved groups. TEEM will help ELs develop the metacognitive and non-cognitive skills key to success using interactive inquiry notebooks for math. The notebooks were not only a tool for collecting and recording classroom activities, but they sought to make student metacognition explicit and provide a structured space for developing reading, writing, and study skills. Unlike many accepted practices for supporting ELs which focus on general academic language development, the program sought to achieve improved outcomes for ELs while keeping a strong disciplinary focus on mathematics, combining four structured inquiry-focused interventions targeting students, teachers, and school leaders. TEEM’s approach to improving the achievement of ELs is based on the novel premise that educators need deep mathematical understanding to provide ELs access to the SMP in a CCSS-aligned curriculum, that “vocabulary is necessary, [but] not sufficient” to provide that access (Moschkovich, 2013), and that providing that access will support ELs in strengthening language skills through reasoning, talking, and writing about complex disciplinary content. To provide a comprehensive approach to increasing ELs’ access to rigorous mathematics, the project combined four evidence-based interventions: 1. The adaptation of interactive inquiry notebooks – a best practice in science teaching – to mathematics 7 2. Summer Professional Development (PD) to immerse teachers in math inquiry and notebooking 3. The adaptation of the Japanese Lesson Study model of professional learning for teachers 4. The purposeful, regular, and intensive participation of school principals in the professional development. Given the need to intervene early, another novel aspect of TEEM was the creation of a PD pipeline for preschool educators who rarely receive opportunities for math-focused PD. Through its focus on mathematics, its combination of research-based interventions, and the purposeful inclusion of school leaders and early education providers, TEEM intended to provide a blueprint for how to increase EL student academic performance and pursuit of STEM in college and careers. TEEM interventions were built upon existing research and promising practice, with adaptations intended to make it an exemplar for new practice. Interactive notebooks. Notebooks have been used successfully in many science classrooms (Hargrove & Nesbit, 2003). Used intentionally, student notebooks provide evidence of the teacher’s teaching practices and the students’ learning (Shimojyo & Shimojyo, 2013) and are a formative assessment tool of great value. They make student metacognition explicit and provide a structured space for the development of reading, writing, and study skills. It is a promising practice, and an inexpensive and sustainable one since paper notebooks are easily obtained. It is a culturally responsive practice: in Mexican and other Latin American classrooms, el cuaderno (notebook) is used in a more structured and rigorous fashion than is common in the United States (Celedon-Pattichis & Ramirez, 2012). Amaral and colleagues (2002) found that the use of interactive notebooks in elementary science classrooms in an impoverished section of inland California improved EL achievement not only in science, but also in reading and writing. TEEM hopes to move the field forward by adapting a promising practice in science education to the mathematics classroom and measuring its impact. We tested the idea that use of purposefully designed math inquiry notebooks would help develop language, organizational, and metacognitive skills, and would accelerate EL achievement not only in math but also in language learning. The combination of interactive notebooks with the lesson study model for PD is a unique and powerful way for teachers to develop effective formative assessment practices. Lesson Study. The transition to CCSS calls for teachers to make major shifts towards greater focus, coherence, and rigor in math instruction (CCSS Initiative, 2014), but observers have noted that fundamental teaching practices have changed little in the last century (National Research Council, 2001). TEEM supported teachers in this shift using the Japanese lesson study model. Lesson study gained attention with the publication of The Teaching Gap (Stigler & Hiebert, 1999) and is the subject of active research (Lewis et. al. 2009). Lesson study 8 incorporates many characteristics of effective PD and school change: it is intensive, ongoing, and connected to practice; it is focused on student learning, it addresses the teaching of specific curriculum content, and it builds strong working relationships among teachers (Darling-Hammond et. al. 2009). In lesson study, teams of teachers develop a practice of inquiring into their students’ thinking and investigating the effectiveness of their instruction. In a cycle of lesson study (Fig. 1), teachers agree to shared learning goals. They research relevant curriculum and pedagogy, and they plan, observe, and refine a “research lesson” addressing the goals. Based on its documented effectiveness and on experience gained in previous projects, TEEM used lesson study to create new opportunities for schools to build a knowledge base for teaching core academic subjects to ELs. Through close observation, teachers develop “the eyes to see students” (Lewis, 2004), seeing past language deficiencies into the content of students’ mathematical thinking. TEEM Figure 1 - The Lesson Study Cycle teachers sought to build and share knowledge of their students’ language needs and learn to anticipate those needs. Student notebooks were a key resource for lesson study, providing teachers with data to inform their study, help discern student needs, set instructional goals and evaluate success of research lessons. Supporting Instructional Leadership in Mathematics. Among all school-related factors, the impact of leadership on student learning is second only to teaching, and this impact is greatest in high-need schools. At the same time, there is little research on how principals continue their professional learning on the job (Leithwood et. al, 2004). While Instructional Rounds (City et. al, 2009) are considered best practice for principal learning, principals without a significant background in mathematics receive little support in considering the mathematical content of the lessons they observe in classrooms. To fill this gap, TEEM’s fourth novel intervention sought to support principals as “lead learners” at schools (Liptak, 2005), developing their capacity as instructional leaders in the transition to CCSS. Principals attended Summer Institute sessions and monthly sessions built into district-wide principal meetings. Project staff supported principals in Instructional Rounds. The proposed interventions and the evaluation of their effectiveness will hopefully benefit schools and districts nationwide: EL populations are growing rapidly across the nation (by 53% from ’98 to ’08; NMAP, 2008) and, to our knowledge, research investigating methods to accelerate EL growth via CCSS, lesson study, and interactive 9

See more

The list of books you might like

Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.