Search for EdWorkingPapers here by author, title, or keywords.
Standards, accountability, assessment, and curriculum
With 55 million students in the United States out of school due to the COVID-19 pandemic, education systems are scrambling to meet the needs of schools and families, including planning how best to approach instruction in the fall given students may be farther behind than in a typical year. Yet, education leaders have little data on how much learning has been impacted by school closures. While the COVID-19 learning interruptions are unprecedented in modern times, existing research on the impacts of missing school (due to absenteeism, regular summer breaks, and school closures) on learning can nonetheless inform projections of potential learning loss due to the pandemic. In this study, we produce a series of projections of COVID-19-related learning loss and its potential effect on test scores in the 2020-21 school year based on (a) estimates from prior literature and (b) analyses of typical summer learning patterns of five million students. Under these projections, students are likely to return in fall 2020 with approximately 63-68% of the learning gains in reading relative to a typical school year and with 37-50% of the learning gains in math. However, we estimate that losing ground during the COVID-19 school closures would not be universal, with the top third of students potentially making gains in reading. Thus, in preparing for fall 2020, educators will likely need to consider ways to support students who are academically behind and further differentiate instruction.
Castañeda v. Pickard (1981) mandated that educational programs for emergent bilinguals be tested for program efficacy. Since English language development (ELD) curricular materials are one part of an instructional program, we assess this mandate by examining the effectiveness of ELD materials in Texas. Using local linear matching, we find that schools that do not purchase any ELD curricula have significantly lower English language proficiency scores relative to schools that purchase state-adopted ELD materials. These results are robust across various matching models—inverse probability weights with regression adjustment, kernel matching, and nearest neighbor matching--and a comparative interrupted time series design. There is no significant difference between schools that adopt the two most popular ELD curricula—Rigby On Our Way to English and National Geographic Reach. This study suggests that emergent bilinguals (EBs) who attend schools that have instructional materials that explicitly foreground English language proficiency standards outperform those in schools that do not have such materials.
Important educational policy decisions, like whether to shorten or extend the school year, often require accurate estimates of how much students learn during the year. Yet, related research relies on a mostly untested assumption: that growth in achievement is linear throughout the entire school year. We examine this assumption using a data set containing math and reading test scores for over seven million students in kindergarten through 8th grade across the fall, winter, and spring of the 2016-17 school year. Our results indicate that assuming linear within-year growth is often not justified, particularly in reading. Implications for investments in extending the school year, summer learning loss, and racial/ethnic achievement gaps are discussed.
High school graduation rates have increased dramatically in the past two decades. Some skepticism has arisen, however, because of the confluence of the graduation rise and the starts of high-stakes accountability for graduation rates with No Child Left Behind (NCLB). In this study we provide some of the first evidence about the role of accountability versus strategic behavior, especially the degree to which the recent graduation rate rise represents increased human capital. First, using national DD analysis of within-state, cross-district variation in proximity to state graduation rate thresholds, we confirm that NCLB accountability increased graduation rates. However, we find limited evidence that this is due to strategic behavior. To test for lowering of graduation standards, we examined graduation rates in states with and without graduation exams and trends in GEDs; neither analysis suggests that the graduation rate rise is due to strategic behavior. We also examined the effects of “credit recovery” courses using Louisiana micro data; while our results suggest an increase in credit recovery, consistent with some lowering of standards, the size of the effect is not nearly enough to explain the rise in graduation rates. Finally, we examine other forms of strategic behavior by schools, though these can only explain inflation of school/district-level graduation rates, not rational rates. Overall, the evidence suggests that the rise in the national graduation rates reflects some strategic behavior, but also a substantial increase in the nation’s stock of human capital. Graduation accountability was a key contributor.
How should schools assign students to more rigorous math courses so as best to help their academic outcomes? We identify several hundred California middle schools that used 7th grade test scores to place students into 8th grade Algebra courses, and use a regression discontinuity design to estimate average impacts and heterogeneity across schools. Enrolling in 8th grade algebra boosts students’ enrollment in advanced math in 9th grade by 30 percentage points and 11th grade by 16 percentage points. Math scores in 10th grade rise by 0.05 standard deviations. Women, students of color, and English-language learners benefit disproportionately from placement into early Algebra. Importantly, the benefits of 8th grade algebra are substantially larger in schools that set their eligibility threshold higher in the baseline achievement distribution. This suggests a potential tradeoff between increased access and rates of subsequent math success.
A huge portion of what we know about how humans develop, learn, behave, and interact is based on survey data. Researchers use longitudinal growth modeling to understand the development of students on psychological and social-emotional learning constructs across elementary and middle school. In these designs, students are typically administered a consistent set of self-report survey items across multiple school years, and growth is measured either based on sum scores or scale scores produced based on item response theory (IRT) methods. While there is great deal of guidance on scaling and linking IRT-based large-scale educational assessment to facilitate the estimation of examinee growth, little of this expertise is brought to bear in the scaling of psychological and social-emotional constructs. Through a series of simulation and empirical studies, we produce scores in a single-cohort repeated measure design using sum scores as well as multiple IRT approaches and compare the recovery of growth estimates from longitudinal growth models using each set of scores. Results indicate that using scores from multidimensional IRT approaches that account for latent variable covariances over time in growth models leads to better recovery of growth parameters relative to models using sum scores and other IRT approaches.
Survey respondents use different response styles when they use the categories of the Likert scale differently despite having the same true score on the construct of interest. For example, respondents may be more likely to use the extremes of the response scale independent of their true score. Research already shows that differing response styles can create a construct-irrelevant source of bias that distorts fundamental inferences made based on survey data. While some initial studies examine the effect of response styles on survey scores in longitudinal analyses, the issue of how response styles affect estimates of growth is underexamined. In this study, we conducted empirical and simulation analyses in which we scored surveys using item response theory (IRT) models that do and do not account for response styles, and then used those different scores in growth models and compared results. Generally, we found that response styles can affect estimates of growth parameters including the slope, but that the effects vary by psychological construct, response style, and model used.
One of the mysteries of education reform is how leaders and educators can successfully instantiate, sustain, and spread student-centered pedagogical practices from a few schools to many others. Advocates for deeper learning grapple with this mystery as they seek to transform teaching and learning to prepare students to meet the demands of the 21st century and to close the opportunity gap between advantaged and disadvantaged groups. While research suggests that deeper learning strategies that support critical thinking and problem-solving can yield improved student outcomes, implementing these strategies is not easy, as they require reimagining school environments and changing traditional approaches to teaching. This report highlights how three networks of schools engaged in deeper learning have managed this feat. It describes the systems and structures the networks have used to instantiate their equitable deeper learning models in diverse public school settings to serve students in more personalized and productive ways.
Research showing that high-quality preschool benefits children’s early learning and later life outcomes has led to increased state engagement in public preschool. However, mixed results from evaluations of two programs—Tennessee’s Voluntary Pre-K program and Head Start—have left many policymakers unsure about how to ensure productive investments. This report presents the most rigorous evidence on the effects of preschool and clarifies how the findings from Tennessee and Head Start relate to the larger body of research showing that high-quality preschool enhances children’s school readiness by supporting substantial early learning gains in comparison to children who do not experience preschool and can have lasting impacts far into children’s later years of school and life. Therefore, the issue is not whether preschool “works,” but how to design and implement programs that ensure public preschool investments consistently deliver on their promise.
Although there is considerable research on the elements of high-quality preschool and its many benefits, particularly for low-income children and English learners, little information is available to policymakers about how to convert their visions of good early education into on-the-ground reality. This study fills that gap by describing and analyzing how four states—Michigan, West Virginia, Washington, and North Carolina—have built high-quality early education systems. Among the common elements of their success are strategies that prioritize quality and continuous improvement, invest in training and coaching for program staff, coordinate the administration of birth-through-grade-3 programs, strategically combine multiple funding sources to increase access and improve quality, and create broad-based coalitions and support.