Search EdWorkingPapers
Search EdWorkingPapers by author, title, or keywords.
Search
Program and policy effects
We study the distributional effects of remote learning. Our approach combines newly collected data on parental preferences with administrative data from Los Angeles. The preference data allow us to account for selection into remote learning while also studying selection patterns and treatment effect heterogeneity. We find a negative average effect of remote learning on reading (–0.14σ) and math (–0.17σ). Notably, we find evidence of positive learning effects for children whose parents have the strongest demand for remote learning. Our results suggest an important subset of students who currently sort into post-pandemic remote learning benefit from expanded choice.
Public policies often target individuals but within-family externalities of such interventions are understudied. Using a regression discontinuity design, we document how a third grade retention policy affects both the target children and their younger siblings. The policy improves test scores of both children while the spillover is up to 30% of the target child effect size. The effects are particularly pronounced in families where one of the children is disabled, for boys, and in immigrant families. Candidate mechanisms include improved classroom inputs and parental school choice.
Improving school quality in low and middle income countries (LMICs) is a global priority. One way to improve quality may be to improve the management skills of school leaders. In this systematic review, we analyze the impact of interventions targeting school leaders' management practices on student learning. We begin by describing the characteristics and responsibilities of school leaders using data from large, multi-country surveys. Second, we review the literature and conduct a meta-analysis of the causal effect of school management interventions on student learning, using 39 estimates from 20 evaluations. We estimate a statistically significant improvement in student learning of 0.04 standard deviations. We show that effect sizes are not related to program scale or intensity. We complement the meta-analysis by identifying common limitations to program effectiveness through a qualitative assessment of the studies included in our review. We find three main factors which mitigate program effectiveness: 1) low take-up; 2) lack of incentives or structure for implementation of recommendations; and 3) the lengthy causal chain linking management practices to student learning. Finally, to assess external validity of our review, we survey practitioners to compare characteristics between evaluated and commonly implemented programs. Our findings suggest that future work should focus on generating evidence on the marginal effect of common design elements in these interventions, including factors that promote school leader engagement and accountability.
Panel or grouped data are often used to allow for unobserved individual heterogeneity in econometric models via fixed effects. In this paper, we discuss identification of a panel data model in which the unobserved heterogeneity both enters additively and interacts with treatment variables. We present identification and estimation methods for parameters of interest in this model under both strict and weak exogeneity assumptions. The key identification insight is that other periods' treatment variables are instruments for the unobserved fixed effects. We apply our proposed estimator to matched student-teacher data used to estimate value-added models of teacher quality. We show that the common assumption that the return to unobserved teacher quality is the same for all students is rejected by the data. We also present evidence that No Child Left Behind-era school accountability increased the effectiveness of teacher quality for lower performing students.
Guidance counselors provide the main source of college advising for low-income high school students, but are woefully understaffed in high-need schools. This paper evaluates an approach to school-based college advising that relies on teachers rather than counselors. Using a randomized control trial in sixty-two Michigan high schools, I estimate the effects of a college planning course for high school seniors on postsecondary enrollment, persistence, and degree receipt. The course teaches about postsecondary education opportunities, application processes, and strategies for persisting toward a degree. I find no effect of the course on the number of students entering college, but an increase in the number persisting and earning a degree, particularly among low-income students. This is due to a shift in the composition of enrollees toward higher-achieving students: the course increases enrollment among high-achieving, low-income students, who have relatively high persistence rates, and reduces enrollment among low-achieving students, who in the course’s absence would have enrolled and then quickly dropped out. The program’s main cost is potential learning loss from displaced time in other subjects, which is difficult to measure but appears small.
Given the importance of early literacy to long-term student success, by 2021, 41 states and the District of Columbia adopted early literacy policies to improve student literacy by the end of third grade. We use an event-study approach to examine the impact of these policies on high- and low-stakes test scores. Our results suggest that adopting an early literacy policy improves elementary students’ reading achievement on high-stakes assessments, particularly in third grade and in states with comprehensive early literacy policies and third-grade retention requirements. We also find suggestive evidence that early literacy policies reduce socioeconomic and racial high-stakes achievement gaps in reading and have positive spillover effects on math achievement. However, we find little evidence of significant gains in low-stakes test scores except in states with comprehensive policies. Our findings highlight the importance of content and incentives for early literacy policies.
The COVID-19 pandemic resulted in substantial unfinished learning for U.S. students, but to differing degrees for various subgroups. For example, students of color, from low-income families, or who attended high-poverty schools experienced greater unfinished learning. In this study we examined the degree of unfinished learning for students who went into the pandemic scoring in the top or bottom 10% in the math or reading achievement distributions. Our results show that students who scored at or below the 10th percentile grew less during the pandemic than their similarly-scoring, pre-COVID peers and, as of the end of the 2021 – 2021 school year, had yet to rebound toward pre-COVID levels of growth or achievement. Conversely, students who scored at or above the 90th percentile largely grew at rates closer to their pre-COVID peers. These students were harmed less academically and have recovered more quickly than their peers scoring at or below the 10th percentile.
Fadeout is a pervasive phenomenon: post-test impacts on cognitive skills commonly decrease in the years following an educational intervention. Less is known, although much is theorized, about social-emotional skill persistence. The current meta-analysis investigated whether educational RCT impacts on social-emotional skills demonstrated greater persistence than impacts on cognitive skills among 87 interventions involving 59,237 participants and 443 outcomes measured at post-test and at least one follow-up. For post-test impacts of the same magnitude, persistence rates were similar (43% of post-test magnitude) across skill types for follow-ups occurring 6 to 12 months after post-test. At 1- to 2-year follow-ups, persistence rates were larger for cognitive skills (37%) than for social-emotional skills. Interestingly, smaller posttest impacts persisted at proportionately higher rates than larger impacts, which may benefit interventions measuring social-emotional outcomes given their smaller post-test impacts. Considered in whole, social-emotional and cognitive skills demonstrated similar patterns of fadeout.
School mobility, compounding socioeconomic inequities, can undermine academic achievement and behavior, particularly during middle school years. This study investigates the effect of a school-based integrated student support intervention – City Connects – on the achievement and behavior of middle school students who experience school mobility. Using administrative data from a large, urban, public school district in the U.S., we apply student fixed effects and event studies methods to analyze the academic and behavioral performance of students changing schools. The results indicate that students who moved to schools implementing the City Connects intervention performed better academically and behaviorally than other students.
Providing consistent, individualized feedback to teachers is essential for improving instruction but can be prohibitively resource-intensive in most educational contexts. We develop M-Powering Teachers, an automated tool based on natural language processing to give teachers feedback on their uptake of student contributions, a high-leverage dialogic teaching practice that makes students feel heard. We conduct a randomized controlled trial in an online computer science course (n=1,136 instructors), to evaluate the effectiveness of our tool. We find that M-Powering Teachers improves instructors’ uptake of student contributions by 13% and present suggestive evidence that it also improves students’ satisfaction with the course and assignment completion. These results demonstrate the promise of M-Powering Teachers to complement existing efforts in teachers’ professional development.