Search EdWorkingPapers

Search for EdWorkingPapers here by author, title, or keywords.

Program and policy effects

Gary T. Henry, J. Edward Guthrie.

Federal education policies gave political and financial support for state education agencies to turnaround low-performing schools on an unprecedented scale. North Carolina’s ambitious program turned around over half of all schools nationwide that underwent turnaround funded by Race to the Top. Exploiting the assignment to turnaround based on schools’ 2009-10 proficiency rates, we implement  regression discontinuity designs to estimate the effects of state turnaround services on student achievement in North Carolina’s lowest-performing schools annually from the 2011-12 through 2014-15. Overall, we find modest positive effects of turnaround when including treated schools at all grade levels, but these effects are sensitive to bandwidth. For secondary schools, we find consistently positive effects that vary from modest to large. For elementary and middle schools, we find consistent, modest negative effects of turnaround on student achievement.

More →


Paul T. von Hippel, Chandi Wagner.

Evidence-based policy is the practice of basing policy decisions on rigorous research evidence, such as randomized experiments. But it is unclear how often evidence-based decisions produce more effective policy. We evaluate an evidence-based policy implemented in 1989-93, after the state of Tennessee completed the famous Project STAR randomized experiment, which showed that reducing average class sizes from 23 to 15 could raise test scores by nearly 0.2 standard deviations (SD). After Project STAR, the state launched Project Challenge, which tried to achieve similar score gains by earmarking $5 million to reduce class sizes in the state’s 17 poorest districts.

We evaluate the effects of Project Challenge by applying regression discontinuity and difference in differences analysis to data from district report cards. Our analysis offers no evidence that Project Challenge districts raised test scores, and even raises questions about whether districts reduced class sizes. After Project Challenge, Tennessee’s Basic Education Plan did reduce class sizes, but only by a token amount, from 26 to 25. In this example, it seems that a successful randomized experiment did not lead to successful policy.

More →


Philip Oreopoulos, Uros Petronijevic.

We present results from a five-year effort to design promising online and text-message interventions to improve college achievement through several distinct channels.  From a sample of nearly 25,000 students across three different campuses, we find some improvement from coaching-based interventions on mental health and study time, but none of the interventions we evaluate significantly influences academic outcomes (even for those students more at risk of dropping out).  We interpret the results with our survey data and a model of student effort.  Students study about five to eight hours fewer each week than they plan to, though our interventions do not alter this tendency.  The coaching interventions make some students realize that more effort is needed to attain good grades but, rather than working harder, they settle by adjusting grade expectations downwards.  Our study time impacts are not large enough for translating into significant academic benefits.  More comprehensive but expensive programs appear more promising for helping college students outside the classroom. 

More →

Appendix A2.09 MB
Appendix B254.91 KB

Gary T. Henry, Erica Harbatkin.

In contrast to prior federally mandated school reforms, the Every Student Succeeds Act (ESSA) allows states more discretion in reforming their lowest performing schools, removes requirements to disrupt the status quo, and does not allocate substantial additional funds. Using a regression discontinuity design, we evaluate a state turnaround initiative aligned with ESSA requirements. We find the effect on student test score growth was not significant in year one and -0.13 in year two. Also in year two, we find that teachers in turnaround schools were 22.5 percentage points more likely to turn over. Teacher turnover appears to have been voluntary rather than the result of strategic staffing decisions.

More →


Francis Pearman, Matthew Springer, Mark Lipsey, Mark Lachowicz, Dale Farran, Walker Swain.

The sustaining environments thesis hypothesizes that PreK effects are more likely to persist into later grades if children experience high-quality learning environments in the years subsequent to PreK. This study tests this hypothesis using data from a statewide PreK randomized experiment in Tennessee that found positive effects at the end of PreK that did not persist past kindergarten. These data were combined with teacher observation and school-level value-added scores from Tennessee’s formal evaluation system to determine whether positive effects of PreK persisted for the subgroup of students exposed to higher-quality learning environments between kindergarten and 3rd-grade. Neither exposure to highly effective teachers nor attending a high-quality school was sufficient by itself to explain differences in achievement between PreK participants and non-participants in 3rd-grade. However, this study found evidence that having both was associated with a sustained advantage for PreK participants in both math and ELA that lasted through at least 3rd-grade. Notably, however, very few children were exposed to high-quality learning environments after PreK, suggesting that maximizing PreK investments may require attending to the quality of learning environments during PreK and beyond.

More →


Aizat Nurshatayeva, Lindsay C. Page.

English-only college education in non-English speaking countries is a rapidly growing phenomenon that has been dubbed as the most important trend in higher education internationalization. Despite worldwide popularity, there is little empirical evidence about how the transition to English-only instruction affects students’ academic outcomes. Using a natural experiment at a selective university in Central Asia and a difference-in-differences strategy, we estimate the causal effect of switching to English-only instruction on students’ college outcomes. We find that the introduction of English-only instruction led to a decrease of GPAs and probability of graduation and an increase in the number of failed course credits. Although negative, the effects were short-lived. The difference-in-differences estimates and the examination of potential mechanisms suggest that at least in selective universities in non-English speaking countries, the switch to English-only instruction may affect college outcomes negatively at the time of transition but may not necessarily imply longer-run negative effects.

More →


Ilana Umansky, Hanna Dumont.

Prior research has shown that EL classification is consequential for students, however, less is known about how EL classification impacts students’ outcomes. In this study, we examine one hypothesized mechanism: teacher perceptions. Using nationally-representative data (ECLS-K:2011), we use coarsened exact matching to estimate the effect of EL status on teachers’ perceptions of students’ skills in language arts, math, science, and social studies in kindergarten through second grade. We further explore whether that impact is moderated by instructional setting (bilingual versus English immersion). We find evidence that EL classification results in lower teacher perceptions across content areas and grade levels. This impact is, however, moderated by bilingual environments. This study adds to research on teacher perceptions and the effects of EL classification. 

More →


Andrew McEachin, Douglas L. Lauen, Sarah C. Fuller, Rachel M. Perera.

The vast majority of literature on school choice, and charter schools in particular, focus on attending an elementary or middle school and often focus on test scores or other proximal outcomes. Much less is known about the long-term effects of attending a charter high school. It is important to fill this information void for a few reasons. First, schools in general affect more than just students’ test scores. Second, high schools make up a larger share of the charter sector. Third, school choice depends on freely available information for parents and students to make informed decisions about where to attend, including potential long-term benefits. We add to the empirical research on charter school effects by using a doubly-robust regression-adjusted propensity score matching approach to evaluate the impacts of charter high school attendance on 9th grade behavioral outcomes and individuals propensity to commit crime and participate in elections as young adults in North Carolina, a state with a large and growing charter school sector.

More →


Andrew C. Barr, Kelli A. Bird, Benjamin L. Castleman.

Student loan borrowing for higher education has emerged as a top policy concern. Policy makers at the institutional, state, and federal levels have pursued a variety of strategies to inform students about loan origination processes and how much a student has cumulatively borrowed, and to provide students with greater access to loan counseling. We conducted an experiment to evaluate the impact of an outreach campaign that prompted loan applicants at a large community college to make informed and active borrowing decisions and that offered them access to remote, one-onone assistance from a loan counselor. The intervention led students to reduce their unsubsidized loan borrowing by 7 percent, resulted in worse academic performance, and increased the likelihood of loan default during the three years after the intervention occurred. Our results suggest policy makers and higher education leaders should carefully examine the potential unintended consequences of efforts to reduce student borrowing, particularly in light of growing evidence regarding the counter-intuitive positive relationship between reduced borrowing levels and worse student academic and financial outcomes.

More →


David M. Houston, Jeffrey R. Henig.

We conduct an online survey experiment in which participants are asked to imagine that they are parents moving to a new metropolitan area. They then choose between the five largest school districts in that area. All participants receive demographic data for each district. In addition, some participants are randomly assigned to receive average achievement and/or average growth data for each district. While there are strong relationships between student demographics and student achievement, the links between student demographics and student growth are much weaker. We find that, on average, the provision of growth data causes participants to choose less white and less wealthy districts. Moreover, the provision of both achievement and growth data causes participants to choose less white and less wealthy districts than the provision of achievement data alone.

More →