Search EdWorkingPapers

Search for EdWorkingPapers here by author, title, or keywords.

Program and policy effects

James D. Paul, Patrick J. Wolf.

Virtual charter schools provide full-time, tuition-free K-12 education through internet-based instruction. Although virtual schools offer a personalized, content-appropriate experience, most research suggests these schools are negatively associated with achievement. Few studies account for differential rates of student mobility, which may produce biased estimates if mobility is jointly associated with virtual school enrollment and subsequent test scores. We account for student mobility in an evaluation of a single, large, anonymous virtual charter school. We estimate treatment effects of the virtual school on student achievement using a hybrid of exact and nearest-neighbor propensity score matching. Relative to their matched peers, we estimate that virtual students produce similar ELA scores and significantly worse math scores after one year. Among a limited sample of students observed for four years, we estimate that virtual students ultimately produce higher ELA scores and similar math scores relative to matched peers. We argue these findings are more reliable indicators of the independent effect of virtual schooling on student achievement because the match on student mobility is a proxy for otherwise unobservable negative selection factors.

More →


Rajashri Chakrabarti, Nicole Gorton, Michael F. Lovenheim.

Most public colleges and universities rely heavily on state financial support. As state budgets have tightened in recent decades, appropriations for higher education have declined substantially. Despite concerns expressed by policymakers and scholars that the declines in state support have reduced the return to education investment for public sector students, little evidence exists that can identify the causal effect of these funds on long-run outcomes. We present the first such analysis in the literature using new data that leverages the merger of two rich datasets: consumer credit records from the New York Fed's Consumer Credit Panel (CCP), sourced from Equifax, and administrative college enrollment and attainment data from the National Student Clearinghouse. We overcome identification concerns related to the endogeneity of state appropriation variation using an instrument that interacts the baseline share of total revenue that comes from state appropriations at each public institution with yearly variation in state-level appropriations. Our analysis is conducted separately for two-year and four-year students, and we analyze individuals into their mid-30s. For four-year students, we find that state appropriation increases lead to substantially lower student debt originations. They also react to appropriation increases by shortening their time to degree, but we find little effect on other outcomes. In the two-year sector, state appropriation increases lead to more collegiate and post-collegiate educational attainment, more educational debt consistent with the increased educational attainment, but lower likelihood of delinquency and default. State support also leads to more car and home ownership with lower adverse debt outcomes, and these students experience substantial increases in their credit score and in the affluence of the neighborhood in which they live. Examining mechanisms, we find state appropriations are passed on to students in the form of lower tuition in the four-year sector with no institutional spending response. For community colleges, we find evidence of both price and quality mechanisms, the latter captured in higher educational resources in key spending categories. These results are consistent with the different pattern of effects we document in the four-year and two-year sectors. Our results underscore the importance of state support for higher education in driving student debt outcomes and the long-run returns to postsecondary investments that students experience.

More →


Oded Gurantz, Taylor Odle.

We replicate and extend prior work on Florida’s Bright Futures merit aid scholarship to consider its effect on college enrollment and degree completion. We estimate causal impacts using a regression discontinuity design to exploit SAT thresholds that strongly determine eligibility. We find no positive impacts on attendance or attainment, and instrumental variable results generally reject estimates as small as 1-2 percentage points. Across subgroups, we do find that eligibility slightly reduces six-year associate degree attainment for lower-SES students and may induce small enrollment shifts among Hispanic and White students. Our findings of these minimal-at-best impacts contrast those of prior works, attributable in part to methodological improvements and more robust data, and further underscore the importance of study replication. (JEL: H75, I21, I22, I23, I28)

More →


Catherine Armstrong Asher, Ethan Scherer, James S. Kim.

Parental text messaging interventions are growing in popularity to encourage at-home reading, school-attendance, and other educational behaviors. These interventions, which often combine multiple components, frequently demonstrate varying amounts of effectiveness, and researchers often cannot determine how individual components work alone or in combination with one another. Using a 2x2x3 factorial experiment, we investigate the effects of individual and interacted components from three behavioral levers to support summer reading: providing updated, personalized information; emphasizing different reading views; and goal setting. We find that the personalized information condition scored 0.03 SD higher on fall reading assessments. Test score effects were enhanced by messages that emphasized reading being useful for both entertainment and building skills compared to skill building alone or entertainment alone.

More →


Michael T. Hartney, Leslie K. Finger.

The COVID-19 pandemic presents a unique opportunity to examine how local governments respond to a public health crisis amid high levels of partisan polarization and an increasing tendency for local issues to become nationalized. As an arena that has, in recent years, been relatively separate from national partisan divides, public schools provide a useful window into these dynamics. Leveraging the fact that all of the nation’s school districts had to adopt a reopening plan for the fall, we test what factors best predict whether a district chose to return students to the classroom or educate them remotely. Contrary to the conventional understanding of school districts as localized and non-partisan actors, we find evidence that politics, far more than science, shaped school district decision-making. Mass partisanship and teacher union strength best explain how school boards approached reopening. Additionally, we find evidence that districts are sensitive to the threat of private school exit. Districts located in counties with a larger number of Catholic schools were less likely to shut down and more likely to return to in-person learning. These findings have important implications for our understanding of education policy and the functioning of American local governments.

More →


Jessica Goldstein, Josh B. McGee.

Jackson, Wigger, and Xiong (2020a, JWX) provide evidence that education spending reductions following the Great Recession had widespread negative impacts on student achievement and attainment. This paper describes our process of duplicating JWX and highlights a variety of tests we employ to investigate the nature and robustness of the relationship between school spending reductions and student outcomes. Though per-pupil expenditures undoubtedly shifted downward due to the Great Recession, contrary to JWX, our findings indicate there is not a clear and compelling story about the impact of those reductions on student achievement. Moreover, we find that the relationship between K-12 spending and college-going rates is likely confounded with contemporaneous higher education funding trends. While we believe that K-12 spending reductions may have negative impacts on student outcomes, our results suggest that estimating generalizable causal effects remains a significant challenge.

More →


Samantha Viano, Gary T. Henry.

Credit recovery (CR) refers to online courses that high school students take after previously failing the course. Many have suggested that CR courses are helping students to graduate from high school without corresponding increases in academic skills. This study analyzes administrative data from the state of North Carolina to evaluate these claims using full data from public and private CR providers. Findings indicate that students who fail courses and enroll in CR have lower test scores of up to two tenths of a standard deviation and are about seven percent more likely to graduate high school on time than students who repeat courses traditionally. Test score differences are particularly large for Biology compared to Math I and English II. Hispanic and economically disadvantaged CR students are more likely to graduate high school than their peers.

More →


Angela Johnson.

This paper reports math and reading academic achievement and growth in grades 2 to 8 for Hispanic participants and nonparticipants of a Spanish-English dual language program. I apply a piecewise multilevel growth model to administrative data from a large school district that enrolls a substantial English Learner student population. Dual language participants started 2nd grade with lower achievement than nonparticipants. In math, dual language participants grew faster than nonparticipants during each school year in grades 2 to 5 but lost more learning during subsequent summers. Thus, despite growing faster in the beginning, dual language students did not learn more than their peers in the long run, and the gap between dual language students and the national average was not closing. In reading, dual language participants grew slightly more slowly during school years but lost less learning during the summers, closing the gap between themselves and the national average. These findings suggest that programs aimed at addressing achievement gaps need to consider summer as well as school-year learning for historically-underserved student populations.

More →


Vivian Yuen Ting Liu, Di Xu.

Nationally, 15% of first-time community college students were high school dual enrollment (DE) students, which raises concerns about how high school peers might influence college enrollees. Using administrative data from a large state community college system, we examine whether being exposed to a higher percentage of DE peers in entry-level (gateway) math and English courses influences non-DE enrollees’ performance. Using a two-way fixed effects model, our results indicate that college enrollees exposed to a higher proportion of DE peers had lower pass rates and grades in gateway courses, and higher course repetition rates. Supplemental student-level analysis suggests that greater exposure to DE peers during a student’s initial semester in college reduces next-term college persistence.

More →


Matthew A. Kraft, Alexander Bolves.

We study the adoption and implementation of a new mobile communication app among a sample of 132 New York City public schools. The app provides a platform for sharing general announcements and news as well as engaging in personalized two-way communication with individual parents. We provide participating schools with free access to the app and randomize schools to receive intensive support (training, guidance, monitoring, and encouragement) for maximizing the efficacy of the app. Although user supports led to higher levels of communication within the app in the treatment year, overall usage remained low and declined in the following year when treatment schools no longer received intensive supports. We find few subsequent effects on perceptions of communication quality or student outcomes. We leverage rich internal user data to explore how take-up and usage patterns varied across staff and school characteristics. These analyses help to identify early adopters and reluctant users, revealing both opportunities and obstacles to engaging parents through new communication technology.

More →