Search EdWorkingPapers

Search EdWorkingPapers by author, title, or keywords.

Program and policy effects

A. Brooks Bowden, Viviana Rodriguez, Zach Weingarten.

In response to widening achievement gaps and increased demand for post-secondary education, local and federal governments across the US have enacted policies that have boosted high school graduation rates without an equivalent rise in student achievement, suggesting a decline in academic standards. To the extent that academic standards can shape effort decisions, these trends can have important implications for human capital accumulation. This paper provides both theoretical and empirical evidence of the causal effect of academic standards on student effort and achievement. We develop a theoretical model of endogenous student effort that depends on grading policies, finding that designs that do not account for either the spread of student ability or the magnitude of leniency can increase achievement gaps. Empirically, under a research design that leverages variation from a statewide grading policy and school entry rules, we find that an increase in leniency mechanically increased student GPA without increasing student achievement. At the same time, this policy induced students to increase their school absences. We uncover stark heterogeneity of effects across student ability, with the gains in GPA driven entirely by high ability students and the reductions in attendance driven entirely by low ability students. These differences in responses compound across high school and ultimately widen long-term achievement gaps as measured by ACT scores.

More →


Katharine Meyer, Lindsay C. Page, Catherine Mata, Eric Smith, B. Tyler Walsh, C. Lindsey Fifield, Amy Eremionkhale, Michael Evans, Shelby Frost.
Despite documented benefits to college completion, more than a third of students who initially enroll in college do not ultimately earn a credential. Completing college requires students to navigate both institutional administrative tasks (e.g., registering for classes) and academic tasks within courses (e.g., completing homework). In postsecondary education, several promising interventions have shown that text-based outreach and communication can be a low-cost, easy to implement, and effective strategy for supporting administrative task navigation. In this paper, we report on two randomized controlled trials testing the effect of a text-based chatbot with artificial intelligence (AI) capability on students' academic task navigation in introductory courses (political science and economics). We find the academic chatbot significantly shifted students’ final grades, increasing the likelihood students received a course grade of B or higher by 5-6 percentage points and reduced the likelihood students dropped the course.

More →


Eric A. Hanushek, Lavinia Kinne, Pietro Sancassani, Ludger Woessmann.

Decisions to invest in human capital depend on people’s time preferences. We show that differences in patience are closely related to substantial subnational differences in educational achievement, leading to new perspectives on longstanding within-country disparities. We use social-media data – Facebook interests – to construct novel regional measures of patience within Italy and the United States. Patience is strongly positively associated with student achievement in both countries, accounting for two-thirds of the achievement variation across Italian regions and one-third across U.S. states. Results also hold for six other countries with more limited regional achievement data.

More →


Monnica Chan, Blake Heller.

Generally, need-based financial aid improves students’ academic outcomes. However, the largest source of need-based grant aid in the United States, the Federal Pell Grant Program (Pell), has a mixed evaluation record. We assess the minimum Pell Grant in a regression discontinuity framework, using Kentucky administrative data. We focus on whether and how year-to-year changes in aid eligibility and interactions with other sources of aid attenuate Pell’s estimated effects on post-secondary outcomes. This evaluation complements past work by assessing explanations for the null or muted impacts found in our analysis and other Pell evaluations. We also discuss the limitations of using regression discontinuity methods to evaluate Pell—or other interventions with dynamic eligibility criteria—with respect to generalizability and construct validity.

More →


Lena Shi.

Students’ college choices can affect their chances of earning a degree, but many lack the support to navigate the opaque college application and admissions process. This paper evaluates whether guaranteeing four-year college admissions based on transparent academic standards affected community college students’ enrollment choices and graduation rates. Guaranteed admissions increased high-GPA graduates’ transfer rates to highly-selective colleges by 30 percent. Increased transfers to highly-selective colleges also accompanied higher graduation rates and lower student debt. Gains were largest for students with historically lower transfer rates. Transparent admissions standards can increase access to selective colleges at low to no cost.

More →


Douglas M. Mosher, James S. Kim.

This study contributes to the science of teaching reading by illustrating how a ubiquitous classroom practice – read alouds – can be enhanced by fostering teacher language practices that support students’ ability to read for understanding. This experimental study examines whether and to what extent providing structured teacher read aloud supplements in a social studies read aloud can allow students to leverage a familiar science schema and thereby positively impact reading comprehension outcomes. Treatment students received a single social studies read-aloud on the story of Apollo 11 with structured teacher read aloud supplements while control students received the same read-aloud story but without structured supplements. Effect sizes from hierarchical linear models indicated that students in the treatment condition significantly outperformed students in the control condition on four measures of domain-specific reading comprehension. Further exploratory analyses using structural equation modeling examined the extent that teacher language mediated the treatment effect. Results indicated that teachers going above and beyond the intervention script explained 67 percent of the treatment effect. Structured supplements for read alouds can help students see important connections between schemas, which ultimately aids in reading comprehension.

More →


Francisco Gallego, Philip Oreopoulos, Noah Spencer.

This paper discusses the importance of incorporating personal assistance into interventions aimed at improving long-term education and labor market success. While existing research demonstrates the cost-effectiveness of low-touch behavioral nudges, this paper argues that the dynamic nature of human capital accumulation requires sustained habits over time. To foster better habits, social connections are critical for encouraging enduring effort and intrinsic motivation. The paper showcases examples from various stages of human capital accumulation, including early childhood, adolescence, and adulthood, in which interventions that incorporate personal assistance substantially out-perform less intensive nudges. We underscore the importance of interactive support, guidance, and motivation in facilitating significant progress and explore the challenges associated with implementing cost-effective policies to provide such assistance.

More →


Danielle Sanderson Edwards, Matthew A. Kraft, Alvin Christian, Christopher A. Candelaria.
We develop a unifying conceptual framework for understanding and predicting teacher shortages at the state, region, district, and school levels. We then generate and test hypotheses about geographic and subject variation in teacher shortages using data on unfilled teaching positions in Tennessee during the fall of 2019. We find that teacher staffing challenges are highly localized, causing shortages and surpluses to coexist. Aggregate descriptions of staffing challenges mask considerable variation between schools and subjects within districts. Schools with fewer local early-career teachers, smaller district salary increases, worse working conditions, and higher historical attrition rates have higher vacancy rates. Our findings illustrate why viewpoints about, and solutions to, shortages depend critically on whether one takes an aggregate or local perspective.

More →


Noam Angrist, Rachael Meager.

Targeted instruction is one of the most effective educational interventions in low- and middle-income countries, yet reported impacts vary by an order of magnitude. We study this variation by aggregating evidence from prior randomized trials across five contexts, and use the results to inform a new randomized trial. We find two factors explain most of the heterogeneity in effects across contexts: the degree of implementation (intention-to-treat or treatment-on-the-treated) and program delivery model (teachers or volunteers). Accounting for these implementation factors yields high generalizability, with similar effect sizes across studies. Thus, reporting treatment-on-the-treated effects, a practice which remains limited, can enhance external validity. We also introduce a new Bayesian framework to formally incorporate implementation metrics into evidence aggregation. Results show targeted instruction delivers average learning gains of 0.42 SD when taken up and 0.85 SD when implemented with high fidelity. To investigate how implementation can be improved in future settings, we run a new randomized trial of a targeted instruction program in Botswana. Results demonstrate that implementation can be improved in the context of a scaling program with large causal effects on learning. While research on implementation has been limited to date, our findings and framework reveal its importance for impact evaluation and generalizability.

More →


Brian McManus, Jessica Howell, Michael Hurwitz.

The impact of test-optional college admissions policies depends on whether applicants act strategically in disclosing test scores. We analyze individual applicants’ standardized test scores and disclosure behavior to 50 major US colleges for entry in fall 2021, when Covid-19 prompted widespread adoption of test-optional policies. Applicants withheld low scores and disclosed high scores, including seeking admissions advantages by conditioning their disclosure choices on their other academic characteristics, colleges’ selectivity and testing policy statements, and the Covid-related test access challenges of the applicants’ local peers. We find only modest differences in test disclosure strategies by applicants’ race and socioeconomic characteristics.

More →