Search EdWorkingPapers

Search EdWorkingPapers by author, title, or keywords.

Program and policy effects

Elizabeth S. Park, Di Xu.

Growing literature documents the promise of active learning instruction in engaging students in college classrooms. Accordingly, faculty professional development (PD) programs on active learning have become increasingly popular in postsecondary institutions; yet, quantitative evidence on the effectiveness of these programs is limited. Using administrative data and an individual fixed effects approach, we estimate the effect of an active learning PD program on student performance and persistence at a large public institution. Findings indicate that the training improved subsequent persistence in the same field. Using a subset of instructors whose instruction was observed by independent observers, we identify a positive association between training and implementation of active learning teaching practices. These findings provide suggestive evidence that active learning PD has the potential to improve student outcomes. 

More →


Sarah Guthery, Lauren P. Bailes.

Texas reduced new teacher preparation requirements in 2001 to allow more alternate paths to licensure. Within five years, this policy change resulted in over half the state’s new teachers being alternatively licensed. Using a series of first difference models, this study examines the relationship between the increased supply of new teachers in Texas and new teacher salaries prior to the policy change and in the fifteen years thereafter. We find that the policy change did increase the supply of new teachers via alternative licensing, but pay for new EC-6 teachers declined by 2 to 13 percent with differential effects based on the rate at which districts hired alternatively licensed teachers.

More →


Taylor Odle, Lauren C. Russell.

Reverse transfer associate degrees are credentials retroactively awarded to current bachelor’s degree seekers that combine current four-year credits with credits previously earned at a community college. Providing students with an associate degree may not only increase motivation and persistence en route to completing a bachelor’s but may also provide important labor market benefits by way of increased marketability and earnings potential. Despite the proliferation of reverse transfer policies across at least 15 states to date, there is no causal evidence documenting their effect on students’ outcomes. Leveraging administrative data from Tennessee matched with records on its statewide reverse transfer program and a difference-in-differences design, we find reverse transfer degrees generally have little impact on students’ short- and intermediate-term academic and labor market outcomes. Our results point to suggestive yet small positive gains in GPA and short-term employment for recipients, but these estimates accompany no impacts on bachelor’s degree attainment and estimates that confidently reject any meaningful impacts on recipients’ earnings. Our findings contrast those of existing descriptive works on reverse transfer that reported large benefits for students, due in part to our methodological improvements and more robust data. These findings should guide policymakers considering the adoption, design, and ongoing operation of reverse transfer programs.

More →


Matthew A. Kraft, John A. List, Jeffrey A. Livingston, Sally Sadoff.

A substantial body of experimental evidence demonstrates that in-person tutoring programs can have large impacts on K-12 student achievement. However, such programs typically are costly and constrained by a limited local supply of tutors. In partnership with CovEducation (CovEd), we conduct a pilot program that has potential to ease both of these concerns. We conduct an experiment where volunteer tutors from all over the country meet 1-on-1 with middle school students online during the school day. We find that the program produces consistently positive (0.07σ for math and 0.04σ for reading) but statistically insignificant effects on student achievement. While these estimates are notably smaller than those found in many higher-dosage in-person tutoring programs, they are from a significantly lower-cost program that was delivered within the challenging context of the COVID-19 pandemic. We provide evidence that is consistent with a dosage model of tutoring where additional hours result in larger effects.

More →


Brian Heseung Kim, Katharine Meyer, Alice Choe.

Interactive, text message-based advising programs have become an increasingly common strategy to support college access and success for underrepresented student populations. Despite the proliferation of these programs, we know relatively little about how students engage in these text-based advising opportunities and whether that relates to stronger student outcomes – factors that could help explain why we’ve seen relatively mixed evidence about their efficacy to date. In this paper, we use data from a large-scale, two-way text advising experiment focused on improving college completion to explore variation in student engagement using nuanced interaction metrics and automated text analysis techniques (i.e., natural language processing). We then explore whether student engagement patterns are associated with key outcomes including persistence, GPA, credit accumulation, and degree completion. Our results reveal substantial variation in engagement measures across students, indicating the importance of analyzing engagement as a multi-dimensional construct. We moreover find that many of these nuanced engagement measures have strong correlations with student outcomes, even after controlling for student baseline characteristics and academic performance. Especially as virtual advising interventions proliferate across higher education institutions, we show the value of applying a more codified, comprehensive lens for examining student engagement in these programs and chart a path to potentially improving the efficacy of these programs in the future.

Open source code on GitHub.

More →


Katharine Meyer, Lindsay C. Page, Eric Smith, B. Tyler Walsh, C. Lindsey Fifield, Michael Evans.

Despite documented benefits to college completion, more than a third of students who initially enroll in college do not ultimately earn a credential. Completing college requires students to navigate both institutional administrative tasks (e.g., registering for classes) and academic tasks within courses (e.g., completing homework). In postsecondary education, several promising interventions have shown that text-based outreach and communication can be a low-cost, easy to implement, and effective strategy for supporting administrative task navigation. In this paper, we report on a randomized controlled trial testing the effect of a text-based chatbot with artificial intelligence (AI) capability on students' academic task navigation. We find the academic chatbot significantly shifted students’ final grades, increasing the likelihood students received a course grade of B or higher by eight percentage points. We find large and significant treatment effects for first-generation students, estimating the intervention increased their final course grades by about 11 points on a 100-point scale (and a 16 percentage point increase in earning a B or higher) as well as their completion of and performance on individual course deliverables (e.g., readings, activities, exams).

More →


Michael F. Lovenheim, Jonathan Smith.

Early research on the returns to higher education treated the postsecondary system as a monolith.  In reality, postsecondary education in the United States and around the world is highly differentiated, with a variety of options that differ by credential (associates degree, bachelor’s degree, diploma, certificate, graduate degree), the control of the institution (public, private not-for-profit, private for-profit), the quality/resources of the institution, field of study, and exposure to remedial education. In this Chapter, we review the literature on the returns to these different types of higher education investments, which has received increasing attention in recent decades. We first provide an overview of the structure of higher education in the U.S. and around the world, followed by a model that helps clarify and articulate the assumptions employed by different estimators used in the literature. We then discuss the research on the return to institution type, focusing on the return to two-year, four-year, and for-profit institutions as well as the return to college quality within and across these institution types. We also present the research on the return to different educational programs, including vocational credentials, remedial education, field of study, and graduate school. The wide variation in the returns to different postsecondary investments that we document leads to the question of how students from different backgrounds sort into these different institutions and programs. We discuss the emerging research showing that lower-SES students, especially in the U.S., are more likely to sort into colleges and programs with lower returns as well as results from recent U.S.-based interventions and policies designed to support success among students from disadvantaged backgrounds. The Chapter concludes with some broad directions for future research.

More →


Douglas D. Ready, Jeanne L. Reid.

New York City’s Pre-K for All (PKA) is the Nation’s largest universal early childhood initiative, currently serving some 70,000 four-year-olds. Stemming from the program’s choice architecture as well as the City’s stark residential segregation, PKA programs are extremely segregated by child race/ethnicity. Our current study explores the complex forces that influence this segregation, including the interplay between family choices, seat availability, site-level enrollment priorities, and the PKA algorithm that weighs these and other considerations. We find that a majority of PKA segregation lies within rather than between local communities, suggesting that reducing segregation would not necessarily require families to choose programs far from home. On a more troubling note, areas with increased options and greater racial/ethnic diversity also exhibit the most extreme segregation.

More →


Christopher Doss, Hans Fricke, Susanna Loeb, Justin B. Doromal.

This study assesses the effects of two text messaging programs for parents that aim to support the development of math skills in prekindergarten students. One program focuses purely on math, while the other takes an identical approach but focuses on a combination of math, literacy, and social-emotional skills. We find no evidence that the math-only program benefits children’s math development. However, the combination program shows greater promise, particularly for girls. Quantile regressions indicate that the effects are concentrated in the lower half of the outcome distribution. We discuss and provide evidence for various hypotheses that could explain these differences.

Highlights:

  • We test the effects of two text messaging curricula that leverage behavioral economics principles to help parents support the math development of prekindergarteners in the home.
  • We find that a program that cycles through literacy, mathematics, and social-emotional skills increases math achievement for girls, while a program focusing solely on mathematics has no effects.
  • Benefits for girls are concentrated on those with weaker performance on mathematics assessments.
  • We posit potential mechanisms based on the literature.

More →


Dan Goldhaber, Cyrus Grout, Kris Holden, Josh B. McGee.

Over the last two decades, twenty-two states have moved away from traditional defined benefit (DB) pension systems and toward pension plan structures like the defined contribution (DC) plans now prevalent in the private sector. Others are considering such a reform as it is seen as a means of limiting future pension funding risk. It is important to understand the implications of such reforms for end-of-career exit patterns and workforce composition. Empirical evidence on the relationship between pension plan structure and retirement timing is currently limited, primarily because, most state pension reforms are so new that few employees enrolled in those alternative plans have reached retirement age. An exception, and the subject of our analysis, is the teacher retirement system in Washington State, which introduced a hybrid DB-DC plan in 1996 and allowed employees in its traditional DB plan to transfer into the new plan. Our analysis focuses on a years-of-service threshold, the crossing of which grants employees early retirement eligibility and, in many cases, a large upward shift in retirement wealth. The financial implications of crossing this threshold are far greater under the state’s traditional DB plan than under the hybrid plan. We find that employees are responsive to crossing the years-of-service threshold, but we fail to find significant evidence that the propensity to exit the workforce varies according to plan enrollment.

More →