Search for EdWorkingPapers here by author, title, or keywords.
Program and policy effects
This study introduces the signal weighted teacher value-added model (SW VAM), a value-added model that weights student-level observations based on each student’s capacity to signal their assigned teacher’s quality. Specifically, the model leverages the repeated appearance of a given student to estimate student reliability and sensitivity parameters, whereas traditional VAMs represent a special case where all students exhibit identical parameters. Simulation study results indicate that SW VAMs outperform traditional VAMs at recovering true teacher quality when the assumption of student parameter invariance is met but have mixed performance under alternative assumptions of the true data generating process depending on data availability and the choice of priors. Evidence using an empirical data set suggests that SW VAM and traditional VAM results may disagree meaningfully in practice. These findings suggest that SW VAMs have promising potential to recover true teacher value-added in practical applications and, as a version of value-added models that attends to student differences, can be used to test the validity of traditional VAM assumptions in empirical contexts.
What guidance does research provide school districts about how to improve system performance and increase equity? Despite over 30 years of inquiry on the topic of effective districts, existing frameworks are relatively narrow in terms of disciplinary focus (primarily educational leadership perspectives) and research design (primarily qualitative case studies). To bridge this gap, we first review the theoretical literatures on how districts are thought to affect student outcomes, arguing that an expanded set of disciplinary perspectives—organizational behavior, political science, and economics—have distinct theories about why districts matter. Next, we conduct a systematic review of quantitative studies that estimate the relationship between district-level inputs and performance outcomes. This review reveals benefits of district-level policies that cross disciplinary perspectives, including higher teacher salaries and strategic hiring, lower student-teacher ratios, and data use. One implication is that future research on district-level policymaking needs to consider multiple disciplinary perspectives. Our review also reveals the need for significant additional causal evidence and provides a multidisciplinary map of theorized pathways through which districts could influence student outcomes that are ripe for rigorous testing.
We study the conditional gender wage gap among faculty at public research universities in the U.S. We begin by using a cross-sectional dataset from 2016 to replicate the long-standing finding in research that conditional on rich controls, female faculty earn less than their male colleagues. Next, we construct a data panel to track the evolution of the wage gap through 2021. We show that the gap is narrowing. It declined by more than 50 percent over the course of our data panel to the point where by 2021, it is no longer detectable at conventional levels of statistical significance.
The absence of federal support leaves undocumented students reliant on state policies to financially support their postsecondary education. We descriptively examine the postsecondary trajectories of tens of thousands of undocumented students newly eligible for California’s state aid program, using detailed application data to compare them to similar peers. In this context, undocumented students who apply and are eligible for the program use grant aid to attend college at rates similar to their peers. Undocumented students remain more likely to enroll in a community college at the expense of attending a broad access four-year college and have higher exit rates from two-year colleges. Yet undocumented students are equally likely to attend the more selective University of California system, and across four-year public colleges have persistence rates similar to their peers, showing that those who do attend four-year colleges perform well.
In spring 2020, nearly every U.S. public school closed at the onset of the Covid-19 pandemic. Existing evidence suggests that local political partisanship and teachers union strength were better predictors of fall 2020 school re-opening status than Covid case and death rates. We replicate and extend these analyses using data collected over the 2020-21 academic year. We demonstrate that Covid case and death rates were meaningfully associated with initial rates of in-person instruction. We also show that all three factors—Covid, partisanship, and teachers unions—became less predictive of in-person instruction as the school year continued. We then leverage data from two nationally representative surveys of Americans’ attitudes toward education and identify an as-yet undiscussed factor that predicts in-person instruction: public support for increasing teacher salaries. We speculate that education leaders were better able to manage the logistical and political complexities of school re-openings in communities with greater support for educators.
Growing literature documents the promise of active learning instruction in engaging students in college classrooms. Accordingly, faculty professional development (PD) programs on active learning have become increasingly popular in postsecondary institutions; yet, quantitative evidence on the effectiveness of these programs is limited. Using administrative data and an individual fixed effects approach, we estimate the effect of an active learning PD program on student performance and persistence at a large public institution. Findings indicate that the training improved subsequent persistence in the same field. Using a subset of instructors whose instruction was observed by independent observers, we identify a positive association between training and implementation of active learning teaching practices. These findings provide suggestive evidence that active learning PD has the potential to improve student outcomes.
Texas reduced new teacher preparation requirements in 2001 to allow more alternate paths to licensure. Within five years, this policy change resulted in over half the state’s new teachers being alternatively licensed. Using a series of first difference models, this study examines the relationship between the increased supply of new teachers in Texas and new teacher salaries prior to the policy change and in the fifteen years thereafter. We find that the policy change did increase the supply of new teachers via alternative licensing, but pay for new EC-6 teachers declined by 2 to 13 percent with differential effects based on the rate at which districts hired alternatively licensed teachers.
Reverse transfer associate degrees are credentials retroactively awarded to current bachelor’s degree seekers that combine current four-year credits with credits previously earned at a community college. Providing students with an associate degree may not only increase motivation and persistence en route to completing a bachelor’s but may also provide important labor market benefits by way of increased marketability and earnings potential. Despite the proliferation of reverse transfer policies across at least 15 states to date, there is no causal evidence documenting their effect on students’ outcomes. Leveraging administrative data from Tennessee matched with records on its statewide reverse transfer program and a difference-in-differences design, we find reverse transfer degrees generally have little impact on students’ short- and intermediate-term academic and labor market outcomes. Our results point to suggestive yet small positive gains in GPA and short-term employment for recipients, but these estimates accompany no impacts on bachelor’s degree attainment and estimates that confidently reject any meaningful impacts on recipients’ earnings. Our findings contrast those of existing descriptive works on reverse transfer that reported large benefits for students, due in part to our methodological improvements and more robust data. These findings should guide policymakers considering the adoption, design, and ongoing operation of reverse transfer programs.
Interactive, text message-based advising programs have become an increasingly common strategy to support college access and success for underrepresented student populations. Despite the proliferation of these programs, we know relatively little about how students engage in these text-based advising opportunities and whether that relates to stronger student outcomes – factors that could help explain why we’ve seen relatively mixed evidence about their efficacy to date. In this paper, we use data from a large-scale, two-way text advising experiment focused on improving college completion to explore variation in student engagement using nuanced interaction metrics and automated text analysis techniques (i.e., natural language processing). We then explore whether student engagement patterns are associated with key outcomes including persistence, GPA, credit accumulation, and degree completion. Our results reveal substantial variation in engagement measures across students, indicating the importance of analyzing engagement as a multi-dimensional construct. We moreover find that many of these nuanced engagement measures have strong correlations with student outcomes, even after controlling for student baseline characteristics and academic performance. Especially as virtual advising interventions proliferate across higher education institutions, we show the value of applying a more codified, comprehensive lens for examining student engagement in these programs and chart a path to potentially improving the efficacy of these programs in the future.
A substantial body of experimental evidence demonstrates that in-person tutoring programs can have large impacts on K-12 student achievement. However, such programs typically are costly and constrained by a limited local supply of tutors. In partnership with CovEducation (CovEd), we conduct a pilot program that has potential to ease both of these concerns. We conduct an experiment where volunteer tutors from all over the country meet 1-on-1 with middle school students online during the school day. We find that the program produces consistently positive (0.07σ for math and 0.04σ for reading) but statistically insignificant effects on student achievement. While these estimates are notably smaller than those found in many higher-dosage in-person tutoring programs, they are from a significantly lower-cost program that was delivered within the challenging context of the COVID-19 pandemic. We provide evidence that is consistent with a dosage model of tutoring where additional hours result in larger effects.