Search EdWorkingPapers

Search for EdWorkingPapers here by author, title, or keywords.

Multiple outcomes of education

Brian Heseung Kim, Katharine Meyer, Alice Choe.

Interactive, text message-based advising programs have become an increasingly common strategy to support college access and success for underrepresented student populations. Despite the proliferation of these programs, we know relatively little about how students engage in these text-based advising opportunities and whether that relates to stronger student outcomes – factors that could help explain why we’ve seen relatively mixed evidence about their efficacy to date. In this paper, we use data from a large-scale, two-way text advising experiment focused on improving college completion to explore variation in student engagement using nuanced interaction metrics and automated text analysis techniques (i.e., natural language processing). We then explore whether student engagement patterns are associated with key outcomes including persistence, GPA, credit accumulation, and degree completion. Our results reveal substantial variation in engagement measures across students, indicating the importance of analyzing engagement as a multi-dimensional construct. We moreover find that many of these nuanced engagement measures have strong correlations with student outcomes, even after controlling for student baseline characteristics and academic performance. Especially as virtual advising interventions proliferate across higher education institutions, we show the value of applying a more codified, comprehensive lens for examining student engagement in these programs and chart a path to potentially improving the efficacy of these programs in the future.

Open source code on GitHub.

More →


Christine Mulhern, Isaac M. Opper.

There is an emerging consensus that teachers impact multiple student outcomes, but it remains unclear how to measure and summarize the multiple dimensions of teacher effectiveness into simple metrics for research or personnel decisions. We present a multidimensional empirical Bayes framework and illustrate how to use noisy estimates of teacher effectiveness to assess the dimensionality and predictive power of teachers' true effects. We find that it is possible to efficiently summarize many dimensions of effectiveness and most summary measures lead to similar teacher rankings; however, focusing on any one specific measure alone misses important dimensions of teacher quality.

More →


Lucas Husted, Isaac M. Opper, R. Jisung Park.

We show that natural disasters affect a region’s aggregate human capital through at least four channels. In addition to causing out-migration, natural disasters reduce student achievement, lower high school graduation rates, and decrease post-secondary attendance. We estimate that disasters that cause at least $500 in per capita property damage reduce the net present value (NPV) of an affected county’s human capital by an average of $505 per person. These negative effects on human capital are not restricted to large disasters: less severe events – disasters with property damages of $100-$500 per capita – also cause significant and persistent reductions in student achievement and post-secondary attendance.

More →


Shamena Anwar, Matthew Baird, John Engberg, Rosanna Smart.

The primary goal of job training programs is to improve employment and earning outcomes of participants. However, effective job training programs may have potential secondary benefits, including in the form of reduced arrests. In this paper, we evaluate the impact of a job training program in New Orleans that was implemented using a randomized controlled trial design. We find that among those who had a prior criminal record, those assigned to the treatment group were two-fifths as likely to get arrested as those assigned to the control group at any time point after randomization. We explore several potential mechanisms for why this effect occurs and find suggestive evidence that the training program’s impact on wages, as well as peer effects from other trainees, can partially explain this effect.

More →


Sidrah Baloch, Thomas J. Kane, Ethan Scherer, Douglas O. Staiger.

Educators must balance the needs of students who start the school year behind grade level with their obligation to teach grade-appropriate content to all students. Educational software could help educators strike this balance by targeting content to students’ differing levels of mastery. Using a regression discontinuity design and detailed software log and administrative data, we compare two versions of an online mathematics program used by students in three education agencies. We find that although students assigned the modified curriculum did progress through content objectives more quickly than students assigned the default curriculum, they did not perform better on pre- and post-objective quizzes embedded in the software, and most never progressed far enough to reach the grade-level content. Furthermore, there was no statistically significant effect of the modified curriculum on formative test scores. These findings suggest policymakers and practitioners should exercise caution when assigning exclusively remedial content to students who start the school year behind grade level, even though this is a common feature of many math educational software programs.

More →


Zid Mancenido.

Many teacher education researchers have expressed concerns with the lack of rigorous impact evaluations of teacher preparation practices. I summarize these various concerns as they relate to issues of internal validity, external validity, and measurement. I then assess the prevalence of these issues by reviewing 166 impact evaluations of teacher preparation practices published in peer-reviewed journals between 2002-2019. Although I find that very few studies address issues of internal validity, external validity and measurement, I highlight some innovative approaches and present a checklist of considerations to assist future researchers in designing more rigorous impact evaluations.

More →


Jeanne Lafortune, Todd Pugatch, José Tessada, Diego Ubfal.

We study the short-run effects of a gamified online entrepreneurship training offered to high school students in Rwanda during the COVID-19 pandemic. Using a randomized controlled trial, we estimate sizeable effects of the 6-week training on entrepreneurial activity. One month after the training, participants in schools offered the training were much more likely to own a business than participants in control schools. The training induced students to participate more actively in their school's business club, to undertake more business-oriented actions, to improve their business practices, and to interact more with other youth and family members about their business ideas. We hypothesize that the training might have motivated treated students to sustain their business activities during the COVID-19 crisis.

More →


Serena Canaan, Pierre Mouganie, Peng Zhang.

Despite the prevalence of school tracking, evidence on whether it improves student success is mixed. This paper studies how tracking within high school impacts high-achieving students’ short- and longer-term academic outcomes. Our setting is a large and selective Chinese high school, where first-year students are separated into high-achieving and regular classrooms based on their performance on a standardized exam. Classrooms differ in terms of peer ability, teacher quality, class size, as well as level and pace of instruction. Using newly collected administrative data and a regression discontinuity design, we show that high-achieving classrooms improve math test scores by 23 percent of a standard deviation, with effects persisting throughout the three years of high school. Effects on performance in Chinese and English language subjects are more muted. Importantly, we find that high-achieving classrooms substantially raise enrollment in elite universities, as they increase scores on the national college entrance exam—the sole determinant of university admission in China.

More →


Soobin Kim, John Yun, Barbara Schneider, Michael Broda, Christopher Klager, I-Chien Chen.

We study the long-term effects of a psychological intervention on longitudinal academic outcomes and degree completion of college students. All freshmen at a large public university were randomized to an online growth mindset, belonging, or control group. We tracked students’ academic outcomes including GPA, number of credits attempted and earned, major choices, and degree completion. We found no evidence of longitudinal academic treatment effects in the full sample. However, the mindset treatment improved term GPAs for Latinx students and the probability for Pell-eligible and Latinx students to major in selective majors. We also found no evidence of increased rates of on-time graduation, however, the treatment raised the probability to graduate with selective majors in four years, especially for Latinx students.

More →


Emily Morton.

Four-day school weeks have proliferated across the United States in recent years, reaching over 650 public school districts in 24 states as of 2019, but little is known about their implementation and there is no consensus on their effects on students. This study uses district level panel data from Oklahoma and a difference-in-differences research design to provide estimates of the causal effect of the four-day school week on high school students’ ACT scores, attendance, and disciplinary incidents during school. Results indicate that four-day school weeks decrease per-pupil bullying incidents by approximately 39% and per-pupil fighting incidents by approximately 31%, but have no detectable effect on other incident types, ACT scores, or attendance.

More →