Search EdWorkingPapers

Search EdWorkingPapers by author, title, or keywords.

EdWorkingPapers

Carly D. Robinson, Cynthia Pollard, Sarah Novicoff, Sara White, Susanna Loeb.

In-person tutoring has been shown to improve academic achievement. Though less well-researched, virtual tutoring has also shown a positive effect on achievement but has only been studied in grade five or above. We present findings from the first randomized controlled trial of virtual tutoring for young children (grades K-2). Students were assigned to 1:1 tutoring, 2:1 tutoring, or a control group. Assignment to any virtual tutoring increased early literacy skills by 0.05-0.08 SD with the largest effects for 1:1 tutoring (0.07-0.12 SD). Students initially scoring well below benchmark and first graders experienced the largest gains from 1:1 tutoring (0.15 and 0.20 SD, respectively). Effects are smaller than typically seen from in-person early literacy tutoring programs but still positive and statistically significant, suggesting promise particularly in communities with in-person staffing challenges.

More →


Carly D. Robinson, Katharine Meyer, Chastity Bailey-Fakhoury, Amirpasha Zandieh, Susanna Loeb.

College students make job decisions without complete information. As a result, they may rely on misleading heuristics (“interesting jobs pay badly”) and pursue options misaligned with their goals. We test whether highlighting job characteristics changes decision making. We find increasing the salience of a job’s monetary benefits increases the likelihood college students apply by 196%. In contrast, emphasizing prosocial, career, or social benefits has no effect, despite students identifying these benefits as primary motivators for applying. The study highlights the detrimental incongruencies in students’ decision making alongside a simple strategy for recruiting college students to jobs that offer enriching experiences.

More →


Steven Michael Carlo.

The National Assessment of Educational Progress (NAEP) has tested the civic, or citizenship knowledge of students across the nation at irregular intervals since its very inception. Despite advancements in reading and mathematics, evidenced by results from the National Assessment of Educational Progress (NAEP), civics proficiency has remained consistently low, which raises concerns among educators and policymakers. This study attempts to provide those educators and policymakers with state-level predictions, not currently provided for the civics assessment. This research addresses this gap in state-level civics education data by applying multilevel regression with poststratification (MRP) to NAEP's nationally representative civics scores, yielding state-specific estimates that account for student demographics. A historical analysis of NAEP's development underscores its significance in national education and highlights the challenges of transitioning to state-level reporting, particularly for civics, which lacks state-level generalizability. Furthermore, this paper evaluates NAEP's frameworks, questioning their alignment with civics education's evolving needs, and investigates the presence of opportunity gaps in civics knowledge across gender and racial/ethnic lines. By comparing MRP estimates with published NAEP results, the study validates the method's credibility and emphasizes the potential of MRP in educational research. The findings reveal persistent racial/ethnic disparities in civic knowledge, with profound implications for civics instruction and policy. The research concludes by stressing the necessity for state-specific data to inform education policy and practice, advocating for teaching methods that enhance civic understanding and engagement, and suggesting future research directions to address the uncovered disparities.

More →


Brian Heseung Kim, Julie J. Park, Pearl Lo, Dominique J. Baker, Nancy Wong, Stephanie Breen, Huong Truong, Jia Zheng, Kelly Ochs Rosinger, OiYan Poon.
Letters of recommendation from school counselors are required to apply to many selective colleges and universities. Still, relatively little is known about how this non-standardized component may affect equity in admissions. We use cutting-edge natural language processing techniques to algorithmically analyze a national dataset of over 600,000 student applications and counselor recommendation letters submitted via the Common App platform. We examine how the length and topical content of letters (e.g., sentences about Personal Qualities, Athletics, Intellectual Promise, etc.) relate to student self-identified race/ethnicity, sex, and proxies for socioeconomic status. Paired with regression analyses, we explore whether demographic differences in letter characteristics persist when accounting for additional student, school, and counselor characteristics, as well as among letters written by the same counselor and among students with comparably competitive standardized test scores. We ultimately find large and noteworthy naïve differences in letter length and content across nearly all demographic groups, many in alignment with known inequities (e.g., many more sentences about Athletics among White and higher-SES students, longer letters and more sentences on Personal Qualities for private school students). However, these differences vary drastically based on the exact controls and comparison groups included – demonstrating that the ultimate implications of these letter differences for equity hinges on exactly how and when letters are used in admissions processes (e.g., are letters evaluated at face value across all students, or are they mostly compared to other letters from the same high school or counselor?). Findings do not point to a clear recommendation whether institutions should keep or discard letter requirements, but reflect the importance of reading letters and overall applications in the context of structural opportunity. We discuss additional implications and possible recommendations for college access and admissions policy/practice.

More →


Douglas N. Harris, Jonathan Mills.

We provide evidence about college financial aid from an eight-year randomized trial where high school ninth graders received a $12,000 merit-based grant offer. The program was designed to be free of tuition/fees at community colleges and substantially lower the cost of four-year colleges. During high school, it increased students’ college expectations and low-cost effort, but not higher-cost effort, such as class attendance. The program likely increased two-year college graduation, perhaps because of the free college framing, but did not affect overall college entry, graduation, employment, incarceration, or teen pregnancy. Additional analysis helps explain these modest effects and variation in results across prior studies.

More →


David Blazar, Max Anthenelli, Wenjing Gao, Ramon Goings, Seth Gershenson.

Mounting evidence supporting the advantages of a diverse teacher workforce prompts policymakers to scrutinize existing recruitment pathways. Following four cohorts of Maryland public high-school students over 12 years reveals several insights. Early barriers require timely interventions, aiding students of color in achieving educational milestones that are prerequisites for teacher candidacy (high school graduation, college enrollment). While alternative pathways that bypass traditional undergraduate teacher preparation may help, current approaches still show persistent racial disparities. Data simulations underscore the need for race-conscious policies specifically targeting or differentially benefiting students of color, as race-neutral strategies have minimal impact. Ultimately, multiple race-conscious policy solutions addressing various educational milestones must demonstrate significant effectsapproximately 30% increasesto reshape the teacher workforce to align with student body demographics.

More →


Maxwell J. Cook, Cory Koedel, Michael Reda.

We estimate the education and earnings returns to enrolling in technical two-year degree programs at community colleges in Missouri. A unique feature of the Missouri context is the presence of a highly regarded, nationally ranked technical college: State Technical College of Missouri (State Tech). We find that enrolling in a technical program in Missouri increases the likelihood of associate degree attainment and post-enrollment earnings, but that the positive effects statewide are driven largely by students who attend State Tech. These findings demonstrate the potential for a high-performing community college to change students’ education and labor market trajectories. At the same time, they exemplify the potential for substantial institutional heterogeneity in the returns to postsecondary education.

More →


Brian Holzman, Jeehee Han, Kalena E. Cortes, Bethany Lewis, Irina Chukhray.

This study investigates the role of college major choices in labor market outcomes, with a focus on racial minorities and immigrants. Drawing upon research on school-to-work linkages, we examine two measures, linkage, the connection between college majors and specific occupations in the labor market, and match, the alignment of workers’ occupations with their college majors. Analyzing data from the American Community Survey, 2013-2017, we show that linkage positively predicts earnings, particularly for workers in matched occupations, and negatively predicts unemployment. Notably, Black, Hispanic, and foreign-born workers in matched occupations benefit more from linkage strength than their White and U.S.-born counterparts. This advantage is more pronounced in states that are popular destinations for immigrants. Our findings suggest that earnings and unemployment disparities experienced among racial minorities and immigrants may diminish if they pursue majors closely tied to jobs in the labor market and secure jobs related to their college majors.

More →


Paiheng Xu, Jing Liu, Nathan Jones, Julie Cohen, Wei Ai.

Assessing instruction quality is a fundamental component of any improvement efforts in the education system. However, traditional manual assessments are expensive, subjective, and heavily dependent on observers’ expertise and idiosyncratic factors, preventing teachers from getting timely and frequent feedback. Different from prior research that focuses on low-inference instructional practices, this paper presents the first study that leverages Natural Language Processing (NLP) techniques to assess multiple high-inference instructional practices in two distinct educational settings: in-person K-12 classrooms and simulated performance tasks for pre-service teachers. This is also the first study that applies NLP to measure a teaching practice that has been demonstrated to be particularly effective for students with special needs. We confront two challenges inherent in NLP-based instructional analysis, including noisy and long input data and highly skewed distributions of human ratings. Our results suggest that pretrained Language Models (PLMs) demonstrate performances comparable to the agreement level of human raters for variables that are more discrete and require lower inference, but their efficacy diminishes with more complex teaching practices. Interestingly, using only teachers’ utterances as input yields strong results for student-centered variables, alleviating common concerns over the difficulty of collecting and transcribing high-quality student speech data in in-person teaching settings. Our findings highlight both the potential and the limitations of current NLP techniques in the education domain, opening avenues for further exploration.

More →


Matthew A. Kraft, Melissa Arnold Lyon.

We examine the state of the U.S. K-12 teaching profession over the last half century by compiling nationally representative time-series data on four interrelated constructs: occupational prestige, interest among students, the number of individuals preparing for entry, and on-the-job satisfaction. We find a consistent and dynamic pattern across every measure: a rapid decline in the 1970s, a swift rise in the 1980s extending into the mid 1990s, relative stability, and then a sustained decline beginning around 2010. The current state of the teaching profession is at or near its lowest levels in 50 years. We identify and explore a range of hypotheses that might explain these historical patterns including economic and sociopolitical factors, education policies, and school environments.

More →