Search EdWorkingPapers

Search EdWorkingPapers by author, title, or keywords.

Program and policy effects

Christopher Cleveland, Ethan Scherer.

A growing body of research shows that students benefit when they demographically match their teachers. However, little is known about how matching affects social-emotional development. We use student-fixed effects to exploit changes over time in the proportion of teachers within a school grade who demographically match a student to estimate matching's effect on social-emotional measures, test scores, and behavioral outcomes. We find improvements for students in grit and interpersonal self-management when matched to teachers of their race and gender. Black female students drive these effects. We also find that matching reduces absences, especially for Black students. Our findings add to the emerging teacher diversity literature by showing its benefits for Black and female students during a critical stage of development.

More →


Catherine Armstrong Asher, Ethan Scherer, James S. Kim, Johanna Norshus Tvedt.

We leverage log data from an educational app and two-way text message records from over 3,500 students during the summers of 2019 and 2020, along with in-depth interviews in Spanish and English, to identify patterns of family engagement with educational technology. Based on the type and timing of technology use, we identify several distinct profiles of engagement, which we group into two categories: Independent Users who engage with technology-based educational software independently, and Interaction-Supported Users who use two-way communications to support their engagement. We also find that as the demands of families from schools increased during the COVID-19 pandemic, Spanish-speaking families were significantly more likely than English-speaking families to engage with educational technology across all categories of families, particularly as Interaction-Supported Users.

More →


Andrew Kwok, Iliana De La Cruz, Michelle Kwok.

Field supervisors are central to clinical teaching, but little is known about how their feedback informs preservice teachers (PSTs) development. This sequential mixed methods study examines over 3,000 supervisor observation evaluations. We qualitatively code supervisor written feedback, which indicates 2 broad pedagogical categories and 9 separate skills. We then quantize these feedback codes to identify the variation in the presence of these codes across PST characteristics, and then use several modeling techniques to indicate that specific feedback codes are negatively associated with evaluation score. Managing student attention was most detrimental to scores in early observations whereas instructional feedback (e.g., lesson delivery) was prioritized later in clinical teaching. Findings inform teacher preparation policy on understanding PST development and improving supervisory feedback.

More →


Lauren P. Bailes, Sarah Guthery.

This study examines the experience of demotion from a principalship to an assistant principalship and how race and gender can differentially impact career trajectories. Using administrative state dataset of 10,946 observations at the principal level, we used probit regression to determine the overall probability of demotion and Kaplan Meier survival analysis to estimate the differences in probability over time. Our analysis describes not only who experiences demotions, but includes the characteristics of the sending and receiving schools. Survival analysis illustrates how small differences over time in demotion by race resulted in statistically significant systemic differences. We also find that experience matters: for every additional year of experience in the principal role, the probability of experiencing demotion decreases by 0.34%.

More →


Kalena E. Cortes, Karen Kortecamp, Susanna Loeb, Carly D. Robinson.

This paper presents the results from a randomized controlled trial of Chapter One, an early elementary reading tutoring program that embeds part-time tutors into the classroom to provide short bursts of 1:1 instruction. Eligible kindergarten students were randomly assigned to receive supplementary tutoring during the 2021-22 school year (N=818). The study occurred in a large Southeastern district serving predominantly Black and Hispanic students. Students assigned to the program were over two times more likely to reach the program’s target reading level by the end of kindergarten (70% vs. 32%). The results were largely homogenous across student populations and extended to district-administered assessments. These findings provide promising evidence of an affordable and sustainable approach for delivering personalized reading tutoring at scale.

More →


James S. Kim, Joshua B. Gilbert, Jackie E. Relyea, Patrick Rich, Ethan Scherer, Mary A. Burkhauser, Johanna N. Tvedt.
We investigated the effectiveness of a sustained and spiraled content literacy intervention that emphasizes building domain and topic knowledge schemas and vocabulary for elementary-grade students. The Model of Reading Engagement (MORE) intervention underscores thematic lessons that provide an intellectual structure for helping students connect new learning to a general schema in Grade 1 (animal survival), Grade 2 (scientific investigation of past events like dinosaur mass extinctions), and Grade 3 (scientific investigation of living systems). A total of 30 elementary schools (N = 2,870 students) were randomized to a treatment or control condition. In the treatment condition (i.e., full spiral curriculum), students participated in content literacy lessons from Grades 1 to 3 during the school year and wide reading of thematically related informational texts in the summer following Grades 1 and 2. In the control condition (i.e., partial spiral curriculum), students participated in lessons in only Grade 3. The Grade 3 lessons for both conditions were implemented online during the COVID-19 pandemic school year. Results reveal that treatment students outperformed control students on science vocabulary knowledge across all three grades. Furthermore, intent-to-treat analyses revealed positive transfer effects on Grade 3 science reading (ES = .14), domain-general reading comprehension (ES = .11), and mathematics achievement (ES = .12). Treatment impacts were sustained at 14-month follow-up on Grade 4 reading comprehension (ES = .12) and mathematics achievement (ES = .16). Findings indicate that a content literacy intervention that spirals topics and vocabulary across grades can improve students’ long-term academic achievement outcomes.

More →


Sarah Novicoff, Thomas S. Dee.

While policymakers have demonstrated considerable enthusiasm for “science of reading” initiatives, the evidence on the impact of related reforms when implemented at scale is limited. In this pre-registered, quasi-experimental study, we examine California’s recent initiative to improve early literacy across the state’s lowest-performing elementary schools. The Early Literacy Support Block Grant (ELSBG) provided teacher professional development grounded in the science of reading as well as aligned supports (e.g., assessments and interventions), new funding (about $1000 per student), spending flexibility within specified guidelines, and expert facilitation and oversight of school-based planning. We find that ELSBG generated significant (and cost-effective) improvements in ELA achievement in its first two years of implementation (0.14 SD) as well as smaller, spillover improvements in math achievement.

More →


Anamarie A. Whitaker, Margaret Burchinal, Jade M. Jenkins, Drew H. Bailey, Tyler W. Watts, Greg J. Duncan, Emma R. Hart, Ellen S. Peisner-Feinberg.

High-quality preschool programs are heralded as effective policy solutions to promote low-income children’s development and life-long wellbeing. Yet evaluations of recent preschool programs produce puzzling findings, including negative impacts, and divergent, weaker results than demonstration programs implemented in the 1960s and 70s. We provide potential explanations for why modern preschool programs have become less effective, focusing on changes in instructional practices and counterfactual conditions. We also address popular theories that likely do not explain weakening program effectiveness, such as lower preschool quality and low-quality subsequent environments. The field must take seriously the smaller positive, null, and negative impacts from modern programs and strive to understand why effects differ and how to improve program effectiveness through rigorous, longitudinal research.

More →


Beth Schueler, Liz Nigro, John Wang.

The improvement of low-performing school systems is one potential strategy for mitigating educational inequality. Some evidence suggests districtwide reform may be more effective than school-level change, but limited research examines district-level turnaround. There is also little scholarship examining the effects of turnaround reforms on outcomes beyond the first few years of implementation, on outcomes beyond test scores, or on the effectiveness of efforts to replicate district improvement successes beyond an initial reform context. We study these topics in Massachusetts, home to the Lawrence district representing a rare case of demonstrated improvements in the early years of state takeover and turnaround and where state leaders have since intervened in three other contexts as a result. We use statewide student-level administrative data (2006-07 to 2018-19) and event study methods to estimate medium-term reform impacts on test and non-test outcomes across four Massachusetts-based contexts: Lawrence, Holyoke, Springfield, and Southbridge. We find substantial district improvement was possible although sustaining the rate of gains was more complicated. Replicating gains in new contexts was also possible but not guaranteed. 

More →


Henry T. Woodyard, Tim R. Sass, Ishtiaque Fazlul.

Numerous studies have demonstrated a strong link between participation in pre-K programs and both short-term student achievement and positive later-life outcomes.  Existing evidence primarily stems from experimental studies of small-scale, high-quality programs conducted in the 1960s and 1970s and analyses of the federal Head Start program.  Meanwhile, evidence on state-funded pre-K programs, with no income restrictions, is scant and inconclusive.  Using enrollment lotteries for over-subscribed school-based sites in Georgia’s universal pre-K program, we analyze the impact of participation on elementary school outcomes.  Lottery winners enter kindergarten more prepared in both math and reading than non-winning peers. Gains fade by the end of kindergarten, and some negative achievement effects emerge by grade 4. Free-and-reduced-price meal (FRPM) students benefit more compared to non-FRPM students in later grades, suggesting greater benefits from attendance for disadvantaged students.  Although we find no effects for discipline, lottery winners had one fewer absence each grade after kindergarten.

More →