Search EdWorkingPapers

Search for EdWorkingPapers here by author, title, or keywords.

Program and policy effects

Jeffrey T. Denning, Todd R. Jones.

Despite large and growing student loan balances, there is relatively little evidence on the effects of access to student loans on borrowing and educational outcomes. We examine the effect of access to credit by using policy variation in the maximum federal student loan amounts available to U.S. college students. In particular, first-, second-, and third-year students have access to different amounts of federal student loans. Using a regression discontinuity and administrative data from a state higher education system, we find that access to higher loan limits increases borrowing for at least 26 percent of borrowers. Despite this increase in borrowing, we find no evidence that eligibility for additional loans affects student GPA, persistence, or graduation.

More →


Min Sun, Jing Liu, Junmeng Zhu, Zachary LeClair.

Although program evaluations using rigorous quasi-experimental or experimental designs can inform decisions about whether to continue or terminate a given program, they often have limited ability to reveal the mechanisms by which complex interventions achieve their effects. To illuminate these mechanisms, this paper analyzes novel text data from thousands of school improvement planning and implementation reports from Washington State, deploying computer-assisted techniques to extract measures of school improvement processes. Our analysis identified 15 coherent reform strategies that varied greatly across schools and over time. The prevalence of identified reform strategies was largely consistent with school leaders’ own perceptions of reform priorities via interviews. Several reform strategies measures were significantly associated with reductions in student chronic absenteeism and improvements in student achievement. We lastly discuss the opportunities and pitfalls of using novel text data to study reform processes.

More →


Gary T. Henry, Shelby M. McNeill, Erica Harbatkin.

This article contributes to the literature on school turnaround by examining the effect of the North Carolina Transformation (NCT) initiative, which was implemented in 75 low-performing schools after the state’s efforts to turn around the lowest performing schools under Race to the Top ended, on student reading score growth in grades K-3. Reading score growth is measured using the mCLASS Dynamic Indicators of Basic Early Literacy (DIBELS) assessment. Utilizing a regression discontinuity design, we find that the NCT intervention had null effects on K-3 reading score growth across both the 2016 and 2017 school years. 

More →


Adam Kho, Gary T. Henry, Ron Zimmer, Lam Pham.

Many districts and states have begun implementing incentives to attract high-performing teachers to low-performing schools.  Previous research has found that these incentives are effective.  However, effects on the schools and students these teachers leave behind has not been examined.  This study focuses on the general equilibrium effects of recruiting effective teachers to Tennessee’s Innovation Zone (iZone) schools, one of the most successful turnaround initiatives in the nation (Zimmer, Henry, & Kho, 2017).  While there is some variation in the effects of losing these teachers, we find they range between -0.04 and -0.12 standard deviations in student test score gains.  However, an estimate including both these negative effects and the positive effects in iZone schools yields overall net positive effects.

More →


Matthew Kraft, Alvin Christian.

Starting in 2011, Boston Public Schools (BPS) implemented major reforms to its teacher evaluation system with a focus on promoting teacher development. We administered independent district-wide surveys in 2014 and 2015 to capture BPS teachers’ perceptions of the evaluation feedback they receive. Teachers generally reported that evaluators were fair and accurate, but that they struggled to provide high-quality feedback. We conduct a randomized controlled trial to evaluate the district’s efforts to improve this feedback through an intensive training program for evaluators. We find little evidence the program affected evaluators’ feedback, teacher retention, or student achievement. Our results suggest that improving the quality of evaluation feedback may require more fundamental changes to the design and implementation of teacher evaluation systems.

More →


James Soland, Yeow Meng Thum.

Effect sizes in the Cohen’s d family are often used in education to compare estimates across studies, measures, and sample sizes.  For example, effect sizes are used to compare gains in achievement students make over time, either in pre- and post-treatment studies or in the absence of intervention, such as when estimating achievement gaps.  However, despite extensive research dating back to the paired t-test literature showing that such growth effect sizes should account for within-person correlations of scores over time, such achievement gains are often standardized relative to the standard deviation from a single timepoint or two timepoints pooled.  Such a tendency likely occurs in part because there are not many large datasets from which a distribution of student- or school-level gains can be derived.  In this study, we present a novel model for estimating student growth in conjunction with a national dataset to show that effect size estimates for student and school growth are often quite different when standardized relative to a distribution of gains rather than static achievement.  In particular, we provide nationally representative empirical benchmarks for student achievement and gains, including for male-female gaps in those gains, and examine the sensitivity of those effect sizes to how they are standardized.  Our results suggest that effect sizes scaled relative to a distribution of gains are less likely to understate the effects of interventions over time, and that resultant effect sizes often more closely match the estimand of interest for most practice, policy, and evaluation questions.

More →


Angela R. Watson, Jay P. Greene, Heidi H. Erickson, Molly I. Beck.

In recent decades, institutions, teachers, and students report a decline in field trip attendance. The impact of this decline on educational and societal outcomes such as social-emotional skill acquisition is unknown. Social-emotional learning (SEL) are skills thought to be important to life and relationship success and are associated with better long-term student outcomes. This study describes the results of the first-ever longitudinal experiment of the effects of multiple arts-related field trips on elementary school students of color in a large urban school district. Treated students attended three field trips to an art museum, a live theater production, and a symphony performance. We find significant educational benefits from attending multiple arts field trips on social-emotional outcomes, including increased feelings of tolerance and social perspective taking. Our findings also suggest that female treatment students exhibit increased conscientiousness as compared to their control group peers; however, these effects dissipate when treatment ceases. Further, female students who receive three additional field trips in a second treatment year act more conscientious than in the prior year of treatment. Increased exposure to the arts through field trip experiences does not, however, appear to increase students’ desire to consume or participate in the arts, nor do we find an impact of treatment on empathy. These findings suggest that arts-related field trips elicit meaningful changes in students’ social-emotional attitudes and actions and that a decline in field trip attendance may be detrimental.

More →


Jay P. Greene, Heidi H. Erickson, Angela R. Watson, Molly I. Beck.

Field trips to see theater performances are a long-standing educational practice, however, there is little systematic evidence demonstrating educational benefits. This article describes the results of five random assignment experiments spanning two years where school groups were assigned by lottery to attend a live theater performance, or for some groups, watch a movie-version of the same story. We find significant educational benefits from seeing live theater, including higher levels of tolerance, social perspective taking, and stronger command of the plot and vocabulary of those plays. Students randomly assigned to watch a movie did not experience these benefits. Our findings also suggest that theater field trips may cultivate the desire among students to frequent the theater in the future.

More →


Andrew C. Barr, Kelli A. Bird, Benjamin L. Castleman, William Skimmyhorn.

Despite broad public interest in Veterans' education, there is relatively little evidence documenting the postsecondary trajectories of military service members after they return to civilian life. In the current report we investigate how U.S. Army service member college enrollment and progression trends compare to a similar population of civilians, using Army administrative personnel data merged with administrative records from the National Student Clearinghouse and the Educational Longitudinal Study (ELS) of 2002. Civilians were nearly three times as likely to enroll in college within one year of high school graduation (or one year of separation). Civilians were also much more likely to earn a bachelor’s degree within the period of study than either of the Army samples. While members of minority race/ethnicity groups in both military samples enroll at higher rates than their white counterparts, racial/ethnic minorities do not graduate at higher rates than their white counterparts. We discuss policy implications of our analyses in the final section of our paper.

More →


Nicole S. Simon, Susan Moore Johnson, Stefanie K. Reinhorn.

Effective teacher hiring is fundamental to improving schools and yet few studies investigate this process. In this exploratory study of six successful, high-poverty schools (three charter, three district) in one Massachusetts city, we analyze the policy contexts that influenced hiring and examine the schools’ hiring practices. Through interviews with 142 teachers and administrators, we learned that, despite significant differences, these schools’ approaches were strikingly similar. Each used a two-way, information-rich hiring process that provided schools and candidates with opportunities to exchange information and assess one another before making an offer or signing a contract. Participants viewed their investment in hiring as an essential part of their school’s success. Based on our findings, we provide recommendations for policymakers, practitioners, and researchers.

More →