Search for EdWorkingPapers here by author, title, or keywords.
Program and policy effects
Research has begun to investigate whether teachers and schools are as effective with certain student subgroups as they are with the overall student population. Most of this research has examined the issue by trying to produce causal estimates of school contributions to short-term student growth (usually using value-added models) and has emphasized rank orderings of schools by subgroup. However, not much is known about whether schools contributing to long-term growth for all students are also contributing to student growth by subgroup in ways that might close achievement gaps. In this study, schools’ contributions to student growth are estimated separately for Black versus White students. Results show that focusing on rank orderings of schools alone can mask troubling trends in relative achievement over time. Options for how policymakers can sensibly hold schools accountable for student growth, including under The Every Student Succeeds Act, are discussed.
This study is a randomized control trial of full- versus half-day pre-kindergarten in a school district near Denver, Colorado. Four-year-old children were randomly assigned an offer of half-day (four days/week) or full-day (five days/week) pre-k that increased class time by over 600 hours. The offer of full-day pre-k produced substantial, positive effects on children’s receptive vocabulary skills (0.267 standard deviations) by the end of pre-k. Among children enrolled in district schools, full-day participants also outperformed their peers on teacher-reported measures of cognition, literacy, math, and physical development. At kindergarten entry, children offered pre-k still outperformed peers on a widely-used measure of basic literacy. The study provides the first rigorous evidence on the impact of full-day preschool on children’s school readiness skills.
The well-documented racial disparities in school discipline have led many school districts in the U.S. to adopt restorative justice practices. The restorative justice philosophy differs from traditional disciplinary action by placing an emphasis on restitution and improving behavior rather than punishment. While models of restorative justice are descriptively and theoretically promising, research on restorative practices in schools is limited. We use student-level administrative data and a difference-in-difference design to measure the changes in student discipline outcomes that occurred under restorative justice in Pacific City schools between the 2008-2017 school years. Results indicate that restorative justice practices led to an overall reduction in disciplinary action. However, results also show that restorative justice practices had differential effects between racial groups, with White students benefiting most from restorative justice. These findings suggest that while the overall effects of restorative justice are promising, these practices may unintentionally widen the racial disproportionality in school discipline they are instituted to mitigate.
Estimates of school voucher impacts on educational attainment have yet to explore heterogeneities in socioeconomic status among disadvantaged minority students. We theorize reasons for these heterogeneities and then estimate experimentally the differential impacts of voucher offers on college enrollment and graduation rates for minority and non-immigrant students from moderately and severely disadvantaged backgrounds. The findings are obtained from a privately sponsored, lottery-based voucher intervention in New York City that began in 1997. College enrollment and degree attainment as of the fall of 2017 were obtained from the National Student Clearinghouse. We find no significant effects of offers on minority students from severely disadvantaged backgrounds but significant effects of six to eight percentage points on those from moderately disadvantaged households. Similar results are obtained for students born of non-immigrant mothers. Some policy implications are discussed.
Despite large and growing student loan balances, there is relatively little evidence on the effects of access to student loans on borrowing and educational outcomes. We examine the effect of access to credit by using policy variation in the maximum federal student loan amounts available to U.S. college students. In particular, first-, second-, and third-year students have access to different amounts of federal student loans. Using a regression discontinuity and administrative data from a state higher education system, we find that access to higher loan limits increases borrowing for at least 26 percent of borrowers. Despite this increase in borrowing, we find no evidence that eligibility for additional loans affects student GPA, persistence, or graduation.
Although program evaluations using rigorous quasi-experimental or experimental designs can inform decisions about whether to continue or terminate a given program, they often have limited ability to reveal the mechanisms by which complex interventions achieve their effects. To illuminate these mechanisms, this paper analyzes novel text data from thousands of school improvement planning and implementation reports from Washington State, deploying computer-assisted techniques to extract measures of school improvement processes. Our analysis identified 15 coherent reform strategies that varied greatly across schools and over time. The prevalence of identified reform strategies was largely consistent with school leaders’ own perceptions of reform priorities via interviews. Several reform strategies measures were significantly associated with reductions in student chronic absenteeism and improvements in student achievement. We lastly discuss the opportunities and pitfalls of using novel text data to study reform processes.
This article contributes to the literature on school turnaround by examining the effect of the North Carolina Transformation (NCT) initiative, which was implemented in 75 low-performing schools after the state’s efforts to turn around the lowest performing schools under Race to the Top ended, on student reading score growth in grades K-3. Reading score growth is measured using the mCLASS Dynamic Indicators of Basic Early Literacy (DIBELS) assessment. Utilizing a regression discontinuity design, we find that the NCT intervention had null effects on K-3 reading score growth across both the 2016 and 2017 school years.
Many districts and states have begun implementing incentives to attract high-performing teachers to low-performing schools. Previous research has found that these incentives are effective. However, effects on the schools and students these teachers leave behind has not been examined. This study focuses on the general equilibrium effects of recruiting effective teachers to Tennessee’s Innovation Zone (iZone) schools, one of the most successful turnaround initiatives in the nation (Zimmer, Henry, & Kho, 2017). While there is some variation in the effects of losing these teachers, we find they range between -0.04 and -0.12 standard deviations in student test score gains. However, an estimate including both these negative effects and the positive effects in iZone schools yields overall net positive effects.
Starting in 2011, Boston Public Schools (BPS) implemented major reforms to its teacher evaluation system with a focus on promoting teacher development. We administered independent district-wide surveys in 2014 and 2015 to capture BPS teachers’ perceptions of the evaluation feedback they receive. Teachers generally reported that evaluators were fair and accurate, but that they struggled to provide high-quality feedback. We conduct a randomized controlled trial to evaluate the district’s efforts to improve this feedback through an intensive training program for evaluators. We find little evidence the program affected evaluators’ feedback, teacher retention, or student achievement. Our results suggest that improving the quality of evaluation feedback may require more fundamental changes to the design and implementation of teacher evaluation systems.
Effect sizes in the Cohen’s d family are often used in education to compare estimates across studies, measures, and sample sizes. For example, effect sizes are used to compare gains in achievement students make over time, either in pre- and post-treatment studies or in the absence of intervention, such as when estimating achievement gaps. However, despite extensive research dating back to the paired t-test literature showing that such growth effect sizes should account for within-person correlations of scores over time, such achievement gains are often standardized relative to the standard deviation from a single timepoint or two timepoints pooled. Such a tendency likely occurs in part because there are not many large datasets from which a distribution of student- or school-level gains can be derived. In this study, we present a novel model for estimating student growth in conjunction with a national dataset to show that effect size estimates for student and school growth are often quite different when standardized relative to a distribution of gains rather than static achievement. In particular, we provide nationally representative empirical benchmarks for student achievement and gains, including for male-female gaps in those gains, and examine the sensitivity of those effect sizes to how they are standardized. Our results suggest that effect sizes scaled relative to a distribution of gains are less likely to understate the effects of interventions over time, and that resultant effect sizes often more closely match the estimand of interest for most practice, policy, and evaluation questions.