Search EdWorkingPapers
Search EdWorkingPapers by author, title, or keywords.
Search
Methodology, measurement and data
Despite wide achievement gaps across California between students from different racial and socioeconomic backgrounds, some school districts have excelled at supporting the learning of all their students. This analysis identifies these positive outlier districts—those in which students of color, as well as White students, consistently achieve at higher levels than students from similar racial/ethnic backgrounds and from families of similar income and education levels in most other districts. These results are predicted, in significant part, by the qualifications of districts’ teachers, as measured by their certification and experience. In particular, the proportion of underprepared teachers—those teaching on emergency permits, waivers, and intern credentials—is associated with decreased achievement for all students, while teaching experience is associated with increased achievement, especially for students of color.
We show that grit, a skill that has been shown to be highly predictive of achievement, is malleable in childhood and can be fostered in the classroom environment. We evaluate a randomized educational intervention implemented in two independent elementary school samples. Outcomes are measured via a novel incentivized real effort task and performance in standardized tests. We find that treated students are more likely to exert effort to accumulate task-specific ability, and hence, more likely to succeed. In a follow up 2.5 years after the intervention, we estimate an effect of about 0.2 standard deviations on a standardized math test.
Despite large schooling and learning gains in many developing countries, children in highly deprived areas are often unlikely to achieve even basic literacy and numeracy. We study how much of this problem can be resolved using a multi-pronged intervention combining several distinct interventions known to be effective in isolation. We conducted a cluster-randomized trial in The Gambia evaluating a literacy and numeracy intervention designed for primary-aged children in remote parts of poor countries. The intervention combines para teachers delivering after-school supplementary classes, scripted lesson plans, and frequent monitoring focusing on improving teacher practice (coaching). A similar intervention previously demonstrated large learning gains in a cluster-randomized trial in rural India. After three academic years, Gambian children receiving the intervention scored 46 percentage points (3.2 SD) better on a combined literacy and numeracy test than control children. This intervention holds great promise to address low learning levels in other poor, remote settings.
Up to three-fourths of college students can be classified as “non-traditional”, yet whether typical policy interventions improves their education and labor market outcomes is understudied. I use a regression discontinuity design to estimate the impacts of a state financial aid program aimed towards non-traditional students. Eligibility has no impacts on degree completion for students intending to enroll in community colleges or four-year colleges but increases bachelor’s degrees for students interested in large, for-profit colleges by four percentage points. I find no impacts on employment or earnings for all applicants. This research highlights challenges in promoting human capital investment for adults.
This paper presents new experimental estimates of the impact of low-ability peers on own outcomes using nationally representative data from China. We exploit the random assignment of students to junior high school classrooms and find that the proportion of low-ability peers, defined as having been retained during primary school (“repeaters”), has negative effects on non-repeaters’ cognitive and non-cognitive outcomes. An exploration of the mechanisms shows that a larger proportion of repeater peers is associated with reduced after-school study time. The negative effects are driven by male repeaters and are more pronounced among students with less strict parental monitoring at home.
The sustaining environments hypothesis refers to the popular idea, stemming from theories in developmental, cognitive, and educational psychology, that the long-term success of early educational interventions is contingent on the quality of the subsequent learning environment. Several studies have investigated whether specific kindergarten classroom and other elementary school factors account for patterns of persistence and fadeout of early educational interventions. These analyses focus on the statistical interaction between an early educational intervention – usually whether the child attended preschool – and several measures of the quality of the subsequent educational environment. The key prediction of the sustaining environments hypothesis is a positive interaction between these two variables. To quantify the strength of the evidence for such effects, we meta-analyze existing studies that have attempted to estimate interactions between preschool and later educational quality in the United States. We then attempt to establish the consistency of the direction and a plausible range of estimates of the interaction between preschool attendance and subsequent educational quality by using a specification curve analysis in a large, nationally representative dataset that has been used in several recent studies of the sustaining environments hypothesis. The meta-analysis yields small positive interaction estimates ranging from approximately .00 to .04, depending on the specification. The specification curve analyses yield interaction estimates of approximately 0. Results suggest that the current mix of methods used to test the sustaining environments hypothesis cannot reliably detect realistically sized effects. Our recommendations are to combine large sample sizes with strong causal identification strategies, and to study combinations of interventions that have a strong probability of showing large main effects.
Policymakers are increasingly including early-career earnings data in consumer-facing college search tools to help students and families make more informed post-secondary education decisions. We offer new evidence on the degree to which existing college-specific earnings data equips consumers with useful information by documenting the level of selection bias in the earnings metrics reported in the U.S. Department of Education’s College Scorecard. Given growing interest in reporting earnings by college and major, we focus on the degree to which earnings differences across four-year colleges and universities can be explained by differences in major composition across institutions. We estimate that more than three-quarters of the variation in median earnings across institutions is explained by observable factors, and accounting for differences in major composition explains over 30 percent of the residual variation in earnings after controlling for institutional selectivity, student composition, and local cost of living differences. We also identify large variations in the distribution of earnings within colleges; as a result, comparisons of early-career earnings can be extremely sensitive to whether the median, 25th, or 75th percentiles are presented. Taken together, our findings indicate that consumers can easily draw misleading conclusions about institutional quality when using publicly available earnings data to compare institutions.
Federal education policies gave political and financial support for state education agencies to turnaround low-performing schools on an unprecedented scale. North Carolina’s ambitious program turned around over half of all schools nationwide that underwent turnaround funded by Race to the Top. Exploiting the assignment to turnaround based on schools’ 2009-10 proficiency rates, we implement regression discontinuity designs to estimate the effects of state turnaround services on student achievement in North Carolina’s lowest-performing schools annually from the 2011-12 through 2014-15. Overall, we find modest positive effects of turnaround when including treated schools at all grade levels, but these effects are sensitive to bandwidth. For secondary schools, we find consistently positive effects that vary from modest to large. For elementary and middle schools, we find consistent, modest negative effects of turnaround on student achievement.
Research has begun to investigate whether teachers and schools are as effective with certain student subgroups as they are with the overall student population. Most of this research has examined the issue by trying to produce causal estimates of school contributions to short-term student growth (usually using value-added models) and has emphasized rank orderings of schools by subgroup. However, not much is known about whether schools contributing to long-term growth for all students are also contributing to student growth by subgroup in ways that might close achievement gaps. In this study, schools’ contributions to student growth are estimated separately for Black versus White students. Results show that focusing on rank orderings of schools alone can mask troubling trends in relative achievement over time. Options for how policymakers can sensibly hold schools accountable for student growth, including under The Every Student Succeeds Act, are discussed.
Although program evaluations using rigorous quasi-experimental or experimental designs can inform decisions about whether to continue or terminate a given program, they often have limited ability to reveal the mechanisms by which complex interventions achieve their effects. To illuminate these mechanisms, this paper analyzes novel text data from thousands of school improvement planning and implementation reports from Washington State, deploying computer-assisted techniques to extract measures of school improvement processes. Our analysis identified 15 coherent reform strategies that varied greatly across schools and over time. The prevalence of identified reform strategies was largely consistent with school leaders’ own perceptions of reform priorities via interviews. Several reform strategies measures were significantly associated with reductions in student chronic absenteeism and improvements in student achievement. We lastly discuss the opportunities and pitfalls of using novel text data to study reform processes.