Search EdWorkingPapers
Search EdWorkingPapers by author, title, or keywords.
Search
EdWorkingPapers
Standards have been at the heart of state and federal efforts to improve education for several decades. Most recently, standards-based reforms have evolved with a focus on more ambitious "college- and career-ready" (CCR) standards. This paper synthesizes the results of a seven-year national research center focused on the implementation and effects of CCR standards. The paper draws on evidence from a quasi-experimental longitudinal study using NAEP data, a cluster-randomized trial of an alignment feedback intervention, and detailed implementation data from state-representative surveys and case studies of five districts. Situating our work in a "policy attributes theory," we find important gaps in the theory of change underlying current standards-based reform efforts. We conclude that the CCR standards movement is not succeeding in achieving its desired outcomes. We make specific suggestions for improving instructional policy, including a) providing more specific instructional guidance, b) reconceptualizing professional learning, c) building buy-in through the involvement of trusted leaders, d) providing better supports for differentiation, and e) devoting attention and guidance to the intersection of content and pedagogy, and f) addressing persistent deficit thinking among educators.
In recent decades, U.S. education leaders have advocated for more intellectually ambitious mathematics instruction in classrooms. Evidence about whether more ambitious mathematics instruction has filtered into contemporary classrooms, however, is largely anecdotal. To address this issue, we analyzed 93 lessons recorded by a national random sample of middle school mathematics teachers. We find that lesson quality varies, with the typical lesson containing some elements of mathematical reasoning and sense-making, but also teacher-directed instruction with limited student input. Lesson quality correlates with teachers’ use of a textbook and with teachers’ mathematical background. We consider these findings in light of efforts to transform U.S. mathematics instruction.
Teachers' sense-making of student behavior determines whether students get in trouble and are formally disciplined. Status categories, such as race, can influence perceptions of student culpability, but the degree to which this contributes to racial disproportionality in discipline receipt is unknown. This study provides the first systematic documentation of teachers' use office discipline referrals (ODRs) in a large, diverse urban school district in California that specifies the identity of both the referred and referring individuals in all ODRs. We identify teachers exhibiting extensive referral behavior, or the top 5% referrers based on the number of ODRs they make in a given year and evaluate their contributions to disciplinary disparities. We find that "top referrers" effectively double the racial gaps in ODRs for both Black-White and Hispanic-White comparisons. These gaps are mainly driven by higher numbers of ODRs issued for Black and Hispanic students due to interpersonal offences and defiance, and also partially convert to racial gaps in suspensions. Both the level and racial compositions of the school sites where "top referrers" serve and their personal traits seem to explain some of their frequent referring behavior. Targeting supports and interventions to "top referrers" might afford an important opportunity to reduce racial disciplinary gaps.
Analyses that reveal how treatment effects vary allow researchers, practitioners, and policymakers to better understand the efficacy of educational interventions. In practice, however, standard statistical methods for addressing Heterogeneous Treatment Effects (HTE) fail to address the HTE that may exist within outcome measures. In this study, we present a novel application of the Explanatory Item Response Model (EIRM) for assessing what we term “item-level” HTE (IL-HTE), in which a unique treatment effect is estimated for each item in an assessment. Results from data simulation reveal that when IL-HTE are present but ignored in the model, standard errors can be underestimated and false positive rates can increase. We then apply the EIRM to assess the impact of a literacy intervention focused on promoting transfer in reading comprehension on a digital formative assessment delivered online to approximately 8,000 third-grade students. We demonstrate that allowing for IL-HTE can reveal treatment effects at the item-level masked by a null average treatment effect, and the EIRM can thus provide fine-grained information for researchers and policymakers on the potentially heterogeneous causal effects of educational interventions.
After near-universal school closures in the United States at the start of the pandemic, lawmakers and educational leaders made plans for when and how to reopen schools for the 2020-21 school year. Educational researchers quickly assessed how a range of public health, political, and demographic factors were associated with school reopening decisions and parent preferences for in-person and remote learning. I review this body of literature, to highlight what we can learn from its findings, limitations, and influence on public discourse. Studies consistently highlighted the influence of partisanship, teachers’ unions, and demographics, with mixed findings on COVID-19 rates. The literature offers useful insight and requires more evidence, and it highlights benefits and limitations to rapid research with large-scale quantitative data.
School principals are viewed as critical mechanisms by which to improve student outcomes, but there remain important methodological questions about how to measure principals' effects. We propose a framework for measuring principals' contributions to student outcomes and apply it empirically using data from Tennessee, New York City, and Oregon. We find that using contemporaneous student outcomes to assess principal performance is flawed. Value-added models misattribute to principals changes in student performance caused by factors that principals minimally control. Further, little to none of the variation in average student test scores or attendance is explained by persistent effectiveness differences between principals.
To boost college graduation rates, policymakers often advocate for academic supports such as coaching or mentoring. Proactive and intensive coaching interventions are effective, but are costly and difficult to scale. We evaluate a relatively lower-cost group coaching program targeted at first-year college students placed on academic probation. Participants attend a workshop where coaches aim to normalize failure and improve self-confidence. Coaches also facilitate a process whereby participants reflect on their academic difficulties, devise solutions to address their challenges, and create an action plan. Participants then hold a one-time follow-up meeting with their coach or visit a campus resource. Using a difference-in-discontinuity design, we show that the program raises students’ first-year GPA by 14.6% of a standard deviation, and decreases the probability of first-year dropout by 8.5 percentage points. Effects are concentrated among lower-income students who also experience a significant increase in the probability of graduating. Finally, using administrative data we provide the first evidence that coaching/mentoring may have substantial long-run effects as we document significant gains in lower-income students’ earnings 7–9 years following entry to the university. Our findings indicate that targeted, group coaching can be an effective way to improve marginal students’ academic and early career outcomes.
What happens when employers would like to screen their employees but only observe a subset of output? We specify a model in which heterogeneous employees respond by producing more of the observed output at the expense of the unobserved output. Though this substitution distorts output in the short-term, we derive three sufficient conditions under which the heterogenous response improves screening efficiency: 1) all employees place similar value on staying in their current role; 2) the employees' utility functions satisfy a variation of the traditional single-crossing condition; 3) employer and worker preferences over output are similar. We then assess these predictions empirically by studying a change to teacher tenure policy in New York City, which increased the role that a single measure -- test score value-added -- played in tenure decisions. We show that in response to the policy teachers increased test score value-added and decreased output that did not enter the tenure decision. The increase in test score value-added was largest for the teachers with more ability to improve students' untargeted outcomes, increasing their likelihood of getting tenure. We find that the endogenous response to the policy announcement reduced the screening efficiency gap -- defined as the reduction of screening efficiency stemming from the partial observability of output -- by 28%, effectively shifting some of the cost of partial observability from the post-tenure period to the pre-tenure period.
Performance-based funding models for higher education, which tie state support for institutions to performance on student outcomes, have proliferated in recent decades. Some states have designed these policies to also address educational attainment gaps by including bonus payments for traditionally low-performing groups. Using a Synthetic Control Method research design, we examine the impact of these funding regimes on race-based completion gaps in Tennessee and Ohio. We find no evidence that performance-based funding narrowed race-based completion gaps. In fact, contrary to their intended purpose, we find that performance-based funding widened existing gaps in certificate completion in Tennessee. Across both states, the estimated impacts on associate degree outcomes are also directionally consistent with performance-based funding exacerbating racial inequities in associate degree attainment.