Methods
Item-Level Heterogeneity in Value Added Models: Implications for Reliability, Cross-Study Comparability, and Effect Sizes
Value added models (VAMs) attempt to estimate the causal effects of teachers and schools on student test scores. We apply Generalizability Theory to show how estimated VA effects depend upon the selection of test items. Standard VAMs estimate causal effects on the items that are included on the… more →
Disparate Teacher Effects, Comparative Advantage, and Match Quality
Does student-teacher match quality exist? While prior research documents disparities in teachers' impacts across student types, it has not distinguished between sorting and causal effects as the drivers of these disparities. I develop a flexible disparate value-added model (DVA) and introduce a… more →
Count Me In? Identifying Factors That Predict Centers’ Application to Boston’s Mixed-Delivery Universal Pre-K Program
Universal prekindergarten (UPK) programs often expand through mixed-delivery systems by offering seats in public schools and community-based centers (CBOs). Although this approach aims to meet varied family needs, little is known about potential systematic differences between CBOs that apply to… more →
Educator Attention: How computational tools can systematically identify the distribution of a key resource for students
Educator attention is critical for student success, yet how educators distribute their attention across students remains poorly understood due to data and methodological constraints. This study presents the first large-scale computational analysis of educator attention patterns, leveraging over… more →
Combining Early Grade Assessments to Study Literacy Skills: Addressing the Variability in Tests Taken across Schools and Students
There is considerable variability in the literacy assessments taken in Kindergarten through second grade, across schools and between multilingual learners and other students, and within students over time. This makes it difficult to study changes in students’ acquisition of ELA skills in these… more →
The Correlated Proxy Problem: Why Control Variables can Obscure the Contribution of Selection Processes to Group-Level Inequality
Whether selection processes contribute to group-level disparities or merely reflect pre-existing inequalities is an important societal question. In the context of observational data, researchers, concerned about omitted-variable bias, assess selection-contributing inequality via a kitchen-sink… more →
Addressing Threats to Validity in Supervised Machine Learning: A Framework and Best Practices for Education Researchers
Given the rapid adoption of machine learning methods by education researchers, and growing acknowledgement of their inherent risks, there is an urgent need for tailored methodological guidance on how to improve and evaluate the validity of inferences drawn from these methods. Drawing upon an… more →
How Not to Fool Ourselves About Heterogeneity of Treatment Effects
Researchers across many fields have called for greater attention to heterogeneity of treatment effects—shifting focus from the average effect to variation in effects between different treatments, studies, or subgroups. True heterogeneity is important, but many reports of heterogeneity… more →
Making the Grade: Accounting for Course Selection in High School Transcripts with Item Response Theory
We apply Item Response Theory (IRT) to high-school transcript data, treating courses as items and grades as ordered responses, to estimate student transcript strength (θ̂) and course difficulty on a common scale. IRT estimation orders courses plausibly by difficulty, differentiates… more →
Integrating Open Science Principles into Quasi-Experimental Social Science Research
Quasi-experimental methods are a cornerstone of applied social science, providing critical answers to causal questions that inform policy and practice. Although open science principles have influenced experimental research norms across the social sciences, these practices are rarely implemented… more →
Examining the Relationship Between Randomization Strategies and Contamination in Higher Education Interventions
Randomized controlled trials are the reference method for causal inference, but field experiments in educational settings must balance statistical power with the risk of contamination. This study examines crossover and spillover contamination in a large-enrollment, in-person college course… more →
Mechanisms of Effect Size Differences Between Researcher Developed and Independently Developed Outcomes: A Meta-Analysis of Item-Level Data
Differences in effect sizes between researcher developed (RD) and independently developed (ID) outcome measures are widely documented but poorly understood in education research. We conduct a meta-analysis using item-level outcome data to test potential mechanisms that explain differences in… more →
Teacher-colleague race congruence and mobility: Do colleague demographics impact teacher retention?
Teacher turnover is especially pronounced among teachers of color who play critically important roles in the success of students of color. A growing literature points to racial isolation as one factor that is associated with Black teacher job satisfaction in particular, which in turn could play… more →
Leveraging Modern Machine Learning to Improve Early Warning Systems and Reduce Chronic Absenteeism in Early Childhood
This study focuses on improving the predictive power of early warning systems (EWSs) to decrease chronic absenteeism in early childhood. Using a demographically diverse sample of students followed from PreK to third grade in Boston Public Schools (N=6,698), we demonstrate how and why two modern… more →
Using Meta-Analytic Data to Examine Fadeout and Persistence of Intervention Impacts on Constrained and Unconstrained Skills
Recent reviews of the educational intervention literature have noted patterns of intervention impact fadeout on cognitive skills, whereby skill trajectories between children in the intervention and control group converge in the years following the end of the intervention. Some early childhood… more →
The Role of Comprehensive Student Support Interventions during School Turnaround
The persistence of underperformance in schools within large urban districts remains a significant challenge in the U.S. K-12 education system. Education policymakers have enacted legislation aiming at improving these schools through ``turnaround'' initiatives. However, students attending… more →
Same Idea, Shifting Standards: An Experimental Study of Racial-Ethnic Biases in Ambitious Math Teaching
Teacher expectations and judgments about student capabilities are predictive of student achievement, yet such judgments may be influenced by salient dimensions of student identity and invite biases. Moreover, ambitious math teaching may also invite teacher biases due to the emphasis on student-… more →
Does Charter School Autonomy Improve Matching of Teacher Attributes with Student Needs?
We examine the efficiency of traditional school districts versus charter schools in providing students with teachers who meet their demographic and education needs. Using panel data from the state of Michigan, we estimate the relationship between enrollment of Black, Hispanic, special education… more →
Using Gaussian Process Regression in Two-Dimensional Regression Discontinuity Designs
Sometimes a treatment, such as receiving a high school diploma, is assigned to students if their scores on two inputs (e.g., math and English test scores) are above established cutoffs. This forms a multidimensional regression discontinuity design (RDD) to analyze the effect of the educational… more →
Classifying Courses at Scale: a Text as Data Approach to Characterizing Student Course-Taking Trends with Administrative Transcripts
Students’ postsecondary course-taking is of interest to researchers, yet has been difficult to study at large scale because administrative transcript data are rarely standardized across institutions or state systems. This paper uses machine learning and natural language processing to standardize… more →