Methods
Hmong but not Asian, Sāmoan but not Pacific Islander: Tracing the ECLS-K Racial Data (Mis)Classification Journey
Race is a socially and politically charged concept that remains contested in the United States. We examine racial data (mis)classification in the Early Childhood Longitudinal Studies (ECLS-K) dataset. Centering the racial data journey of Asian American, Native Hawaiian, and Pacific Islander (AA… more →
The Challenge of Capturing Higher-Order Thinking Skills at Scale
Higher-order thinking skills are important for K-12 students’ long-term success. However, the lack of widely administered assessments designed to capture this construct has made it difficult to measure higher-order skills at scale. This paper examines the measurement properties of an approach to… more →
Returns to Education in the United States: A Comparison of OLS and Double Machine Learning Methods
This study examines the economic returns to education in the U.S. using 2024 CPS data and compares Ordinary Least Squares (OLS) regression with a Double Machine Learning (DML) framework incorporating models such as random forests, boosted trees, lasso, GAMs, and neural networks (MLP). Results… more →
Digital Incentives in Surveys: Response Rates and Sociodemographic Effects in a Large-Scale Parental Nudge Intervention
This study examines how digital incentives influence survey participation and engagement in a large randomized controlled trial of parents across six school districts. We test how incentive amount and information about vendor options affect response behavior and explore differences by language… more →
From Statistical to Analytic Generalization: New Directions for Qualitative Research on Teacher Retention
Quantitative research has played a prominent role in studies and policies focused on teacher retention. However, the field would benefit from qualitative research that utilizes analytic generalization, an approach where researchers generalize from empirical data by creating theoretical… more →
IDEA-Aligned Estimates of Racial Disproportionality in Special Education versus Conventional Approaches: A cautionary note on included-variable bias when achievement and socioeconomic status proxy for special education need
Racial disproportionality in special education is a contested policy space. Federal oversight has traditionally focused on minority over-representation through IDEA’s significant disproportionality framework. However, observational studies report that Black students appear under-identified based… more →
Capturing Voter Turnout at the School District Level: Validating a Geospatial Strategy
School boards are critical sites of education policymaking, yet scholarship on these institutions is scarce because of severe data limitations. We introduce a geospatial strategy and open-source R package, called “Query, Overlay, Recover” (QOR), that generates high-quality estimates of voter… more →
Why Fadeout is (Probably) Worse Than We Think: Adjusting for Correlated Sampling Error in Meta-Analyses of Behavioral Interventions
The extent to which intervention effects persist or fade over time is an important question in the behavioral sciences. In meta-analysis, persistence is typically assessed by meta-regressing effect sizes at followup on effect sizes at endline. While common, the standard meta-regression does not… more →
Leveraging Large Language Models to Assess Short Text Responses
Educational practitioners and researchers often score short, unstructured text for the presence or strength of domain-specific constructs. Manual scoring, however, faces limitations, including time- and labor-intensiveness. Large language models (LLMs) offer an automated alternative to manual… more →
The Chronic(les) of Absenteeism Measurement: Unpacking the Many Measures of Attendance and Evidence for a Lower Chronic Absenteeism Threshold
Chronic absenteeism has surged in recent years, drawing growing policy and research attention. However, a complicating factor often overlooked is that the measurement of absenteeism is inconsistent, with substantial researcher degrees of freedom. This study investigates how researchers’… more →
Using experimental variation to examine the (co-)development of cognitive and social-emotional skills in early childhood
Questions about the stability of psychological constructs, skill generalization, and transfer have long motivated psychological research. Despite a proliferation of theory, the field has rarely established causal effects. We employed a novel approach to test the stability and codevelopment of… more →
ChatGPT vs. Machine Learning: Assessing the Efficacy and Accuracy of Large Language Models for Automated Essay Scoring
Automated Essay Scoring (AES) is a critical tool in education that aims to enhance the efficiency and objectivity of educational assessments. Recent advancements in Large Language Models (LLMs), such as ChatGPT, have sparked interest in their potential for AES. However, comprehensive comparisons… more →
A Framework for Building High-Quality Education Data for R&D in the Age of AI: The EDSI Dataset and Expert Insights
The Gates Foundation, the Walton Family Foundation, and the Chan Zuckerberg Initiative have launched a series of collaborative investments in building large-scale datasets that can support and accelerate data infrastructure for AI R&D efforts in education. In partnership with researchers… more →
Beg to DIFfer: Resolving Statistical Complications of Intersectional DIF Analyses
Modern test developers conduct differential item functioning (DIF) analyses to ensure fairness in educational and psychological testing. To address previously unrecognized biases, researchers have recently demonstrated the importance of conducting intersectional DIF analyses that attend to the… more →
Controlling For Measurement Error in Evaluation Models When Treatment Group Assignment is Based on Noisy Measures: Evaluation of an Achievement Gap-Closing Initiative
This paper develops new models to evaluate the effects of interventions and intervention-by-site heterogeneity when treatment group assignment is based on a fallible variable and the outcome of interest is determined in part by the corresponding true control variables (measured without error).… more →
Comparing Machine Learning Methods for Estimating Heterogeneous Treatment Effects in Randomized Trials: A Comprehensive Simulation Study
This study compares 18 machine learning methods for estimating heterogeneous treatment effects in randomized controlled trials, using simulations calibrated to two large-scale educational experiments. We evaluate performance across continuous and binary outcomes with diverse and realistic… more →
Measuring “Noncognitive” Skills at Scale: Building Longitudinal Student Behavior Composites Using Administrative Data
“Noncognitive” skills, especially student behavior, are critical predictors of academic and life outcomes. However, measuring student behavior at scale remains challenging, particularly for longitudinal research. This study uses a demographically diverse sample of students followed from… more →
A Critical Appraisal of the Evidence on Racial Disproportionality in Special Education
This essay provides a two-pronged critical assessment of a subset of the literature on racial disproportionality in special education: that which aims to estimate racial disparities among otherwise similar children. This body of research has shown that Black students are less likely than… more →
The Sensitivity of Value-Added Estimates to Test Scoring Decisions
Value-Added Models (VAMs) are both common and controversial in education policy and accountability research. While the sensitivity of VAMs to model specification and covariate selection is well documented, the extent to which test scoring methods (e.g., mean scores vs. IRT-based scores) may… more →
From Sentence-Corrections to Deeper Dialogue: Qualitative Insights from LLM and Teacher Feedback on Student Writing
Effective writing feedback is a powerful tool for enhancing student learning, encouraging revision, and increasing motivation and agency. Yet, teachers face many challenges that prevent them from consistently providing effective writing feedback. Recent advances in generative artificial… more →