Methods
Shine a Light: The U.S. Federal Government Should Increase Access to Educational Administrative Data
The Institute of Educational Sciences invested substantial resources into the Statewide Longitudinal Data Systems grant program. These grants helped states convert their educational records to longitudinal datasets for research and evaluation. However, external researchers still have unreliable… more →
When Youth Enter The Chat: An Epistemic Shift in the Validation of LLM-Based Measures of Student Talk
Large Language Models (LLMs) are being used increasingly to measure aspects of student discourse (e.g. talk moves, collaboration, equity of voice) at scale. Typically, LLM-based measures of student talk use transcriptions of classroom conversations that only include verbal contributions, which… more →
Unpacking Racial Disparities in School Spending: Why “progressiveness” and “comparability” are not enough
Studies of racial and economic disparities in school funding typically rely on simple spending comparisons or "progressiveness" measures that ignore variation in the cost of achieving common educational outcomes. This article argues that such approaches essentially ignore how much it costs… more →
An Applied Researchers’ Guide to Estimating Effects from Blocked Cluster Randomized Trials: Estimands, Estimators, and Estimates
Blocked cluster randomized trials (CRTs) are widely used to evaluate educational interventions. In such trials, researchers face critical choices about how to define and estimate the average treatment effect (ATE). These choices—both the estimand (e.g., person- vs. cluster-weighted) and the… more →
Conditional Hypothesis Generation for LLM-Based Text Analysis with Researcher-Specified Covariates
A core goal of computational social science is to discover interpretable differences in how language varies across outcomes of interest, such as political affiliation or instructional quality. Recent LLM-based hypothesis generation methods describe such differences in natural language, but… more →
Hmong but not Asian, Sāmoan but not Pacific Islander: Tracing the ECLS-K Racial Data (Mis)Classification Journey
Race is a socially and politically charged concept that remains contested in the United States. We examine racial data (mis)classification in the Early Childhood Longitudinal Studies (ECLS-K) dataset. Centering the racial data journey of Asian American, Native Hawaiian, and Pacific Islander (AA… more →
The Challenge of Capturing Higher-Order Thinking Skills at Scale
Higher-order thinking skills are important for K-12 students’ long-term success. However, the lack of widely administered assessments designed to capture this construct has made it difficult to measure higher-order skills at scale. This paper examines the measurement properties of an approach to… more →
Returns to Education in the United States: A Comparison of OLS and Double Machine Learning Methods
This study examines the economic returns to education in the U.S. using 2024 CPS data and compares Ordinary Least Squares (OLS) regression with a Double Machine Learning (DML) framework incorporating models such as random forests, boosted trees, lasso, GAMs, and neural networks (MLP). Results… more →
Digital Incentives in Surveys: Response Rates and Sociodemographic Effects in a Large-Scale Parental Nudge Intervention
This study examines how digital incentives influence survey participation and engagement in a large randomized controlled trial of parents across six school districts. We test how incentive amount and information about vendor options affect response behavior and explore differences by language… more →
From Statistical to Analytic Generalization: New Directions for Qualitative Research on Teacher Retention
Quantitative research has played a prominent role in studies and policies focused on teacher retention. However, the field would benefit from qualitative research that utilizes analytic generalization, an approach where researchers generalize from empirical data by creating theoretical… more →
IDEA-Aligned Estimates of Racial Disproportionality in Special Education versus Conventional Approaches: A cautionary note on included-variable bias when achievement and socioeconomic status proxy for special education need
Racial disproportionality in special education is a contested policy space. Federal oversight has traditionally focused on minority over-representation through IDEA’s significant disproportionality framework. However, observational studies report that Black students appear under-identified based… more →
Capturing Voter Turnout at the School District Level: Validating a Geospatial Strategy
School boards are critical sites of education policymaking, yet scholarship on these institutions is scarce because of severe data limitations. We introduce a geospatial strategy and open-source R package, called “Query, Overlay, Recover” (QOR), that generates high-quality estimates of voter… more →
Why Fadeout is (Probably) Worse Than We Think: Adjusting for Correlated Sampling Error in Meta-Analyses of Behavioral Interventions
The extent to which intervention effects persist or fade over time is an important question in the behavioral sciences. In meta-analysis, persistence is often assessed by meta-regressing effect sizes at followup on effect sizes at endline. While common, the standard meta-regression does not… more →
Leveraging Large Language Models to Assess Short Text Responses
Educational practitioners and researchers often score short, unstructured text for the presence or strength of domain-specific constructs. Manual scoring, however, faces limitations, including time- and labor-intensiveness. Large language models (LLMs) offer an automated alternative to manual… more →
The Chronic(les) of Absenteeism Measurement: Unpacking the Many Measures of Attendance and Evidence for a Lower Chronic Absenteeism Threshold
This study investigates how absenteeism can be more rigorously measured as an early warning indicator of future academic risk.
Using experimental variation to examine the (co-)development of cognitive and social-emotional skills in early childhood
How cognitive and social-emotional skills emerges in childhood is a central question in developmental psychology. Despite a proliferation of theory and empirical work, the field has struggled to establish causal effects. We employed a novel approach to test the stability and codevelopment of… more →
ChatGPT vs. Machine Learning: Assessing the Efficacy and Accuracy of Large Language Models for Automated Essay Scoring
Automated Essay Scoring (AES) is a critical tool in education that aims to enhance the efficiency and objectivity of educational assessments. Recent advancements in Large Language Models (LLMs), such as ChatGPT, have sparked interest in their potential for AES. However, comprehensive comparisons… more →
A Framework for Building High-Quality Education Data for R&D in the Age of AI: The EDSI Dataset and Expert Insights
The Gates Foundation, the Walton Family Foundation, and the Chan Zuckerberg Initiative have launched a series of collaborative investments in building large-scale datasets that can support and accelerate data infrastructure for AI R&D efforts in education. In partnership with researchers… more →
Beg to DIFfer: Resolving Statistical Complications of Intersectional DIF Analyses
Modern test developers conduct differential item functioning (DIF) analyses to ensure fairness in educational and psychological testing. To address previously unrecognized biases, researchers have recently demonstrated the importance of conducting intersectional DIF analyses that attend to the… more →
Controlling For Measurement Error in Evaluation Models When Treatment Group Assignment is Based on Noisy Measures: Evaluation of an Achievement Gap-Closing Initiative
This paper develops new models to evaluate the effects of interventions and intervention-by-site heterogeneity when treatment group assignment is based on a fallible variable and the outcome of interest is determined in part by the corresponding true control variables (measured without error).… more →