Evidence-Based Practices For Assessing Students’ Social And Emotional Well-Being
Category: Student Well-Being
Blocked cluster randomized trials (CRTs) are widely used to evaluate educational interventions. In such trials, researchers face critical choices about how to define and estimate the average treatment effect (ATE). These choices—both the estimand (e.g., person- vs. cluster-weighted) and the estimator (e.g., regression models, multilevel models)—can influence study conclusions. This paper provides an applied guide to estimands and estimators for blocked CRTs and empirically examines their performance using 26 large-scale blocked CRTs. We estimate ATEs and standard errors for 50 outcomes using 19 estimators and compare results across and within estimands. Findings show that estimator choice can substantially affect point estimates and precision: ranges in ATE estimates exceed 0.10 SD in 18% of cases, and standard error estimates differ by more than 50% in 16% of cases. Some estimators fail or produce unstable results in common real-world designs. These findings underscore the importance of pre-specifying estimation strategies in a public pre-analysis plan. We provide practical guidance for researchers planning and analyzing blocked CRTs.
Keywords
blocked, cluster-randomized experiments; randomized trials; causal inference; program evaluation
Education level
Topics
Document Object Identifier (DOI)
10.26300/tb6j-0e41
