College Mathematics Beliefs and Belonging Survey
Category: Student Well-Being
Randomized controlled trials are the reference method for causal inference, but field experiments in educational settings must balance statistical power with the risk of contamination. This study examines crossover and spillover contamination in a large-enrollment, in-person college course implementing an AI-enabled chatbot intervention. We compare two randomization approaches, individual-level and laboratory-level, to assess contamination risks. Contrary to expectations, no crossover occurred under student-level randomization. However, survey data suggest potential spillover, with treatment-group students reporting that they shared chatbot messages with peers. Using estimated contamination levels, we assess changes in minimum detectable effect size (MDES) and show that individual-level randomization remains preferable. Our findings offer practical guidance for balancing contamination risk and statistical power when designing RCTs in interactive educational settings.
Keywords
randomization, causal inference, behavioral economics, nudge, higher education
Education level
Topics
Document Object Identifier (DOI)
10.26300/rq74-c249
