Let’s Chat: Leveraging Chatbot Outreach for Improved Course Performance

This study reports on the causal effects of using a non-generative artificial intelligence (AI) chatbot to provide course-specific, proactive outreach and support to students in large-enrollment undergraduate courses. Across both an American Government and Microeconomics course, students randomly assigned to receive chatbot messaging were four percentage points more likely to earn an A or B in the courses. Students assigned to treatment were more likely to complete homework and use supplemental instruction opportunities, which provide evidence that increased course engagement may be driving grade outcomes. We also find suggestive evidence the chatbot reduced the likelihood of students dropping or withdrawing from each course. Treatment effects were generally consistent across student demographics, with the exception of women in Microeconomics, who earned final grades that were seven points higher than women in the control group. The chatbot was well-received by students: 82 percent of students who completed an end-of-course survey recommended its continued use and expansion to other courses. This study provides promising evidence that integrating virtual outreach and communication to students in their college courses can enhance student engagement and learning. It also illustrates the capacity of AI for providing timely responses to students’ questions, reducing instructors’ time answering common questions and allowing them to devote more time to the students who need it most.