- Hunter Gehlbach
Search for EdWorkingPapers here by author, title, or keywords.
Success in postsecondary education requires students to engage with their institution both academically and administratively. As with the transition to college, administrative requirements students face once enrolled can be substantial. Missteps with required processes can threaten students’ ability to persist. During the 2018-19 academic year, Georgia State University implemented an artificially intelligent text-based chatbot to provide proactive outreach and support to help undergraduates navigate administrative processes and take advantage of campus resources. A team of centralized university administrators orchestrated outreach “campaigns” to support students across three broad domains: (1) academic supports; (2) social and career supports; and (3) administrative processes. We investigate GSU’s implementation of this persistence-focused chatbot through an experimental study. Of the three message domains, outreach was most effective when focused on administrative processes, many of which were time-sensitive and for which outreach could be targeted specifically to students for whom it was relevant based on administrative data. In contrast, outreach to encourage take up of other supports had little effect on student behavior. By the end of the academic year, rates of FAFSA filing and registration for the subsequent fall semester were approximately three percentage points higher, suggesting positive effects on year-to-year college persistence. The positive effects on fall enrollment persisted into summer 2019, at which time the GSU administration judged that the study results were compelling enough to conclude the experiment and roll the chatbot system out to all students. We situate our findings in the literature on nudge-type efforts to support college access and success to draw lessons regarding their effective use.
We examine through a field experiment whether outreach and support provided through an AI-enabled chatbot can reduce summer melt and improve first-year college enrollment at a four-year university and at a community college. At the four-year college, the chatbot increased overall success with navigating financial aid processes, such that student take up of educational loans increased by four percentage points. This financial aid effect was concentrated among would-be first-generation college goers, for whom loan acceptances increased by eight percentage points. In addition, the outreach increased first-generation students’ success with course registration and fall semester enrollment each by three percentage points. For the community college, where the randomized experiment could not be robustly implemented due to limited cell phone number information, we present a qualitative analysis of organizational readiness for chatbot implementation. Together, our findings suggest that proactive outreach to students is likely to be most successful when targeted to those who may be struggling (for example, in keeping up with required administrative tasks). Yet, such targeting requires university systems to have ready access to and ability to make use of their administrative data.