Search and Filter

Submit a paper

Not yet affiliated? Have a paper you wish to post? Check out the EdWorkingPapers' scope and FAQs, and then submit your manuscript here.

Measuring Conversational Uptake: A Case Study on Student-Teacher Interactions

In conversation, uptake happens when a speaker builds on the contribution of their interlocutor by, for example, acknowledging, repeating or reformulating what they have said. In education, teachers' uptake of student contributions has been linked to higher student achievement. Yet measuring and improving teachers' uptake at scale is challenging, as existing methods require expensive annotation by experts. We propose a framework for computationally measuring uptake, by (1) releasing a dataset of student-teacher exchanges extracted from US math classroom transcripts annotated for uptake by experts; (2) formalizing uptake as pointwise Jensen-Shannon Divergence (pJSD), estimated via next utterance classification; (3) conducting a linguistically-motivated comparison of different unsupervised measures and (4) correlating these measures with educational outcomes. We find that although repetition captures a significant part of uptake, pJSD outperforms repetition-based baselines, as it is capable of identifying a wider range of uptake phenomena like question answering and reformulation. We apply our uptake measure to three different educational datasets with outcome indicators. Unlike baseline measures, pJSD correlates significantly with instruction quality in all three, providing evidence for its generalizability and for its potential to serve as an automated professional development tool for teachers.

Education level
Topics
Document Object Identifier (DOI)
10.26300/4ycn-m631
EdWorkingPaper suggested citation:
Demszky, Dorottya, Jing Liu, Zid Mancenido, Julie Cohen, Heather C. Hill, Dan Jurafsky, and Tatsunori Hashimoto. (). Measuring Conversational Uptake: A Case Study on Student-Teacher Interactions. (EdWorkingPaper: -424). Retrieved from Annenberg Institute at Brown University: https://doi.org/10.26300/4ycn-m631

Machine-readable bibliographic record: RIS, BibTeX