Although program evaluations using rigorous quasi-experimental or experimental designs can inform decisions about whether to continue or terminate a given program, they often have limited ability to reveal the mechanisms by which complex interventions achieve their effects. To illuminate these mechanisms, this paper analyzes novel text data from thousands of school improvement planning and implementation reports from Washington State, deploying computer-assisted techniques to extract measures of school improvement processes. Our analysis identified 15 coherent reform strategies that varied greatly across schools and over time. The prevalence of identified reform strategies was largely consistent with school leaders’ own perceptions of reform priorities via interviews. Several reform strategies measures were significantly associated with reductions in student chronic absenteeism and improvements in student achievement. We lastly discuss the opportunities and pitfalls of using novel text data to study reform processes.
Using a Text-as-Data Approach to Understand Reform Processes: A Deep Exploration of School Improvement Strategies
Keywords
Text as data, School improvement, Reform processes
Education level
Topics
Document Object Identifier (DOI)
10.26300/8gjy-6r75
EdWorkingPaper suggested citation:
Sun, Min, Jing Liu, Junmeng Zhu, and Zachary LeClair. (). Using a Text-as-Data Approach to Understand Reform Processes: A Deep Exploration of School Improvement Strategies. (EdWorkingPaper:
-68). Retrieved from
Annenberg Institute at Brown University: https://doi.org/10.26300/8gjy-6r75Published Edworkingpaper:
Sun, M., Liu, J., Zhu, J., & LeClair, Z. (2019). Using a Text-as-Data Approach to Understand Reform Processes: A Deep Exploration of School Improvement Strategies. Educational Evaluation and Policy Analysis. https://doi.org/10.3102/0162373719869318Related Resources from the EdExchange
A library of instruments for education research and school improvement
A new model for informed decision-making in education