Search and Filter
Assessment
The Challenge of Capturing Higher-Order Thinking Skills at Scale
Topics: MethodsTags: Assessment, College readinessHigher-order thinking skills are important for K-12 students’ long-term success. However, the lack of widely administered assessments designed to capture this construct has made it difficult to measure higher-order skills at scale. This paper examines the measurement properties of an approach to… more →
The Anatomy of a High-Return Question: Text, Skills, and the Economics of Achievement Measurement
Standardized test scores aggregate item (question) responses into a single scalar, collapsing distinct skills into an undifferentiated measure of proficiency. Which of these component skills matter most for long-run economic outcomes is a question that aggregate scores cannot answer. We develop… more →
The Reliability of Classroom Observations and Student Surveys in Non-Research Settings: Evidence from a Middle-Income Country
Topics: Teacher and Leader DevelopmentWe present one of the first Generalizability studies of non-test measures of teaching effectiveness administered by practitioners in a middle-income country. The reliability of observations varies widely (from 0 to 0.75 on a 0-1 scale) and depends upon their context (whether they are conducted… more →
Are Work-Based Professional Skills Associated with Postsecondary Entrance and Persistence? Novel Evidence from the Cristo Rey Network
Professional skills such as initiative, communication, and adaptability are thought to shape postsecondary success, but most evidence comes from self- or teacher-reported measures collected in school settings. This study uses employer ratings of students’ professional skills gathered through… more →
Testing frequency and student achievement: A systematic review
Jens Dietrichson, Julie Kaas Seerup, Anja Bondebjerg Mølgaard, Malene Wallach Kildemoes, Frederikke Lykke Witthöft Schytt, Mikkel Vembye, Elizabeth Bengtsen, Bjørn Christian Ar-leth Viinholt, Morten Kjær Thomsen.Topics: Student LearningTags: AssessmentSchool-based testing is widely used for monitoring students’ academic progress. Proponents argue that testing ensures accountability and guides teachers and managers, whereas opponents point to adverse consequences such as teaching to the test, and frequent testing creating anxiety and stress.… more →
Leveraging Large Language Models to Assess Short Text Responses
Topics: MethodsEducational practitioners and researchers often score short, unstructured text for the presence or strength of domain-specific constructs. Manual scoring, however, faces limitations, including time- and labor-intensiveness. Large language models (LLMs) offer an automated alternative to manual… more →
ChatGPT vs. Machine Learning: Assessing the Efficacy and Accuracy of Large Language Models for Automated Essay Scoring
Topics: MethodsAutomated Essay Scoring (AES) is a critical tool in education that aims to enhance the efficiency and objectivity of educational assessments. Recent advancements in Large Language Models (LLMs), such as ChatGPT, have sparked interest in their potential for AES. However, comprehensive comparisons… more →
The reliability of classroom observations and student surveys in non-research settings: Evidence from Argentina
Topics: Teacher and Leader DevelopmentThere is a growing consensus on the need to measure teaching effectiveness using multiple instruments. Yet, guidance on how to achieve reliable ratings derives largely from formal research in high-income countries. We study the reliability of classroom observations and student surveys conducted… more →
What is the impact of changing schools on the academic outcomes of elementary and middle school students?
Topics: Student LearningWe study how different kinds of school changes shape achievement in grades 4–8 using data from six California districts (2016–17 through 2019–20). We estimate the effects of structural (promotional), nonstructural summer, and midyear moves and find that structural and summer moves have near-zero… more →
Beg to DIFfer: Resolving Statistical Complications of Intersectional DIF Analyses
Topics: MethodsTags: Assessment, EquityModern test developers conduct differential item functioning (DIF) analyses to ensure fairness in educational and psychological testing. To address previously unrecognized biases, researchers have recently demonstrated the importance of conducting intersectional DIF analyses that attend to the… more →
Controlling For Measurement Error in Evaluation Models When Treatment Group Assignment is Based on Noisy Measures: Evaluation of an Achievement Gap-Closing Initiative
Topics: MethodsTags: Assessment, EquityThis paper develops new models to evaluate the effects of interventions and intervention-by-site heterogeneity when treatment group assignment is based on a fallible variable and the outcome of interest is determined in part by the corresponding true control variables (measured without error).… more →
Labor supply, learning time, and the efficiency of school spending: Evidence from school finance reforms
Topics: Policy, Politics, and GovernanceDoes school spending raise achievement? I show that effects, benchmarked by schools’ daily value added, are one-tenth to one-third as large as spending growth. Using school finance reforms for identification, I show that schools did not raise quality measured by value added. Instead, schools… more →
Parent Perspectives on School Choice: Experimental Evidence from a Nationally Representative Sample
Topics: School ChoiceParental attitudes and perspectives of student “success” will likely drive their educational choices, whether residentially assigned district public schools, alternative public schools, private schools, or homeschooling. However, little research has examined the importance of these attitudes on… more →
High School Equivalency Credentialing and Post-Secondary Success: Pre-Registered Quasi-Experimental Evidence from the GED® Test
For the over 24 million American adults who do not hold a traditional high school diploma, high school equivalency (HSE) credentials represent the primary “second-chance” pathway to many careers or educational opportunities. This project uses current, representative data to assess whether, how,… more →
The Sensitivity of Value-Added Estimates to Test Scoring Decisions
Topics: MethodsTags: AssessmentValue-Added Models (VAMs) are both common and controversial in education policy and accountability research. While the sensitivity of VAMs to model specification and covariate selection is well documented, the extent to which test scoring methods (e.g., mean scores vs. IRT-based scores) may… more →
Testing Away from One's Own School: Exam Location and Performance in High-Stakes Exams
High-stakes exams are often administered at designated test centers, requiring many students to test in unfamiliar environments. We investigate whether such arrangements impact students' test performance and, by extension, access to educational opportunities.
Bridging Literacy Gaps: The Impact of AI-Driven Personalised Learning on Reading Skills and Educational Equity
Topics: Student LearningTags: Assessment, EquityPersistent literacy skills deficits hinder educational attainment, limit labour market opportunities, and exacerbate socioeconomic inequalities. This paper evaluates the causal effect of an AI-driven Computer-Assisted Learning (CAL) program implemented by the Government of Madrid, which features… more →
Puzzling Over Declining Academic Achievement
Topics: Student LearningTags: Assessment, School reformMany are concerned about the large decline in K-12 student achievement since 2019. And rightly so, given what it signals about student learning and later life outcomes. Less noted is the pre-pandemic sustained decline in student achievement growth that followed 30 years of increases. We examine… more →
Item-Level Heterogeneity in Value Added Models: Implications for Reliability, Cross-Study Comparability, and Effect Sizes
Topics: MethodsTags: Assessment, EfficacyValue added models (VAMs) attempt to estimate the causal effects of teachers and schools on student test scores. We apply Generalizability Theory to show how estimated VA effects depend upon the selection of test items. Standard VAMs estimate causal effects on the items that are included on the… more →
Disparate Teacher Effects, Comparative Advantage, and Match Quality
Topics: MethodsDoes student-teacher match quality exist? While prior research documents disparities in teachers' impacts across student types, it has not distinguished between sorting and causal effects as the drivers of these disparities. I develop a flexible disparate value-added model (DVA) and introduce a… more →