Search and Filter
Assessment
HBCU Enrollment and Longer-Term Outcomes
Using data from nearly 1.2 million Black SAT takers, we estimate the impacts of initially enrolling in an Historically Black College and University (HBCU) on educational, economic, and financial outcomes. We control for the college application portfolio and compare students with similar… more →
Disentangling Person-Dependent and Item-Dependent Causal Effects: Applications of Item Response Theory to the Estimation of Treatment Effect Heterogeneity
Topics: MethodsAnalyzing heterogeneous treatment effects (HTE) plays a crucial role in understanding the impacts of educational interventions.
Practice-Based Teacher Education Pedagogies Improve Responsiveness: Evidence from a Lab Experiment
Zid Mancenido, Heather C. Hill, Jeannette Garcia Coppersmith, Hannah Carter, Cynthia Pollard, Chris Monschauer.Topics: Teacher and Leader DevelopmentGiven the limited time available during teacher preparation, teacher educators must make zerosum choices about the pedagogies they choose to prepare pre-service teachers. Yet the field lacks rigorous causal evidence regarding the relative efficacy of different pedagogies to inform teacher… more →
Leveraging Item Parameter Drift to Assess Transfer Effects in Vocabulary Learning
Topics: MethodsLongitudinal models of individual growth typically emphasize between-person predictors of change but ignore how growth may vary within persons because each person contributes only one point at each time to the model. In contrast, modeling growth with multi-item assessments allows evaluation of… more →
Estimating Learning When Test Scores Are Missing: The Problem and Two Solutions
Topics: MethodsTags: Assessment, Learning environmentsLongitudinal studies can produce biased estimates of learning if children miss tests. In an application to summer learning, we illustrate how missing test scores can create an illusion of large summer learning gaps when true gaps are close to zero. We demonstrate two methods that reduce bias by… more →
Different methods for assessing pre-service teachers’ instruction: Why measures matter
Topics: Teacher and Leader DevelopmentTeacher preparation programs are increasingly expected to use data on pre-service teacher (PST) skills to drive program improvement and provide targeted supports. Observational ratings are especially vital, but also prone to measurement issues. Scores may be influenced by factors unrelated to… more →
Strategic Disclosure of Test Scores: Evidence from US College Admissions
The impact of test-optional college admissions policies depends on whether applicants act strategically in disclosing test scores. We analyze individual applicants’ standardized test scores and disclosure behavior to 50 major US colleges for entry in fall 2021, when Covid-19 prompted widespread… more →
The Unintended Consequences of Academic Leniency
In response to widening achievement gaps and increased demand for post-secondary education, local and federal governments across the US have enacted policies that have boosted high school graduation rates without an equivalent rise in student achievement, suggesting a decline in academic… more →
Multiply by 37 (or Divide by 0.023): A Surprisingly Accurate Rule of Thumb for Converting Effect Sizes from Standard Deviations to Percentile Points
Topics: MethodsTags: AssessmentEducational researchers often report effect sizes in standard deviation units (SD), but SD effects are hard to interpret. Effects are easier to interpret in percentile points, but converting SDs to percentile points involves a calculation that is not transparent to educational stakeholders. We… more →
How Much Teacher Is in Teacher Rating Scales?
Tags: Gifted education, AssessmentTeacher rating scales (TRS) are often used to make service eligibility decisions for exceptional learners. Although TRS are regularly used to identify student exceptionalism either as part of an informal nomination process or through behavioral rating scales, there is little research documenting… more →
A Global Regression Discontinuity Design: Theory and Application to Grade Retention Policies
Topics: MethodsWe use a marginal treatment effect (MTE) representation of a fuzzy regression discontinuity setting to propose a novel estimation approach. The estimator can be thought of as extrapolating a traditional fuzzy regression discontinuity estimate or as an observational study that adjusts for… more →
Identification of Non-Additive Fixed Effects Models: Is the Return to Teacher Quality Homogeneous?
Topics: MethodsPanel or grouped data are often used to allow for unobserved individual heterogeneity in econometric models via fixed effects. In this paper, we discuss identification of a panel data model in which the unobserved heterogeneity both enters additively and interacts with treatment variables. We… more →
Within-School Heterogeneity in Quality: Do Schools Provide Equal Value Added to All Students?
Topics: Families and CommunitiesLow-socioeconomic status (SES), minority, and male students perform worse than their high-SES, non-minority, and female peers on standardized tests. This paper investigates how within-school differences in school quality contribute to these educational achievement gaps. Using individual-level… more →
How Measurement Affects Causal Inference: Attenuation Bias is (Usually) More Important Than Scoring Weights
Topics: MethodsTags: AssessmentWhen analyzing treatment effects on test scores, researchers face many choices and competing guidance for scoring tests and modeling results. This study examines the impact of scoring choices through simulation and an empirical application. Results show that estimates from multiple methods applied… more →
Heterogeneity of item-treatment interactions masks complexity and generalizability in randomized controlled trials
Ishita Ahmed, Masha Bertling, Lijin Zhang, Andrew D. Ho, Prashant Loyalka, Hao Xue, Scott Rozelle, Benjamin W. Domingue.Topics: MethodsResearchers use test outcomes to evaluate the effectiveness of education interventions across numerous randomized controlled trials (RCTs). Aggregate test data—for example, simple measures like the sum of correct responses—are compared across treatment and control groups to determine whether an… more →
Out of Sight, Out of Mind? The Gap between Students’ Test Performance and Teachers’ Estimations in India and Bangladesh
This is one of the first studies of the mismatch between students’ test scores and teachers’ estimations of those scores in low- and middle-income countries. Prior studies in high-income countries have found strong correlations between these metrics. We leverage data on actual and estimated… more →
The Impact of Armed Conflict on College Students
Given the spike of homicides in conflict zones of Colombia after the 2016 peace agreement, I study the causal effect of violence on college test scores. Using a difference-in-difference design with heterogeneous effects, I show how this increase in violence had a negative effect on college… more →
Measuring grading standards at high schools: a methodology and an example
At schools with low grading standards, students receive higher school-awarded grades across multiple courses than students with the same skills receive at schools with high grading standards. A new methodology shows grading standards vary substantially, certainly enough to affect post-secondary… more →
What We Can Learn About Latin American Educational Systems from International Tests: A Brief Foray
Topics: Student LearningThe Revista del Centro de Estudios Educativos, numero 3, 1971 included an early Carnoy article on the economics of education: “Un enfoque de sistemas para evaluar la educación, ilustrado con datos de Puerto Rico.” The article used a unique data set that had student test scores, students’ family… more →
Measuring returns to experience using supervisor ratings of observed performance: The case of classroom teachers
Topics: MethodsTags: Human capital, AssessmentWe study the returns to experience in teaching, estimated using supervisor ratings from classroom observations. We describe the assumptions required to interpret changes in observation ratings over time as the causal effect of experience on performance. We compare two difference-in-differences… more →