Search and Filter
Assessment
Does One Plus One Always Equal Two? Examining Complementarities in Educational Interventions
Topics: MethodsTags: Assessment, EfficacyPublic policies targeting individuals based on need often impose disproportionate burden on communities that lack the resources to implement these policies effectively. In an elementary school setting, I examine whether community-level interventions focusing on similar needs and providing… more →
GED® College Readiness Benchmarks and Post-Secondary Success
Tags: College readiness, AssessmentIn 2016, the GED® introduced college readiness benchmarks designed to identify testers who are academically prepared for credit-bearing college coursework. The benchmarks are promoted as awarding college credits or exempting “college-ready” GED® graduates from remedial coursework. I show… more →
Are Students On Track?: Comparing the Predictive Validity of Administrative and Survey Measures of Cognitive and Noncognitive Skills for Long-Term Outcomes
Education leaders need valid metrics to predict students’ long-term success. We use a unique dataset with cognitive skills, self-regulation, behavior, course performance, and test scores for 8th-grade students from a Northeast school district. We link these data to students' high school outcomes… more →
HBCU Enrollment and Longer-Term Outcomes
Using data from nearly 1.2 million Black SAT takers, we estimate the impacts of initially enrolling in an Historically Black College and University (HBCU) on educational, economic, and financial outcomes. We control for the college application portfolio and compare students with similar… more →
Disentangling Person-Dependent and Item-Dependent Causal Effects: Applications of Item Response Theory to the Estimation of Treatment Effect Heterogeneity
Topics: MethodsAnalyzing heterogeneous treatment effects (HTE) plays a crucial role in understanding the impacts of educational interventions.
Practice-Based Teacher Education Pedagogies Improve Responsiveness: Evidence from a Lab Experiment
Zid Mancenido, Heather C. Hill, Jeannette Garcia Coppersmith, Hannah Carter, Cynthia Pollard, Chris Monschauer.Topics: Teacher and Leader DevelopmentGiven the limited time available during teacher preparation, teacher educators must make zerosum choices about the pedagogies they choose to prepare pre-service teachers. Yet the field lacks rigorous causal evidence regarding the relative efficacy of different pedagogies to inform teacher… more →
Leveraging Item Parameter Drift to Assess Transfer Effects in Vocabulary Learning
Topics: MethodsLongitudinal models of individual growth typically emphasize between-person predictors of change but ignore how growth may vary within persons because each person contributes only one point at each time to the model. In contrast, modeling growth with multi-item assessments allows evaluation of… more →
Estimating Learning When Test Scores Are Missing: The Problem and Two Solutions
Topics: MethodsTags: Assessment, Learning environmentsLongitudinal studies can produce biased estimates of learning if children miss tests. In an application to summer learning, we illustrate how missing test scores can create an illusion of large summer learning gaps when true gaps are close to zero. We demonstrate two methods that reduce bias by… more →
Different methods for assessing pre-service teachers’ instruction: Why measures matter
Topics: Teacher and Leader DevelopmentTeacher preparation programs are increasingly expected to use data on pre-service teacher (PST) skills to drive program improvement and provide targeted supports. Observational ratings are especially vital, but also prone to measurement issues. Scores may be influenced by factors unrelated to… more →
Strategic Disclosure of Test Scores: Evidence from US College Admissions
The impact of test-optional college admissions policies depends on whether applicants act strategically in disclosing test scores. We analyze individual applicants’ standardized test scores and disclosure behavior to 50 major US colleges for entry in fall 2021, when Covid-19 prompted widespread… more →
The Unintended Consequences of Academic Leniency
In response to widening achievement gaps and increased demand for post-secondary education, local and federal governments across the US have enacted policies that have boosted high school graduation rates without an equivalent rise in student achievement, suggesting a decline in academic… more →
Multiply by 37 (or Divide by 0.023): A Surprisingly Accurate Rule of Thumb for Converting Effect Sizes from Standard Deviations to Percentile Points
Topics: MethodsTags: AssessmentEducational researchers often report effect sizes in standard deviation units (SD), but SD effects are hard to interpret. Effects are easier to interpret in percentile points, but converting SDs to percentile points involves a calculation that is not transparent to educational stakeholders. We… more →
How Much Teacher Is in Teacher Rating Scales?
Tags: Gifted education, AssessmentTeacher rating scales (TRS) are often used to make service eligibility decisions for exceptional learners. Although TRS are regularly used to identify student exceptionalism either as part of an informal nomination process or through behavioral rating scales, there is little research documenting… more →
A Global Regression Discontinuity Design: Theory and Application to Grade Retention Policies
Topics: MethodsWe use a marginal treatment effect (MTE) representation of a fuzzy regression discontinuity setting to propose a novel estimation approach. The estimator can be thought of as extrapolating a traditional fuzzy regression discontinuity estimate or as an observational study that adjusts for… more →
Identification of Non-Additive Fixed Effects Models: Is the Return to Teacher Quality Homogeneous?
Topics: MethodsPanel or grouped data are often used to allow for unobserved individual heterogeneity in econometric models via fixed effects. In this paper, we discuss identification of a panel data model in which the unobserved heterogeneity both enters additively and interacts with treatment variables. We… more →
Within-School Heterogeneity in Quality: Do Schools Provide Equal Value Added to All Students?
Topics: Families and CommunitiesLow-socioeconomic status (SES), minority, and male students perform worse than their high-SES, non-minority, and female peers on standardized tests. This paper investigates how within-school differences in school quality contribute to these educational achievement gaps. Using individual-level… more →
How Measurement Affects Causal Inference: Attenuation Bias is (Usually) More Important Than Scoring Weights
Topics: MethodsTags: AssessmentWhen analyzing treatment effects on test scores, researchers face many choices and competing guidance for scoring tests and modeling results. This study examines the impact of scoring choices through simulation and an empirical application. Results show that estimates from multiple methods applied… more →
Heterogeneity of item-treatment interactions masks complexity and generalizability in randomized controlled trials
Ishita Ahmed, Masha Bertling, Lijin Zhang, Andrew D. Ho, Prashant Loyalka, Hao Xue, Scott Rozelle, Benjamin W. Domingue.Topics: MethodsResearchers use test outcomes to evaluate the effectiveness of education interventions across numerous randomized controlled trials (RCTs). Aggregate test data—for example, simple measures like the sum of correct responses—are compared across treatment and control groups to determine whether an… more →
Out of Sight, Out of Mind? The Gap between Students’ Test Performance and Teachers’ Estimations in India and Bangladesh
This is one of the first studies of the mismatch between students’ test scores and teachers’ estimations of those scores in low- and middle-income countries. Prior studies in high-income countries have found strong correlations between these metrics. We leverage data on actual and estimated… more →
The Impact of Armed Conflict on College Students
Given the spike of homicides in conflict zones of Colombia after the 2016 peace agreement, I study the causal effect of violence on college test scores. Using a difference-in-difference design with heterogeneous effects, I show how this increase in violence had a negative effect on college… more →