Assessment
Signal Weighted Value-Added Models
Topics: MethodsTags: Assessment, EfficacyThis study introduces the signal weighted teacher value-added model (SW VAM), a value-added model that weights student-level observations based on each student’s capacity to signal their assigned teacher’s quality. Specifically, the model leverages the repeated appearance of a given student to… more →
Bias in kindergarten ability group placement: Does parental lobbying make it worse? Do formal assessments make it better?
Von Hippel & Cañedo (2021) reported that US kindergarten teachers placed girls, Asian-Americans, and children from families of high socioeconomic status (SES) into higher ability groups than their test scores alone would warrant. The results fit the view that teachers were biased.
Navigating Remote Delivery of Assessments for Head Start Children During the COVID-19 Pandemic
Leiah Groom-Thomas, Monica Lee, Cate Smith Todd, Kathleen Lynch, Susanna Loeb, Scott McConnell, Lydia Carlis.Many preschool agencies nationwide continue to experience closures and/or conversions to virtual or hybrid instruction due to the ongoing COVID-19 pandemic. Despite the importance of understanding young children’s learning and development during the COVID emergency, limited knowledge exists on… more →
A Bridge to Graduation: Post-Secondary Effects of an Alternative Pathway for Students Who Fail High School Exit Exams
Tags: High schools, AssessmentHigh school exit exams are meant to standardize the quality of public high schools and to ensure that students graduate with a set of basic skills and knowledge. Evidence suggests that a common perverse effect of exit exams is an increase in dropout for students who have difficulty passing tests… more →
Using Predicted Academic Performance to Identify At-Risk Students in Public Schools
Topics: Student LearningTags: Assessment, EquityMeasures of student disadvantage—or risk—are critical components of equity-focused education policies. However, the risk measures used in contemporary policies have significant limitations, and despite continued advances in data infrastructure and analytic capacity, there has been little… more →
Can learning be measured by phone? Evidence from Kenya
Topics: MethodsSchool closures induced by COVID-19 placed heightened emphasis on alternative ways to measure student learning besides in-person exams. We leverage the administration of phone-based assessments (PBAs) measuring numeracy and literacy for primary school children in Kenya, along with in-person… more →
Students' Grade Satisfaction Influences Evaluations of Teaching: Evidence from Individual-level Data and an Experimental Intervention
Student surveys are widely used to evaluate university teaching and increasingly adopted at the K-12 level, although there remains considerable debate about what they measure. Much disagreement focuses on the well-documented correlation between student grades and their evaluations of instructors… more →
Bridging human and machine scoring in experimental assessments of writing: tools, tips, and lessons learned from a field trial in education
Topics: MethodsIn a randomized trial that collects text as an outcome, traditional approaches for assessing treatment impact require that each document first be manually coded for constructs of interest by human raters. An impact analysis can then be conducted to compare treatment and control groups, using the… more →
Measuring and Summarizing the Multiple Dimensions of Teacher Effectiveness
Topics: Teacher and Leader DevelopmentTags: AssessmentThere is an emerging consensus that teachers impact multiple student outcomes, but it remains unclear how to measure and summarize the multiple dimensions of teacher effectiveness into simple metrics for research or personnel decisions. We present a multidimensional empirical Bayes framework and… more →
Teacher Preparation Programs and Graduates' Growth in Instructional Effectiveness
Topics: Teacher and Leader DevelopmentMany prior studies have examined whether there are average differences in levels of teaching effectiveness among graduates from different teacher preparation programs (TPPs); other studies have investigated which features of preparation predict graduates’ average levels of teaching effectiveness… more →
College Entrance Exam-Taking Strategies in Georgia
Using administrative data from Georgia, we provide the first study of the full set of college entrance exam-taking strategies, including who takes the ACT and the SAT (or both), when they take the exams, and how many times they take each exam. We have several main findings. First, one-third of… more →
Characterizing Cross-Site Variation in Local Average Treatment Effects in Multisite Regression Discontinuity Design Contexts with an Application to Massachusetts High School Exit Exam
Topics: MethodsTags: Assessment, High schoolsIn multisite experiments, we can quantify treatment effect variation with the cross-site treatment effect variance. However, there is no standard method for estimating cross-site treatment effect variance in multisite regression discontinuity designs (RDD). This research rectifies this gap in… more →
Using Implementation Fidelity to Aid in Interpreting Program Impacts: A Brief Review
Topics: MethodsTags: Assessment, CurriculumPoor program implementation constitutes one explanation for null results in trials of educational interventions. For this reason, researchers often collect data about implementation fidelity when conducting such trials. In this article, we document whether and how researchers report and measure… more →
New Schools and New Classmates: The Disruption and Peer Group Effects of School Reassignment
Topics: Policy, Politics, and GovernancePolicy makers periodically consider using student assignment policies to improve educational outcomes by altering the socio-economic and academic skill composition of schools. We exploit the quasi-random reassignment of students across schools in the Wake County Public School System to estimate… more →
Understanding Performance in Test Taking: The Role of Question Difficulty Order
Tags: AssessmentStandardized assessments are widely used to determine access to educational resources with important consequences for later economic outcomes in life. However, many design features of the tests themselves may lead to psychological reactions influencing performance. In particular, the level of… more →
Measuring Teaching Practices at Scale: A Novel Application of Text-as-Data Methods
Topics: MethodsValid and reliable measurements of teaching quality facilitate school-level decision-making and policies pertaining to teachers. Using nearly 1,000 word-to-word transcriptions of 4th- and 5th-grade English language arts classes, we apply novel text-as-data methods to develop automated measures… more →
A Half Century of Progress in U. S. Student Achievement: Agency and Flynn Effects; Ethnic and SES Differences
Topics: Student LearningPrincipals (policymakers) disagree as to whether U. S. student performance has changed over the past half century. To inform conversations, agents administered seven million psychometrically linked tests in math (m) and reading (rd) in 160 survey waves to national probability samples of cohorts… more →
The Distribution of School Spending Impacts
Tags: Efficacy, AssessmentWe examine all known "credibly causal" studies to explore the distribution of the causal effects of public K-12 school spending on student outcomes in the United States. For each of the 31 included studies, we compute the same marginal spending effect parameter estimate. Precision-weighted… more →
Achievement Gaps in the Wake of COVID-19
Topics: Student LearningA survey targeting education researchers conducted in November, 2020 provides both short- and longer-term predictions of how much achievement gaps between low- and high-income students in U.S elementary schools will change as a result of COVID-related disruptions to schooling and family life.… more →
How Much Does Teacher Quality Vary Across Teacher Preparation Programs? Reanalyses from Six States
Topics: MethodsAt least sixteen US states have taken steps toward holding teacher preparation programs (TPPs) accountable for teacher value-added to student test scores. Yet it is unclear whether teacher quality differences between TPPs are large enough to make an accountability system worthwhile. Several… more →