Search EdWorkingPapers

Search for EdWorkingPapers here by author, title, or keywords.

Standards, accountability, assessment, and curriculum

Todd Pugatch, Paul Thompson.

Can public university honors programs deliver the benefits of selective undergraduate education within otherwise nonselective institutions? We evaluate the impact of admission to the Honors College at Oregon State University, a large nonselective public university. Admission to the Honors College depends heavily on a numerical application score. Nonlinearities in admissions probabilities as a function of this score allow us to compare applicants with similar scores, but different admissions outcomes, via a fuzzy regression kink design. The first stage is strong, with takeup of Honors College programming closely following nonlinearities in admissions probabilities. To estimate the causal effect of Honors College admission on human capital formation, we use these nonlinearities in the admissions function as instruments, combined with course-section fixed effects to account for strategic course selection. Honors College admission increases course grades by 0.10 grade points on the 0-4 scale, or 0.14 standard deviations. Effects are concentrated at the top of the course grade distribution. Previous exposure to Honors sections of courses in the same subject is a leading potential channel for increased grades. However, course grades of first-generation students decrease in response to Honors admission, driven by low performance in natural science courses. Results suggest that selective Honors programs can accelerate skill acquisition for high-achieving students at public universities, but not all students benefit from Honors admission.

More →

Edward J. Kim.

This study introduces the signal weighted teacher value-added model (SW VAM), a value-added model that weights student-level observations based on each student’s capacity to signal their assigned teacher’s quality. Specifically, the model leverages the repeated appearance of a given student to estimate student reliability and sensitivity parameters, whereas traditional VAMs represent a special case where all students exhibit identical parameters. Simulation study results indicate that SW VAMs outperform traditional VAMs at recovering true teacher quality when the assumption of student parameter invariance is met but have mixed performance under alternative assumptions of the true data generating process depending on data availability and the choice of priors. Evidence using an empirical data set suggests that SW VAM and traditional VAM results may disagree meaningfully in practice. These findings suggest that SW VAMs have promising potential to recover true teacher value-added in practical applications and, as a version of value-added models that attends to student differences, can be used to test the validity of traditional VAM assumptions in empirical contexts.

More →

Ana P. Cañedo, Paul T. von Hippel.

Von Hippel & Cañedo (2021) reported that US kindergarten teachers placed girls, Asian-Americans, and children from families of high socioeconomic status (SES) into higher ability groups than their test scores alone would warrant. The results fit the view that teachers were biased.

This comment asks whether parents’ lobbying for higher placement might explain these results. The answer, for the most part, is no. Measures of parent-teacher contact explained little variation in children’s ability group placement, and did not account for the higher placement of girls, Asian-Americans, or high-SES children. In fact, Asian-American parents had less teacher contact than did white children. It appears that the biases observed by von Hippel & Cañedo resided primarily in teachers, not in parents.

We also ask whether teachers who used more objective assessment techniques were less biased in placing children into higher and lower ability groups. The answer, again, was no. Unfortunately, biases persisted in the face of objective information about students’ skill. Fortunately, the biases were not terribly large.

More →

Heather McCambly, Quinn Mulroy.

Public discussions of racial inclusion and equal opportunity initiatives in the U.S. are often met with claims that expanding access to an institution, space, or public good is likely to diminish its quality. Examples of this pattern include: anticipated (and real) property value declines when predominantly white neighborhoods become more racially diverse; fears that the excellence of white schools will decline when the population of Black and brown students grows; apprehensions that equitable hiring practices necessarily entail lower standards for job candidates. In this paper, we examine how a federal agency, the Fund for the Improvement of Postsecondary Education (FIPSE), charged with addressing the aftermath of the ‘access wave’ of new college students promulgated by the Higher Education Act of 1965, came to reconcile its commitments to educational equity and quality. Through a novel examination of the historical development of what we term (e)quality politics in the administration of civil rights policy in higher education, we trace how two concepts - equity and quality – became discursively linked and contested in American politics. (E)quality politics refers to the introduction of a policy paradigm that reframes equity discussions and goals around the professed need to preserve and advance institutional “quality” using measures and standards that are, importantly, defined and instantiated under the era of segregation that precedes equal access policies. In particular, we uncover the discursive patterns by which the perceived threats to “quality” posed by racial diversity can prompt administrators to compensate, protect, and maintain the prerogatives of high-status institutions or groups that benefited under previous eras of exclusion. Understood as part of a backlash to egalitarian reforms, we argue, these quality measures undermine equity goals.

More →

Leiah Groom-Thomas, Monica Lee, Cate Smith Todd, Kathleen Lynch, Susanna Loeb, Scott McConnell, Lydia Carlis.

Many preschool agencies nationwide continue to experience closures and/or conversions to virtual or hybrid instruction due to the ongoing COVID-19 pandemic. Despite the importance of understanding young children’s learning and development during the COVID emergency, limited knowledge exists on adaptable practices of assessing young children during the pandemic. We detail practices used to assess learning in 336 Head Start children across four states during three different time periods in the 2020-21 school year, using adaptation of traditionally in-person assessments of early numeracy, early literacy, and executive functioning. In doing so, we distill early lessons for the field from the application of a novel, virtual assessment method with the early childhood population. The paper describes adaptations of assessment administration for virtual implementation and incorporation of feedback into continued virtual delivery of assessments. Applications and limitations in broader contexts are discussed.

More →

Jane Arnold Lincove, Catherine Mata, Kalena E. Cortes.

High school exit exams are meant to standardize the quality of public high schools and to ensure that students graduate with a set of basic skills and knowledge. Evidence suggests that a common perverse effect of exit exams is an increase in dropout for students who have difficulty passing tests, with a larger effect on minority students. To mitigate this, some states offer alternative, non-tested pathways to graduation for students who have failed their exit exams. This study investigates the post-secondary effects of an alternative high school graduation program. Among students who initially fail an exit exam, those who eventually graduate through an alternative project-based pathway have lower college enrollment, but similar employment outcomes to students who graduate by retaking and passing their exit exams. Compared to similar students who fail to complete high school, those students who take the alternative pathway have better post-secondary outcomes in both education and employment.

More →

Ishtiaque Fazlul, Cory Koedel, Eric Parsons.

We develop a new framework for identifying at-risk students in public schools. Our framework has two fundamental advantages over status quo systems: (1) it is based on a clear definition of what it means for a student to be at risk and (2) it leverages states’ rich administrative data systems to produce more informative risk measures. Our framework is more effective than common alternatives at identifying students who are at risk of low academic performance and we use policy simulations to show that it can be used to target resources toward these students more efficiently. It also offers several other benefits relative to status quo systems. We provide an alternative approach to risk measurement that states can use to inform funding, accountability, and other policies, rather than continuing to rely on broad categories tied to the nebulous concept of “disadvantage.”

More →

Kathryn Gonzalez, Kathleen Lynch, Heather C. Hill.

Despite growing evidence that classroom interventions in science, technology, engineering, and mathematics (STEM) can increase student achievement, there is little evidence regarding how these interventions affect teachers themselves and whether these changes predict student learning. We present results from a meta-analysis of 37 experimental studies of preK-12 STEM professional learning and curricular interventions, seeking to understand how STEM classroom interventions affect teacher knowledge and classroom instruction, and how these impacts relate to intervention impacts on student achievement. Compared with control group teachers, teachers who participated in STEM classroom interventions experienced improvements in content and pedagogical content knowledge and classroom instruction, with a pooled average impact estimate of +0.56 standard deviations. Programs with larger impacts on teacher practice yielded larger effects on student achievement, on average. Findings highlight the positive effects of STEM instructional interventions on teachers, and shed light on potential teacher-level mechanisms via which these programs influence student learning.

More →

Aaron Phipps.

Using administrative data from D.C. Public Schools, I use exogenous variation in the presence and intensity of teacher monitoring to show it significantly improves student test scores and reduces suspensions. Uniquely, my setting allows me to separately identify the effect of pre-evaluation monitoring from post-evaluation feedback. Monitoring's effect is strongest among teachers with a large incentive to increase student test scores. As tests approach, unmonitored teachers sacrifice higher-level learning, classroom management, and student engagement, even though these pedagogical tasks are among the most effective. One possible explanation is teachers ``teach to the test'' as a risk mitigation strategy, even if it is less effective on average. This is supported by showing teaching to the test has a smaller effect on student test score variance than other teaching approaches. These results illustrate the importance of monitoring in contexts where teachers have the strongest incentive to deviate from pedagogically sound practices.

More →

Meghan Comstock, Kenneth A. Shores, Camila Polanco, Erica Litke, Kirsten Lee Hill, Laura M. Desimone.

As states and districts expand their goals for equitable mathematics instruction to focus on cultural responsiveness and rigor, it is critical to understand how teachers integrate multiple teaching approaches. Drawing on survey data from a larger study of professional learning, we use mixture modeling to identify seven unique ways that middle school mathematics teachers integrate ambitious, traditional, and culturally responsive (CR) mathematics instruction. The resulting typology is driven almost exclusively by variation in CR teaching. About half of teachers reported rarely engaging in CR teaching. Teachers who emphasized CR teaching tended to be teachers of color and have high CR teaching self-efficacy. Findings suggest that tailoring teacher development to how teachers blend multiple approaches may best support equitable mathematics instruction.

More →