Standards, accountability, assessment, and curriculum

Students’ postsecondary course-taking is of interest to researchers, yet has been difficult to study at large scale because administrative transcript data are rarely standardized across institutions or state systems. This paper uses machine learning and natural language processing to standardize college transcripts at scale. We demonstrate the approach’s utility by showing how the disciplinary orientation of students’ courses and majors align and diverge at 18 diverse four-year institutions in the College and Beyond II dataset. Our findings complicate narratives that student participation in the liberal arts is in great decline. Both professional and liberal arts majors enroll in a large amount of liberal arts coursework, and in three of the four core liberal arts disciplines, the share of course-taking in those fields is meaningfully higher than the share of majors in those fields. To advance the study of student postsecondary pathways, we release the classification models for public use.

More →

Despite well-designed curriculum materials, teachers often face challenges in their implementation due to diverse classroom needs. This paper investigates whether Large Language Models (LLMs) can support middle-school math teachers by helping create high-quality curriculum scaffolds, which we define as the adaptations and supplements teachers employ to ensure all students can access and engage with the curriculum. Through Cognitive Task Analysis with expert teachers, we identify a three-stage process for curriculum scaffolding: observation, strategy formulation, and implementation. We incorporate these insights into three LLM approaches to create warmup tasks that activate background knowledge. The best-performing approach, which provides the model with the original curriculum materials and an expert-informed prompt, generates warmups that are rated significantly higher than warmups created by expert teachers in terms of alignment to learning objectives, accessibility to students working below grade level, and teacher preference. This research demonstrates the potential of LLMs to support teachers in creating effective scaffolds and provides a methodology for developing AI-driven educational tools.

More →

Despite evidence that teacher professional development interventions in mathematics and science can increase student achievement, our understanding of the mechanisms by which this occurs – particularly how these interventions affect teachers themselves, and whether teacher-level changes predict student learning – remains limited. The current meta-analysis synthesizes 46 experimental studies of preK-12 mathematics and science professional development interventions to investigate how these interventions affect teachers’ knowledge and classroom instruction, and how these impacts relate to intervention effects on student achievement. Compared with controls, treatment group teachers had stronger performance on measures of knowledge and classroom instruction (pooled average impact estimate: +0.53 SD). Programs with larger impacts on teacher practice had significantly larger mean effects on student achievement. However, mean effects on student achievement were not significantly related to impacts on teacher knowledge. We discuss implications for future research and practice.

More →

This paper reports findings from a nationally representative survey of K-12 teachers in May 2023 that examines the potential long-term impacts of COVID-19 on public schooling. The findings suggest fundamental ways in which school operations, instructional practice and parent-teacher interaction have changed since the pandemic. Some changes seem promising; others suggest caution. While policymakers may not be able to directly influence some of the reported changes in the short run, monitoring the evolution of school practices (and their consequences for children) will position educational leaders to help teachers and students address the consequences of the COVID-19 pandemic going forward.

More →

The immediate impacts of COVID-19 on K12 schooling are well known. Over nearly 18 months, students’ academic performance and mental health deteriorated dramatically. This study aims to identify if and how the pandemic led to longer-term changes in core aspects of schooling. Drawing on semi-structured interviews with 31 teachers and administrators across 12 districts in two states, we find that schools today look quite different in several areas including the availability and use of instructional technology, instructional practice, parent-teacher communication, and the balance between academics and social-emotional well-being. We interpret these findings through the lens of institutional theory, and discuss implications of the changes for practitioners and policymakers.

More →

In theory, monitoring can improve employee motivation and effort, particularly in settings lacking measurable outputs, but research assessing monitoring as a motivator is limited to laboratory settings. To address this gap, I leverage exogenous variation in the presence and intensity of teacher monitoring, in the form of unannounced in-class observations as part of D.C. Public Schools’ IMPACT program. As monitoring intensifies, teachers use more individualized teaching and emphasize higher-level learning. When teachers are unmonitored, their students have lower test scores and increased suspensions. This novel evidence validates monitoring as a potential tool for enhancing teacher pedagogy and employee performance more broadly.

More →

Limited scholarship examines districtwide turnaround reforms beyond the first few years of implementation or efforts to replicate successes in new contexts. We study Massachusetts, home to a state takeover of the Lawrence school district that led to academic gains in early reform years, and where state leaders attempted to replicate this success in three additional communities. We use statewide student-level administrative data (2006-07 to 2018- 19) and event study methods to estimate medium-term impacts on student outcomes across four districts. We find the initial improvements were largely sustained in Lawrence. We observe evidence of successful replication in Springfield but not Holyoke or Southbridge. The two turnarounds with positive outcomes both struck a unique balance between state and local input into decision-making.

More →

Online credit recovery (OCR) courses are the most common means through which students retake courses required for high school graduation. Yet a growing body of research has raised concerns regarding student learning in these courses, with low quality assessments posited as one contributing factor. To address this concern, we reviewed every assessment item from a widely used OCR Algebra 1 course. We also examined pathways for passing the course mastery tests without learning content. In addition, we identified if and how states regulate OCR. We found OCR assessments as executed lacked rigor and validity. We offer recommendations to improve rigor, close pathways that call into question the validity of results, strengthen implementation procedures, and increase state-level oversight of providers.

More →

Hybrid school enrollments are trending up and many parents express a diverse range of reasons for enrolling their children in hybrid schools. Yet little is known about the pedagogical goals pursued by hybrid schools. We aim to help close this gap in the literature with a stated preferences experiment of hybrid school leaders’ perceptions of program success. Sixty-three school leaders participated in a survey experiment in which we randomly assigned attributes to hypothetical programs and asked school leaders to identify the most successful program. We find that hybrid school leaders consider a broad range of student outcomes when evaluating program success, including labor market outcomes, civic outcomes, and family life. Students’ religious observance produced the largest effect sizes, a reasonable finding considering that roughly two-thirds of the schools represented in our sample have some religious affiliation. We do not find evidence that test score outcomes and higher education matriculation contribute meaningfully to perceived success.

More →

The National Assessment of Educational Progress (NAEP) has tested the civic, or citizenship knowledge of students across the nation at irregular intervals since its very inception. Despite advancements in reading and mathematics, evidenced by results from the National Assessment of Educational Progress (NAEP), civics proficiency has remained consistently low, which raises concerns among educators and policymakers. This study attempts to provide those educators and policymakers with state-level predictions, not currently provided for the civics assessment. This research addresses this gap in state-level civics education data by applying multilevel regression with poststratification (MRP) to NAEP's nationally representative civics scores, yielding state-specific estimates that account for student demographics. A historical analysis of NAEP's development underscores its significance in national education and highlights the challenges of transitioning to state-level reporting, particularly for civics, which lacks state-level generalizability. Furthermore, this paper evaluates NAEP's frameworks, questioning their alignment with civics education's evolving needs, and investigates the presence of opportunity gaps in civics knowledge across gender and racial/ethnic lines. By comparing MRP estimates with published NAEP results, the study validates the method's credibility and emphasizes the potential of MRP in educational research. The findings reveal persistent racial/ethnic disparities in civic knowledge, with profound implications for civics instruction and policy. The research concludes by stressing the necessity for state-specific data to inform education policy and practice, advocating for teaching methods that enhance civic understanding and engagement, and suggesting future research directions to address the uncovered disparities.

More →