Search EdWorkingPapers by author, title, or keywords.
Program and policy effects
The Cobb Teaching & Learning System (CTLS) is a digital learning initiative developed for and by the Cobb County School District (CCSD) in Georgia. CTLS became a crucial initiative used by the district to maintain student academic progress during the COVID-19 pandemic. Adopting a mixed-methods approach, this case study seeks to analyze CTLS’s design and implementation, focusing on digital transformation and professional collaboration within CCSD. This case study highlights how CCSD maintains complete ownership in a customized digital learning initiative supported by technology providers.
CTLS’s success comes from its strategic partnership with external technology providers, most notably EdIncites, commitment to professional collaboration, investment in novel technologies, and focus on real-time data. Looking at district-by-district comparisons, Cobb’s level of achievement and learning recovery resembles that of higher performing suburban districts in Georgia as opposed to its closest geographically and demographically comparable peers. Furthermore, 2019-2022 testing data indicates that all GA Milestone End-Of-Course proficiency percentages have already exceeded a 2014 baseline. This suggests that CTLS played a central role in CCSD’s successful recovery after the COVID-19 pandemic.
The overall response to the digital learning initiative from the end users that it is intended to serve has also been overwhelmingly positive. The initiative is now well-positioned to broaden learning opportunities across all schools and improve communication with parents and other stakeholders. CCSD’s experience in scaling CTLS offers useful lessons for districts that are ready to launch and to own their transformative digital learning environment.
We provide evidence that graduated driver licensing (GDL) laws, originally intended to improve public safety, impact human capital accumulation. Many teens use automobiles to access both school and employment. Because school and work decisions are interrelated, the effects of automobile-specific mobility restrictions are ambiguous. Using a novel triple-difference research design, we find that restricting mobility significantly reduces high school dropout rates and teen employment. We develop a multiple discrete choice model that rationalizes unintended consequences and reveals that school and work are weak complements. Thus, improved educational outcomes reflect decreased access to leisure activities rather than reduced labor market access.
Faced with decreasing funds and increasing costs, a growing number of school districts across the United States are switching to four-day school weeks (4DSWs). Although previously used only by rural districts, the policy has begun to gain traction in metropolitan districts. We examine homeowner, teacher, and student outcomes in one of the first metropolitan school districts to adopt the 4DSW. We find 2 to 4 percent home price declines relative to surrounding school districts, a 5 percent decrease in teacher retention for experienced teachers, and a 0.2 to 0.3 standard deviation decrease in student test scores. These results suggest the decision to adopt a 4DSW in a metropolitan setting should not be taken lightly.
Short-cycle higher education programs (SCPs) can play a central role in skill development and higher education expansion, yet their quality varies greatly within and among countries. In this paper we explore the relationship between programs’ practices and inputs (quality determinants) and student academic and labor market outcomes. We design and conduct a novel survey to collect program-level information on quality determinants and average outcomes for Brazil, Colombia, Dominican Republic, Ecuador, and Peru. Categories of quality determinants include training and curriculum, infrastructure, faculty, link with productive sector, costs and funding, and practices on student admission and institutional governance. We also collect administrative, student-level data on higher education and formal employment for SCP students in Brazil and Ecuador and match it to survey data. Using machine learning methods, we select the quality determinants that predict outcomes at the program and student levels. Estimates indicate that some quality determinants may favor academic and labor market outcomes while others may hinder them. Two practices predict improvements in all labor market outcomes in Brazil and Ecuador—teaching numerical competencies and providing job market information—and one practice—teaching numerical competencies—additionally predicts improvements in labor market outcomes for all survey countries. Since quality determinants account for 20-40 percent of the explained variation in student-level outcomes, quality determinants might have a role shrinking program quality gaps. Findings have implications for the design and replication of high-quality SCPs, their regulation, and the development of information systems.
Despite the growing popularity of free college proposals, countries with higher college subsidies tend to have higher enrollment rates but not higher graduation rates. To capture this evidence and evaluate potential free college policies, we rely on a dynamic model of college enrollment, performance, and graduation estimated using rich student-level data from Colombia. In the model, student effort affects class completion and mitigates the risk of performing poorly or dropping out. Among our simulated policies, universal free college expands enrollment the most but has virtually no effect on graduation rates, helping explain the cross-country evidence. Performance-based free college triggers a more modest enrollment expansion but delivers a higher graduation rate at a lower fiscal cost. While both programs lower student uncertainty relative to the baseline, performance-based free college does it to a lower extent, which in turn promotes better student outcomes. Overall, free college programs expand enrollment but have limited impacts on graduation and attainment due to their limited impact on student effort.
Growing up in poverty presents numerous nonacademic barriers that impede academic progress for economically disadvantaged students (Duncan and Murnane, 2016). Because schools alone have limited capacity to address the systemic nature of economic inequalities that directly affects student outcomes, policymakers and researchers in recent years have increased calls for the use of comprehensive, integrated support models and wraparound services (Wasser Gish, 2019). Although research on the effects of such interventions has been mixed, evaluations of one model – City Connects – have found significant achievement gains for students who received the intervention in elementary school (Walsh et al., 2014). Given the need to understand the replicability of interventions beyond initial sites of implementation, we assessed the degree to which the intervention effect on math and English Language Arts (ELA) achievement in elementary and middle school replicates in a new site with an important geographical variation. Results from two-way fixed effects and event-study models suggest positive treatment effects of nearly half a standard deviation in both subjects following five years of implementation, supporting the replicability of City Connects.
Teachers affect a wide range of students’ educational and social outcomes, but how they contribute to students’ involvement in school discipline is less understood. We estimate the impact of teacher demographics and other observed qualifications on students’ likelihood of receiving a disciplinary referral. Using data that track all disciplinary referrals and the identity of both the referred and referring individuals from a large and diverse urban school district in California, we find students are about 0.2 to 0.5 percentage points (7% to 18%) less likely to receive a disciplinary referral from teachers of the same race or gender than from teachers of different demographic backgrounds. Students are also less likely to be referred by more experienced teachers and by teachers who hold either an English language learners or special education credential. These results are mostly driven by referrals for defiance and violence infractions, Black and Hispanic male students, and middle school students. While it is unclear whether these findings are due to variation in teachers’ effects on actual student behavior, variation in teachers’ proclivities to make disciplinary referrals, or a combination of the two, these results nonetheless suggest that teachers play a central role in the prevalence of, and inequities in, office referrals and subsequent student discipline.
Providing consistent, individualized feedback to teachers is essential for improving instruction but can be prohibitively resource intensive in most educational contexts. We develop an automated tool based on natural language processing to give teachers feedback on their uptake of student contributions, a high-leverage teaching practice that supports dialogic instruction and makes students feel heard. We conduct a randomized controlled trial as part of an online computer science course, Code in Place (n=1,136 instructors), to evaluate the effectiveness of the feedback tool. We find that the tool improves instructors’ uptake of student contributions by 27% and present suggestive evidence that our tool also improves students’ satisfaction with the course and assignment completion. These results demonstrate the promise of our tool to complement existing efforts in teachers’ professional development.
The impact of school resources on student outcomes was first raised in the 1960s and has been controversial since then. This issue enters into the decision making on school finance in both legislatures and the courts. The historical research found little consistent or systematic relationship of spending and achievement, but this research frequently suffers from significant concerns about the underlying estimation strategies. More recent work has re-opened the fundamental resource-achievement relationship with more compelling analyses that offer stronger identification of resource impacts. A thorough review of existing studies, however, leads to similar conclusions as the historical work: how resources are used is key to the outcomes. At the same time, the research has not been successful at identifying mechanisms underlying successful use of resources or for ascertaining when added school investments are likely to be well-used. Direct investigations of alternative input policies (capital spending, reducing class size, or salary incentives for teachers) do not provide clear support for such specific policy initiatives.
What happens when employers screen their employees but only observe a subset of output? We specify a model with heterogeneous employees and show that their response to the screening affects output in both the probationary period and the post-probationary period. The post-probationary impact is due to their heterogeneous responses affecting which individuals are retained and hence the screening efficiency. We show that the impact of the endogenous response on both the unobserved outcome and screening efficiency depends on whether increased effort on one task increases or decreases the marginal cost of effort on the other task. If the response decreases unobserved output in the probationary period then it increases the screening efficiency, and vice versa. We then assess these predictions empirically by studying a change to teacher tenure policy in New York City, which increased the role that a single measure -- test score value-added -- played in tenure decisions. We show that in response to the policy teachers increased test score value-added and decreased output that did not enter the tenure decision. The increase in test score value-added was largest for the teachers with more ability to improve students' untargeted outcomes, increasing their likelihood of getting tenure. We estimate that the endogenous response to the policy announcement reduced the screening efficiency gap -- defined as the reduction of screening efficiency stemming from the partial observability of output -- by 28%, effectively shifting some of the cost of partial observability from the post-tenure period to the pre-tenure period.