Search EdWorkingPapers
Search for EdWorkingPapers here by author, title, or keywords.
Search
EdWorkingPapers
The educative Teacher Performance Assessment (edTPA) - a performance-based examination for prospective PreK-12 teachers to guarantee teaching readiness - has gained popularity in recent years. This research offers the first causal evidence about the effects of this nationwide initiative on teacher supply and student outcomes of new teachers. We leverage the quasi-experimental setting of different adoption timing by states and analyze multiple data sources containing a national sample of prospective teachers and students of new teachers in the US. We find that the new license requirement reduced the number of graduates from teacher preparation programs by 14%. The negative effect is stronger for non-white prospective teachers at less-selective universities. Contrary to the policy intention, we find evidence that edTPA has adverse effects on student learning.
This paper examines how financial aid reform based on postsecondary institutional performance impacts student choice. Federal and state regulations often reflect concerns about the private, for-profit sector's poor employment outcomes and high loan defaults, despite the sector's possible theoretical advantages. We use student level data to examine how eliminating public subsidies to attend low-performing for-profit institutions impacts students' college enrollment and completion behavior. Beginning in 2011, California tightened eligibility standards for their state aid program, effectively eliminating most for-profit eligibility. Linking data on aid application to administrative payment and postsecondary enrollment records, this paper utilizes a differences-in-differences strategy to investigate students' enrollment and degree completion responses to changes in subsidies. We find that restricting the use of the Cal Grant at for-profit institutions resulted in significant state savings but led to relatively small changes in students' postsecondary trajectories. For older, non-traditional students we find no impact on enrollment or degree completion outcomes. Similarly, for high school graduates, we find that for-profit enrollment remains strong. Unlike the older, non-traditional students, however, there is some evidence of declines in for-profit degree completion and increased enrollment at community colleges among the high school graduates, but these results are fairly small and sensitive to empirical specification. Overall, our results suggest that both traditional and non-traditional students have relatively inelastic preferences for for-profit colleges under aid-restricting policies.
We examine all known "credibly causal" studies to explore the distribution of the causal effects of public K-12 school spending on student outcomes in the United States. For each of the 31 included studies, we compute the same marginal spending effect parameter estimate. Precision-weighted method of moments estimates indicate that, on average, a $1000 increase in per-pupil public school spending (for four years) increases test scores by 0.0352 standard deviations, high school graduation by 1.92 percentage points, and college-going by 2.65 percentage points. These pooled averages are significant at the 0.0001 level. When benchmarked against other interventions, test score impacts are smaller than those on educational attainment -- suggesting that test-score impacts understate the value of school spending.
The benefits to marginal capital spending increases take about five years to materialize, and are about half as large as (and less consistently positive than) those of non-capital-specific spending increases. The marginal spending impacts for all spending types are less pronounced for economically advantaged populations -- though not statistically significantly so. Consistent with a cumulative effect, the educational attainment impacts are larger with more years of exposure to the spending increase. Average impacts are similar across a wide range of baseline spending levels and geographic characteristics -- providing little evidence of diminishing marginal returns at current spending levels.
To assuage concerns that pooled averages aggregate selection or confounding biases across studies, we use a meta-regression-based method that tests for, and removes, certain biases in the reported effects. This approach is straightforward and can remove biases in meta-analyses where the parameter of interest is a ratio, slope, or elasticity. We fail to reject that the meta-analytic averages are unbiased. Moreover, policies that generate larger increases in per-pupil spending tend to generate larger improvements in outcomes, in line with the pooled average.
To speak to generalizability, we estimate the variability across studies attributable to effect heterogeneity (as opposed to sampling variability). This heterogeneity explains between 76 and 88 percent of the variation across studies. Estimates of heterogeneity allow us to provide a range of likely policy impacts. Our estimates suggest that a policy that increases per-pupil spending for four years will improve test scores and/or educational attainment over 90 percent of the time. We find evidence of small possible publication bias among very imprecise studies, but show that any effects on our precision-weighted estimates are minimal.
Colleges have increasingly turned to predictive analytics to target at-risk students for additional support. Most of the predictive analytic applications in higher education are proprietary, with private companies offering little transparency about their underlying models. We address this lack of transparency by systematically comparing two important dimensions: (1) different approaches to sample and variable construction and how these affect model accuracy; and (2) how the selection of predictive modeling approaches, ranging from methods many institutional researchers would be familiar with to more complex machine learning methods, impacts model performance and the stability of predicted scores. The relative ranking of students’ predicted probability of completing college varies substantially across modeling approaches. While we observe substantial gains in performance from models trained on a sample structured to represent the typical enrollment spells of students and with a robust set of predictors, we observe similar performance between the simplest and most complex models.
Is public housing bad for children? Critics charge that public housing projects concentrate poverty and create neighborhoods with limited opportunities, including low-quality schools. However, whether the net effect is positive or negative is theoretically ambiguous and likely to depend on the characteristics of the neighborhood and schools compared to origin neighborhoods. In this paper, we draw on detailed individual-level longitudinal data on students moving into New York City public housing and examine their academic outcomes over time. Exploiting plausibly random variation in the precise timing of entry into public housing, we estimate credibly causal effects of public housing using both difference-in-differences and event study designs. We find credibly causal evidence of positive effects of moving into public housing on student test scores, with larger effects over time. Stalled academic performance in the first year of entry may reflect, in part, disruptive effects of residential and school moves. Neighborhood matters: effects are larger for students moving out of low-income neighborhoods or into higher-income neighborhoods, and these students move to schools with higher average test scores and lower shares of economically disadvantaged peers. We also find some evidence of improved attendance outcomes and reduction in incidence of childhood obesity for boys following public housing residency. Our study results refute the popular belief that public housing is bad for kids and probe the circumstances under which public housing may work to improve academic outcomes for low-income students.
School bullying is widespread and has substantial social costs. One in five U.S. high school students report being bullied each school year and these students face greater risks of serious mental health challenges that extend into adulthood. As the COVID-19 pandemic forced most students into online education, many have worried that cyberbullying prevalence would grow dramatically. We use data from Google internet searches to examine changing bullying patterns as COVID-19 disrupted in-person schooling. Pre-pandemic historical patterns show that internet searches provide useful information about actual bullying behavior. Real-time data then shows that searches for school bullying and cyberbullying both dropped about 30-40 percent as schools shifted to remote learning in spring 2020. This drop is sustained through the fall and winter of the 2020-21 school year, though the gradual return to in-person instruction partially returns bullying searches to pre-pandemic levels. These results highlight how in-person interaction is an important mechanism underlying not only in-person school bullying, but also cyberbullying. We discuss how this otherwise damaging shock to students and schools provides insight into the mixed impact of the pandemic on student well-being.
Covid-19-induced school closures generated great interest in tutoring as a strategy to make up for lost learning time. Tutoring is backed by a rigorous body of research, but it is unclear whether it can be delivered effectively remotely. We study the effect of teacher-student phone call interventions in Kenya when schools were closed. Schools (n=105) were randomly assigned for their 3rd, 5th and 6th graders (n=8,319) to receive one of two versions of a 7-week weekly math-focused intervention—5-minute accountability checks or 15-minute mini-tutoring sessions—or to the control group. Although calls increased student perceptions that teachers cared, accountability checks had no effect on math performance up to four months after the intervention and tutoring decreased math achievement among students who returned to their schools after reopening. This was, in part, because the relatively low-achieving students most likely to benefit from calls were least likely to return and take in-person assessments. Tutoring substituted away from more productive uses of time, at least among returning students. Neither intervention affected enrollment. Tutoring remains a valuable tool but to avoid unintended consequences, careful attention should be paid to aligning tutoring interventions with best practices and targeting interventions to those who will benefit most.
Do Americans hold a consistent set of opinions about their public schools and how to improve them? From 2013 to 2018, over 5,000 unique respondents participated in more than one consecutive iteration of the annual, nationally representative Education Next poll, offering an opportunity to examine individual-level attitude stability on education policy issues over a six-year period. The proportion of participants who provide the same response to the same question over multiple consecutive years greatly exceeds the amount expected to occur by chance alone. We also find that teachers offer more consistent responses than their non-teaching peers. By contrast, we do not observe similar differences in attitude stability between parents of school-age children and their counterparts without children.
We consider the case in which the number of seats in a program is limited, such as a job training program or a supplemental tutoring program, and explore the implications that peer effects have for which individuals should be assigned to the limited seats. In the frequently-studied case in which all applicants are assigned to a group, the average outcome is not changed by shuffling the group assignments if the peer effect is linear in the average composition of peers. However, when there are fewer seats than applicants, the presence of linear-in-means peer effects can dramatically influence the optimal choice of who gets to participate. We illustrate how peer effects impact optimal seat assignment, both under a general social welfare function and under two commonly used social welfare functions. We next use data from a recent job training RCT to provide the first evidence of large peer effects in the context of job training for disadvantaged adults. Finally, we combine the two results to show that the program's effectiveness varies greatly depending on whether the assignment choices account for or ignore peer effects.
Sixty-seven school finance reforms (SFRs), a combination of court-ordered and legislative reforms, have taken place since 1990; however, there is little empirical evidence on the heterogeneity of SFR effects. In this study, we estimate the effects of SFRs on revenues and expenditures between 1990 and 2014 for 26 states. We find that, on average, per pupil spending increased, especially in low-income districts relative to high-income districts. However, underlying these average effect estimates, the distribution of state-level effect sizes ranges from negative to positive---there is substantial heterogeneity. When predicting SFR impacts, we find that multiple state-level SFRs, union strength, and some funding formula components are positively associated with SFR effect sizes in low-income districts. We also show that, on average, states without SFRs adopted funding formula components and increased K-12 state revenues similarly to states with SFRs.