Search for EdWorkingPapers here by author, title, or keywords.
Federal policy has both incentivized and supported better use of research evidence by educational leaders. However, the extent to which these leaders are well-positioned to understand foundational principles from research design and statistics, including those that underlie the What Works Clearinghouse ratings of research studies, remains an open question. To investigate educational leaders’ knowledge of these topics, we developed a construct map and items representing key concepts, then conducted surveys containing those items with a small pilot sample (n=178) and a larger nationally representative sample (n=733) of educational leaders. We found that leaders’ knowledge was surprisingly inconsistent across topics. We also found most items were answered correctly by less than half of respondents, with cognitive interviews suggesting that some of those correct answers derived from guessing or test-taking techniques. Our findings identify a roadblock to policymakers’ contention that educational leaders should use research in decision-making.
The evidence that student learning declines sharply (or stagnates) during the summer has motivated a substantial interest in programs that provide intensive academic instruction during the summer. However, the existing literature suggests that such programs, which typically focus on just one or two subjects, have modest effects on students’ achievement and no impact on measures of their engagement in school. In this quasi-experimental study, we present evidence on the educational impact of a unique and mature summer learning program that serves low-income middle school students and features unusual academic breadth and a social emotional curriculum with year-to-year scaffolding. Our results indicate that this program led to substantial reductions in unexcused absences, chronic absenteeism and suspensions and a modest gain in ELA test scores. We find evidence that the gains in behavioral engagement grow over time and with additional summers of participation. Our results also suggest that these effects were particularly concentrated among boys and Latinx students.
We study the adoption and implementation of a new mobile communication app among a sample of 132 New York City public schools. The app provides a platform for sharing general announcements and news as well as engaging in personalized two-way communication with individual parents. We provide participating schools with free access to the app and randomize schools to receive intensive support (training, guidance, monitoring, and encouragement) for maximizing the efficacy of the app. Although user supports led to higher levels of communication within the app in the treatment year, overall usage remained low and declined in the following year when treatment schools no longer received intensive supports. We find few subsequent effects on perceptions of communication quality or student outcomes. We leverage rich internal user data to explore how take-up and usage patterns varied across staff and school characteristics. These analyses help to identify early adopters and reluctant users, revealing both opportunities and obstacles to engaging parents through new communication technology.
This article takes stock of where the field of behavioral science applied to education policy seems to be at, which avenues seem promising and which ones seem like dead ends. I present a curated set of studies rather than an exhaustive literature review, categorizing interventions by whether they nudge (keep options intact) or “shove” (restrict choice), and whether they apply a high or low touch (whether they use face-to-face interaction or not). Many recent attempts to test large-scale low touch nudges find precisely estimated null effects, suggesting we should not expect letters, text messages, and online exercises to serve as panaceas for addressing education policy’s key challenges. Programs that impose more choice-limiting structure to a youth’s routine, like mandated tutoring, or programs that nudge parents, appear more promising.
International assessments are important to benchmark the quality of education across countries. However, on low-stakes tests, students’ incentives to invest their maximum effort may be minimal. Research stresses that ignoring students’ effort when interpreting results from low-stakes assessments can lead to biased interpretations of test performance across groups of examinees. We use data from the Programme for International Student Assessment (PISA), a low-stakes test, to analyze the extent to which student effort helps to explain test scores heterogeneity across countries and by gender groups. Our results highlight the importance of accounting for differences in student effort to understand cross-country heterogeneity in performance and variations in gender achievement gaps across nations. We find that, once we account for differential student effort across gender groups, the estimated gender achievement gap in math and science could be up to 12 and 6 times wider, respectively, and up to 49 percent narrower in reading, in favor of boys. In math and science, the gap widens in most countries, even among some of the top 20 most gender-equal countries. Altogether, our effort measures on average explain between 36 and 40 percent of the cross-country variation in test scores.
The worldwide school closures in early 2020 led to losses in learning that will not easily be made up for even if schools quickly return to their prior performance levels. These losses will have lasting economic impacts both on the affected students and on each nation unless they are effectively remediated.
While the precise learning losses are not yet known, existing research suggests that the students in grades 1-12 affected by the closures might expect some 3 percent lower income over their entire lifetimes. For nations, the lower long-term growth related to such losses might yield an average of 1.5 percent lower annual GDP for the remainder of the century. These economic losses would grow if schools are unable to re-start quickly.
The economic losses will be more deeply felt by disadvantaged students. All indications are that students whose families are less able to support out-of-school learning will face larger learning losses than their more advantaged peers, which in turn will translate into deeper losses of lifetime earnings.
The present value of the economic losses to nations reach huge proportions. Just returning schools to where they were in 2019 will not avoid such losses. Only making them better can. While a variety of approaches might be attempted, existing research indicates that close attention to the modified re-opening of schools offers strategies that could ameliorate the losses. Specifically, with the expected increase in video-based instruction, matching the skills of the teaching force to the new range of tasks and activities could quickly move schools to heightened performance. Additionally, because the prior disruptions are likely to increase the variations in learning levels within individual classrooms, pivoting to more individualised instruction could leave all students better off as schools resume.
As schools move to re-establish their programmes even as the pandemic continues, it is natural to focus considerable attention on the mechanics and logistics of safe re-opening. But the long-term economic impacts also require serious attention, because the losses already suffered demand more than the best of currently considered re-opening approaches.
State testing programs regularly release previously administered test items to the public. We provide an open-source recipe for state, district, and school assessment coordinators to combine these items flexibly to produce scores linked to established state score scales. These would enable estimation of student score distributions and achievement levels. We discuss how educators can use resulting scores to estimate achievement distributions at the classroom and school level. We emphasize that any use of such tests should be tertiary, with no stakes for students, educators, and schools, particularly in the context of a crisis like the COVID-19 pandemic. These tests and their results should also be lower in priority than assessments of physical, mental, and social–emotional health, and lower in priority than classroom and district assessments that may already be in place. We encourage state testing programs to release all the ingredients for this recipe to support low-stakes, aggregate-level assessments. This is particularly urgent during a crisis where scores may be declining and gaps increasing at unknown rates.
This study examines the effects of English Learner (EL) status on subsequent Special Education (SPED) placement. Through a research-practice partnership, we link student demographic data and initial English proficiency assessment data across seven cohorts of test takers and observe EL and SPED programmatic participation for these students over seven years. Our regression discontinuity estimates consistently differ substantively from results generated through regression analyses. We find evidence that the effect of EL status on SPED placement was either null or tied to slight under-identification. Our results suggest that under-identification occurred two years after EL classification. We also find that EL status led to under-identification for Spanish speakers and proportionate representation for Mandarin/Cantonese speakers and speakers of all other languages.
In this paper we study the impact on student absenteeism of a large school-based community crime monitoring program that employed local community members to monitor and report crime on designated city blocks during students’ travel to and from school. We find that the program resulted in a 0.78 percentage point reduction in the school-level absence rate (11 percent effect). We explore two potential channels to explain this: we find improvements “outside of the school walls” in the form of reduced crime near treated schools and “inside of the school walls” in the form of reduced incidents of serious student misconduct.
StudentU is a comprehensive program that provides education, nutrition, and social support services to disadvantaged students outside of the regular school day. In this paper I investigate the effects of this multi-year program on the early high school outcomes of participating students by exploiting data from oversubscribed admissions lotteries. I estimate that lottery winners who entered the comprehensive program with low baseline achievement earned more course credits (0.82 credits), achieved higher grade point averages (0.37 grade points), and were less likely to be suspended (17.1 percentage points) during ninth grade than their lottery loser counterparts. Investigation of candidate channels indicates that increased student effort and improved behavior in school are likely mechanisms. Using an index of early high school outcomes, I predict that lottery winners are around 4 percentage points more likely to graduate from high school than lottery losers (5 percent effect). These results suggest that comprehensive services delivered outside of the regular school day have the potential to improve the educational outcomes of disadvantaged students.