Program and policy effects

We leverage log data from an educational app and two-way text message records from over 3,500 students during the summers of 2019 and 2020, along with in-depth interviews in Spanish and English, to identify patterns of family engagement with educational technology. Based on the type and timing of technology use, we identify several distinct profiles of engagement, which we group into two categories: Independent Users who engage with technology-based educational software independently, and Interaction-Supported Users who use two-way communications to support their engagement. We also find that as the demands of families from schools increased during the COVID-19 pandemic, Spanish-speaking families were significantly more likely than English-speaking families to engage with educational technology across all categories of families, particularly as Interaction-Supported Users.

More →

In 2006, the federal government effectively uncapped student borrowing for graduate programs with the introduction of the Graduate PLUS loan program. Access to additional federal loans increased graduate students’ borrowing and shifted the composition of their loans from private to federal debt. However, the increase in borrowing limits did not improve access to existing programs overall or for underrepresented groups. Nor did access to additional loan aid result in significant increase in constrained students’ persistence or degree receipt. We document that among programs in which a larger share of graduate students had exhausted their annual federal loan eligibility before the policy change—and thus were more exposed to the expansion in access to credit—federal borrowing and prices increased.

More →

School finance court cases have proceeded one or more times in all but two states. Plaintiffs ask the courts to rule that the existing funding formula is unconstitutional under state constitutions, and the defendants call for continuation of the existing finance formula. By compiling and analyzing the universe of such cases, we can accurately describe the nature of the cases, the decisions made, and the long run impact on overall financing of schools. Defendants win a slight majority of decisions with, surprisingly, their victories coming most frequently in low spending states and in low achieving states. And, while plaintiff victories on average yield an immediate increase in funding, they have no influence on long run growth in school spending.

More →

This study uses implementation fidelity data from PreK to 1st grade in the Boston Public Schools (BPS) to measure instructional alignment and examine whether stronger alignment is associated with sustained benefits of BPS PreK on children’s language, literacy, and math skills through first grade. The study includes N = 498 students (mean age = 5.47, SD = 0.30 in K fall). Children who experienced strong instructional alignment across grades had faster gains in literacy (SD = .47) and math (SD = .28) skills through the spring of first grade compared with non-BPS PreK attenders. Mis-alignment predicted faster convergence in literacy skills. Results highlight that instructional alignment may help to sustain the initial benefits of PreK programs through first grade in a subset of outcome domains. Implications for further research measuring alignment in a broader range of settings and implications for practice are discussed.

More →

The debate on the stringency of licensure exams for prospective public school teachers is on-going, including the recent controversial roll-out of the educative Teacher Performance Assessment (edTPA). We leverage the quasi-experimental setting of different adoption timing by states and analyze multiple data sources containing a national sample of prospective teachers and students of new teachers in the US. With extensive controls of concurrent policies, we  find that the edTPA reduced prospective teachers in undergraduate programs, less-selective and minority-concentrated universities. Contrary to the policy intention, we do not  find evidence that edTPA increased student test scores.

More →

The media discourse on student loans plays a significant role in the way that policy actors conceptualize challenges and potential solutions related to student debt. This study examines the racialized language in student loan news articles published in eight major news outlets between 2006 and 2021. We found that 18% of articles use any racialized language, though use has accelerated since 2018. This increase appears to be driven by terms that denote groups of people instead of structural problems, with 8% of articles mentioning “Black” but less than 1% mentioning “racism.” These findings emphasize the importance of treating the media as a policy actor capable of shaping the salience of racialization in discussions about student loans.

More →

Efforts to attract and retain effective educators in high poverty public schools have had limited success. Dallas ISD addressed this challenge by using information produced by its evaluation and compensation reforms as the basis for effectiveness-adjusted payments that provided large compensating differentials to attract and retain effective teachers in its lowest achievement schools. The Accelerating Campus Excellence (ACE) program offers salary supplements to educators with records of high performance who are willing to work in the most educationally disadvantaged schools. We document that ACE resulted in immediate and sustained increases in student achievement, providing strong evidence that the multi-measure evaluation system identifies effective educators who foster the development of cognitive skills. The improvements at ACE schools were dramatic, bringing average achievement in the previously lowest performing schools close to the district average. When ACE stipends are largely eliminated, a substantial fraction of highly effective teachers leaves, and test scores fall. This highlights the central importance of the performance-based incentives to attract and retain effective educators in previously low-achievement schools.

More →

We investigated the effectiveness of a sustained and spiraled content literacy intervention that emphasizes building domain and topic knowledge schemas and vocabulary for elementary-grade students. The Model of Reading Engagement (MORE) intervention underscores thematic lessons that provide an intellectual framework for helping students connect new learning to a general schema in Grade 1 (animal survival), Grade 2 (how scientists study past events), and Grade 3 (our human body, a living system that helps us survive). A total of 30 elementary schools (N = 2,870 students) were randomized to a treatment or control condition. In the treatment condition (i.e., full spiral curriculum schools), students participated in content literacy lessons from Grades 1 to 3 during the school year and wide reading of thematically related informational texts in the summer following Grades 1 and 2. In the control condition (i.e., partial spiral curriculum schools), students participated in Grade 3 MORE lessons. Grade 3 lessons for both conditions were implemented online during the COVID-19 pandemic school year. Results reveal that treatment group students outperformed control students on science vocabulary knowledge across all three grades. Furthermore, we found positive transfer effects on Grade 3 science reading (ES = .14), domain-general reading comprehension (ES = .11), and mathematics achievement (ES = .12). Treatment impacts were sustained at 14-month follow-up on Grade 4 reading comprehension (ES = .12) and mathematics achievement (ES = .16). Findings indicate that a content literacy intervention that spirals topics and vocabulary across grades can improve students’ long-term academic achievement outcomes.

More →

A fundamental question for education policy is whether outcomes-based accountability including comprehensive educator evaluations and a closer relationship between effectiveness and compensation improves the quality of instruction and raises achievement. We use synthetic control methods to study the comprehensive teacher and principal evaluation and compensation systems introduced in the Dallas Independent School District (Dallas ISD) in 2013 for principals and 2015 for teachers. Under this far-reaching reform, educator evaluations that are used to support teacher growth and determine salary depend on a combination of supervisor evaluations, student achievement, and student or family survey responses. The reform replaced salary scales based on experience and educational attainment with those based on evaluation scores, a radical departure from decades of rigid salary schedules. The synthetic control estimates reveal positive and significant effects of the reforms on math and reading achievement that increase over time. From 2015 through 2019, the average achievement for the synthetic control district fluctuates narrowly between -0.27 s.d. and -0.3 s.d., while the Dallas ISD average increases steadily from -0.28 s.d. in 2015 to -0.08 s.d. in 2019, the final year of the sample. Though the increase for reading is roughly half as large, it is also highly significant.

More →

We estimate the societal costs associated with corequisite and traditional pre-requisite English developmental education and compare them to societal benefits. Our context is the randomized controlled trial conducted by Miller et al. (2022) that estimated the effects of three different approaches to English corequisites implemented in 5 Texas community colleges. The main drivers of differential costs across pathways and colleges are the number of credit and contact hours in each pathway, class sizes, and the type of faculty used to teach courses (adjunct or full-time). Corequisites are less expensive than pre-requisite pathways in two colleges, they are more expensive yet roughly similar in two other colleges, and they are much more expensive in one college. Miller et al. (2022) find that corequisites induced more students to pass the required college-level English course in all colleges, but do not find that they impacted persistence in college. Considering the enormous societal benefit of a college education, corequisites are most likely the preferred policy from a societal point of view even when they are more expensive to implement and given that they only have a small impact on the likelihood of completing college. From students’ point of view, corequisites are always preferred because they require less tuition and have a higher likelihood of success.

More →