The Effect of Retention Threat on Student Learning Gains: Evidence from Ohio

Sarah Souders and Robert Bifulco

Eighteen states currently use test-based retention in third grade as the final component of their K-3 literacy policy, and four more are set to adopt it by 2029. At the same time, three states have repealed theirs, including Ohio (the setting for this study) beginning in 2024-25. Forty-two states have considered adding or removing a retention component in recent years, which makes this one of the most actively contested policies in early literacy.

The debate has centered on the students who get held back. Advocates argue that retention gives struggling readers another year to build skills. Opponents point to the costs, the stigma, and evidence that retention widens gaps because families differ in their ability to advocate for promotion.

That framing leaves out most of the students the policy touches. In Ohio, roughly a third of third graders fall below the retention threshold on the fall reading test and spend the year knowing they may be held back. About 3% are actually retained. The remaining students, along with their families and teachers, spend a full school year responding to a risk that never materializes. This raises the question: what happens to students who face the threat of retention but are never retained?

This study answers that question using a feature of Ohio's Third Grade Reading Guarantee that few states share. Ohio administered the promotion test in both the fall and the spring of third grade, and students could secure promotion by passing either administration. Students who scored just below the fall threshold spent the year under threat of retention. Students who scored just above it did not. Comparing these two groups isolates the effect of a year spent under retention threat, separate from the effect of being held back.

STUDY AND METHODS

This study uses Ohio administrative records covering 503,783 students across 1,869 public schools, following four cohorts who entered third grade for the first time between 2014 and 2017 through eighth grade.

Ohio's Third Grade Reading Guarantee has an unusual feature that makes this analysis possible. The state gives the third grade reading test twice, once in late October and again in April, and passing either administration secures promotion to fourth grade. Students who score below the threshold in the fall receive a report showing their score against the cutoff, along with a description of what retention would mean. Those students spend the year under threat of retention. Students who clear the fall threshold do not.

Two different tests do two different jobs here:

• The K-3 diagnostic: This is administered by September 30, and its job is to trigger services. The diagnostic classifies each student as “On-Track” or “Not On-Track” relative to the reading level expected at the end of the previous grade. Students flagged as “Not On-Track” must receive a Reading Improvement and Monitoring Plan (RIMP), which brings parent notification and supplemental literacy services.
• The Grade 3 ELA test: This determines retention threat and promotion. It runs in late October and again in the spring, and passing either administration secures promotion to fourth grade. It also feeds the report card measures and the district bonus payments.

Figure 1: Student pathways under Ohio's Third Grade Reading Guarantee.

The authors use a regression discontinuity design to compare students who scored just below the fall cutoff to students who scored just above it. Among students that close to the threshold, which side of the line a student lands on comes down to a few test questions, so the two groups are effectively interchangeable at the start of the year. The authors confirm this: the groups match on gender, race, economic disadvantage, English learner status, disability status, and predicted spring achievement, and there is no evidence that scores were manipulated near the cutoff.

Three questions organize the study:

1. Does facing the threat of retention change what students learn? The authors compare spring third-grade reading and math scores for students on either side of the fall threshold.
2. Do any learning gains last? They follow the same students through eighth grade, and also look at absences, disciplinary incidents, and whether students were retained in a later grade.
3. Why would the threat of retention produce gains? They examine whether students under threat were more likely to be assigned reading interventions, and whether effects differed by district spending and by the share of at-risk students a school serves.

KEY FINDINGS

1. Students who faced the threat of retention scored higher in reading at the end of third grade, and the gains persisted through seventh grade.
   • Students under retention threat scored 0.077 SD higher on the spring third-grade ELA test than students who narrowly cleared the fall threshold.
   • Gains persisted through seventh grade. This pattern matters because the retention literature has been shaped by findings that gains fade. Students who are actually retained tend to post large early gains that shrink as their peers catch up. Here the gains shrink but hold, five years after the threat has passed.
   • Persistence is also the strongest evidence that something real happened. A student cramming for a single high-stakes test in third grade would not still be ahead in seventh.
2. Math scores rose as well, a subject that carried no promotion stakes, which suggests the increase in scores was actual student learning rather than test preparation.
   • Students under retention threat scored higher in Grade 3 math, with smaller gains of persisting through Grade 6. Nothing in the policy rewarded math performance. The most plausible implication is that a year of increased attention, instruction, and effort produced improvement that was not confined to the tested subject.
   • Taken together with the persistence of the reading effects, this is the study's central argument that the policy changed learning rather than test performance.
3. The effect of retention threat grew as Ohio added school-level accountability and financial incentives, which suggests the threat works as part of a system rather than on its own.
   • Estimated third grade reading effects were 0.056 and 0.034 standard deviations for the 2014 and 2015 cohorts, then rose to 0.101 and 0.078 for the 2016 and 2017 cohorts. Ohio began paying districts bonuses tied to third grade proficiency and started publishing an early literacy measure on school report cards. That measure credits schools specifically for students flagged as behind who reach proficiency, which places the pressure on the same students who showed the largest gains.
4. Retention threat helped students flagged as “Not On-Track” more than it helped students flagged as “On-Track,” and the difference grew through sixth grade.
   • Among “Not On-Track” students, the effect of retention threat was 0.102 standard deviations, compared with 0.063 for students flagged as “On-Track.” By sixth grade, the difference had grown rather than narrowed
   • Ohio's report card and funding formula reward improvement specifically among “Not On-Track” students who reach proficiency. The students the accountability system pointed schools toward are the students who gained the most, which suggests school behavior rather than student motivation drove much of the difference.
5. Students under retention threat missed less school, and the study found no evidence of the harms critics of retention have raised, such as disengagement or increased disciplinary incidents.
   • Absence rates decreased by a small amount.
   • Disciplinary incidents showed no consistent effect in any grade, and neither did the probability of being retained in a later grade. A common worry about retention policies is that students who struggle early get pushed toward disengagement, and this study finds no sign of that among students who faced the threat.
6. High-needs schools largely responded at the grade or classroom level rather than the student level.
   • Schools serving the highest concentrations of at-risk students produced the largest achievement gains while showing the smallest increases in individualized intervention assignment.
   • The pattern is consistent with high-need schools responding at the grade or classroom level rather than student by student, which is plausible when a majority of a grade is flagged. It is also consistent with these schools facing the strongest report card pressure. Either way, the individualized plan is not the only route to results.
7. Whether retention threat produced any gains depended on the district. The highest-spending districts saw the largest effects of retention threat on third-grade reading, and the lowest-spending districts showed no detectable gains.
   • A retention mandate creates pressure on schools to respond, and pressure by itself does not purchase teacher time, tutoring, or reading specialists. Where districts had resources to convert that pressure into instruction, students gained. Where they did not, the threat appears to have accomplished little.

POLICY AND PRACTICE IMPLICATIONS

1. Evaluations that count only retained students understate what a retention mandate does, in either direction.
   • In Ohio, about 3% of third graders are retained while roughly a third spend the year under threat. Any accounting of the policy's costs and benefits that looks only at the retained students is describing a small fraction of the students affected.
   • The effect of retention threat operated almost entirely without actual retention.
2. Ohio's fall testing gave students, families, and schools most of a school year to respond, a design feature most states do not share.
   • Students received a report in the fall showing their score against the retention threshold, and could secure promotion by passing in either the fall or the spring. Most states administer the high-stakes test in the spring, with a summer retake afterward.
   • States that identify at-risk students only in the spring would not be expected to produce the same response within Grade 3.
3. Ohio's results describe a retention mandate embedded in a broader early literacy system, and a state adopting the mandate alone should not expect the same effects.
   • Student-level and school-level accountability appear to reinforce each other here, so evidence on retention threat from Ohio does not transfer cleanly to a state that adopts a retention mandate without the surrounding measures.
4. Requiring a reading plan may not be enough on its own, since students who had a plan and also faced retention gained substantially more than students who had a plan and did not face retention.
   • Because both groups carried the same legal requirement, the difference points to what happened after the plan was written rather than to the plan itself. The study finds that students under threat were more likely to participate in individualized reading interventions, although other potential responses—such as closer monitoring, more contact with families, or teachers prioritizing those students—cannot be directly observed.
5. District capacity shaped how much students gained, with no detectable effect in the lowest-spending districts.
   • In districts in the bottom fifth of pre-policy per-pupil spending, the estimated effect on third grade reading was near zero. States adopting retention policies without accompanying funding may see gains concentrated in districts that need them least.
6. This study measures the benefits to students who face retention threat and does not weigh them against the costs to students who are retained.
   • The evidence on actual retention remains mixed, with short-run gains that tend to fade, and retention carries real costs to students and to taxpayers. Those costs fall on a different group of students than the one studied here.
   • A complete assessment of a retention mandate needs both sides. This study establishes that the benefit side is larger than previously measured, without establishing that it outweighs the cost side.

FULL WORKING PAPER:

This report is based on the EdWorkingPaper "The Effect of Retention Threat on Student Learning Gains: Evidence from Ohio," published in July 2026. The full research paper can be found here: https://doi.org/10.26300/ps8z-qv79.