Search EdWorkingPapers
Search EdWorkingPapers by author, title, or keywords.
Search
EdWorkingPapers
Although learners are being connected 1:1 with instructors at an increasing scale, most of these instructors do not receive effective, consistent feedback to help them improved. We deployed M-Powering Teachers, an automated tool based on natural language processing to give instructors feedback on dialogic instructional practices —including their uptake of student contributions, talk time and questioning practices — in a 1:1 online learning context. We conducted a randomized controlled trial on Polygence, a re-search mentorship platform for high schoolers (n=414 mentors) to evaluate the effectiveness of the feedback tool. We find that the intervention improved mentors’ uptake of student contributions by 10%, reduced their talk time by 5% and improves student’s experi-ence with the program as well as their relative optimism about their academic future. These results corroborate existing evidence that scalable and low-cost automated feedback can improve instruction and learning in online educational contexts.
We investigated the effectiveness of a sustained and spiraled content literacy intervention that emphasizes building domain and topic knowledge schemas and vocabulary for elementary-grade students. The Model of Reading Engagement (MORE) intervention underscores thematic lessons that provide an intellectual framework for helping students connect new learning to a general schema in Grade 1 (animal survival), Grade 2 (how scientists study past events), and Grade 3 (our human body, a living system that helps us survive). A total of 30 elementary schools (N = 2,870 students) were randomized to a treatment or control condition. In the treatment condition (i.e., full spiral curriculum schools), students participated in content literacy lessons from Grades 1 to 3 during the school year and wide reading of thematically related informational texts in the summer following Grades 1 and 2. In the control condition (i.e., partial spiral curriculum schools), students participated in Grade 3 MORE lessons. Grade 3 lessons for both conditions were implemented online during the COVID-19 pandemic school year. Results reveal that treatment group students outperformed control students on science vocabulary knowledge across all three grades. Furthermore, we found positive transfer effects on Grade 3 science reading (ES = .14), domain-general reading comprehension (ES = .11), and mathematics achievement (ES = .12). Treatment impacts were sustained at 14-month follow-up on Grade 4 reading comprehension (ES = .12) and mathematics achievement (ES = .16). Findings indicate that a content literacy intervention that spirals topics and vocabulary across grades can improve students’ long-term academic achievement outcomes.
Expected earnings matter for college major choices, and majors differ in both their average earnings and the age profile of their earnings. We show that students' family background is strongly related to the earnings paths of the major they choose. Students with more educated parents, especially those who have graduate degrees, choose majors with lower early-career earnings but much faster earnings growth. They are also less likely to choose safe majors with little early-career earnings or unemployment downside. Parental income has a weaker relationship with major choice and operates mostly through the type of institution the student attends.
A fundamental question for education policy is whether outcomes-based accountability including comprehensive educator evaluations and a closer relationship between effectiveness and compensation improves the quality of instruction and raises achievement. We use synthetic control methods to study the comprehensive teacher and principal evaluation and compensation systems introduced in the Dallas Independent School District (Dallas ISD) in 2013 for principals and 2015 for teachers. Under this far-reaching reform, educator evaluations that are used to support teacher growth and determine salary depend on a combination of supervisor evaluations, student achievement, and student or family survey responses. The reform replaced salary scales based on experience and educational attainment with those based on evaluation scores, a radical departure from decades of rigid salary schedules. The synthetic control estimates reveal positive and significant effects of the reforms on math and reading achievement that increase over time. From 2015 through 2019, the average achievement for the synthetic control district fluctuates narrowly between -0.27 s.d. and -0.3 s.d., while the Dallas ISD average increases steadily from -0.28 s.d. in 2015 to -0.08 s.d. in 2019, the final year of the sample. Though the increase for reading is roughly half as large, it is also highly significant.
Temporary college closures in response to the COVID-19 pandemic created an exodus of students from college towns just as the decennial census count was getting underway. We use aggregate cellular mobility data to evaluate if this population movement affected the distributional accuracy of the 2020 Census. Based on the outflow of devices in late March 2020, we estimate that counties with a college were undercounted by two percent, likely affecting Congressional apportionment. For college towns, student populations can impact government funding allocations, policy program decisions, and planning for infrastructure, public health, and more. The Census Bureau is allowing governmental entities to request count reviews through June 2023. Colleges should cooperate with state and local government efforts to ensure an accurate count.
When analyzing treatment effects on test score data, education researchers face many choices for scoring tests and modeling results. This study examines the impact of those choices through Monte Carlo simulation and an empirical application. Results show that estimates from multiple analytic methods applied to the same data will vary because, as predicted by Classical Test Theory, two-step models using sum or IRT-based scores provide downwardly biased standardized treatment effect coefficients compared to latent variable models. This bias dominates any other differences between models or features of the data generating process, such as the variability of item discrimination parameters. An errors-in-variables (EIV) correction successfully removes the bias from two-step models. Model performance is not substantially different in terms of precision, standard error calibration, false positive rates, or statistical power. An empirical application to data from a randomized controlled trial of a second-grade literacy intervention demonstrates the sensitivity of the results to model selection and tradeoffs between model selection and interpretation. This study shows that the psychometric principles most consequential in causal inference are related to attenuation bias rather than optimal scoring weights.
We estimate the societal costs associated with corequisite and traditional pre-requisite English developmental education and compare them to societal benefits. Our context is the randomized controlled trial conducted by Miller et al. (2022) that estimated the effects of three different approaches to English corequisites implemented in 5 Texas community colleges. The main drivers of differential costs across pathways and colleges are the number of credit and contact hours in each pathway, class sizes, and the type of faculty used to teach courses (adjunct or full-time). Corequisites are less expensive than pre-requisite pathways in two colleges, they are more expensive yet roughly similar in two other colleges, and they are much more expensive in one college. Miller et al. (2022) find that corequisites induced more students to pass the required college-level English course in all colleges, but do not find that they impacted persistence in college. Considering the enormous societal benefit of a college education, corequisites are most likely the preferred policy from a societal point of view even when they are more expensive to implement and given that they only have a small impact on the likelihood of completing college. From students’ point of view, corequisites are always preferred because they require less tuition and have a higher likelihood of success.
U.S. public school students increasingly attend schools with sworn law enforcement officers present. Yet, little is known about how these school resource officers (SROs) affect school environments or student outcomes. Our study uses a fuzzy regression discontinuity (RD) design with national school-level data from 2014 to 2018 to estimate the impacts of SRO placement. We construct this discontinuity based on the application scores for federal school based policing grants of linked police agencies. We find that SROs effectively reduce some forms of violence in schools, but do not prevent gun-related incidents. We also find that SROs intensify the use of suspension, expulsion, police referral, and arrest of students. These increases in disciplinary and police actions are consistently largest for Black students, male students, and students with disabilities.
We demonstrate how mothers, fathers, and 15–17-year-old students alter their schedules around the K-12 academic year. Using regression discontinuity (RDD) methods, combined with dates on school year start and end dates by locality, we document several notable results. First, mothers are substantially more affected by the school year than are fathers. When school is in session, mothers sleep less, spend more time caring for family members and driving them around, and spend less time on eating, free time and exercise. Fathers see changes that are generally similar in sign but smaller in magnitude compared to mothers. 15–17-year-olds naturally reduce time spent in educational pursuits when school is out (a decrease of about 5.5 hours per day on weekdays), and most of that time is substituted toward free time (an additional 2+ hours per day) and sleep (1+ hours per day). Our results provide a holistic picture of how families build their days around the K-12 school calendar and have implications for policies targeted toward women’s and teenage children’s health and well-being.
Graduating from college into a recession is associated with earnings losses, but less is known about how these effects vary across colleges. Using restricted-use data from the National Survey of College Graduates, we study how college quality influences the effects of graduating into worse economic conditions in the context of the Great Recession. We find that earnings losses are concentrated among graduates from relatively high-quality colleges. Key mechanisms include substitution out of the labor force and into graduate school, decreased graduate degree completion, and differences in the economic stability of fields of study between graduates of high- and low-quality colleges.