Search EdWorkingPapers

Search for EdWorkingPapers here by author, title, or keywords.

Susanna Loeb

Carly D. Robinson, Biraj Bisht, Susanna Loeb.

Billions of dollars are invested in opt-in, educational resources to accelerate students’ learning. Although advertised to support struggling, marginalized students, there is no guarantee these students will opt in. We report results from a school system’s implementation of on-demand tutoring. The take up was low. At baseline, only 19% of students ever accessed the platform, and struggling students were far less likely to opt in than their more engaged and higher achieving peers. We conducted a randomized controlled trial (N=4,763) testing behaviorally-informed approaches to increase take-up. Communications to parents and students together increase the likelihood students access tutoring by 46%, which led to a four-percentage point decrease in course failures. Nonetheless, take-up remained low, showing concerns that opt-in resources can increase—instead of reduce—inequality are valid. Without targeted investments, opt-in educational resources are unlikely to reach many students who could benefit.

More →

Sara White, Leiah Groom-Thomas, Susanna Loeb.

This study synthesizes existing research on the implementation of tutoring programs which we define as one-to-one or small-group instruction in which a human tutor supports students grades K-12 in an academic subject area. Tutoring has emerged as an especially promising strategy for supporting students’ academic success with strong causal evidence finding large, positive effects on students' math and reading test scores across grade levels. Prior studies have reviewed this causal evidence of effects, but none have summarized the evidence on implementation. We iteratively developed search and selection criteria to identify studies addressing key research questions and synthesized these 40 studies which employ a range of research methodologies to describe how tutoring is implemented and experienced. We find that existing research provides rich descriptions of tutoring implementation within specific programs of focus, with most studies describing after-school tutoring and small-scale programs run by university professors. While few elements of implementation are studied in depth across multiple studies, common patterns emerge. Tutoring program launch is often facilitated by strategic relationships between schools and external tutoring providers and strengthened by transparent assessments of program quality and effectiveness. Successful tutoring implementation often hinges on the support of key school leaders with the power to direct the use of school funding, space, and time. Tutoring setting and schedule, tutor recruitment and training, and curriculum identification influence whether students are able to access tutoring services and the quality of the instruction provided. Ultimately, the evidence points to strong tutoring being driven by positive student-tutor relationships through which tutors provide instruction strategically targeted for students’ strengths and needs driving towards a long-term academic goal.

More →

Christopher Doss, Hans Fricke, Susanna Loeb, Justin B. Doromal.

This study assesses the effects of two text messaging programs for parents that aim to support the development of math skills in prekindergarten students. One program focuses purely on math, while the other takes an identical approach but focuses on a combination of math, literacy, and social-emotional skills. We find no evidence that the math-only program benefits children’s math development. However, the combination program shows greater promise, particularly for girls. Quantile regressions indicate that the effects are concentrated in the lower half of the outcome distribution. We discuss and provide evidence for various hypotheses that could explain these differences.

Highlights:

  • We test the effects of two text messaging curricula that leverage behavioral economics principles to help parents support the math development of prekindergarteners in the home.
  • We find that a program that cycles through literacy, mathematics, and social-emotional skills increases math achievement for girls, while a program focusing solely on mathematics has no effects.
  • Benefits for girls are concentrated on those with weaker performance on mathematics assessments.
  • We posit potential mechanisms based on the literature.

More →

Leiah Groom-Thomas, Monica Lee, Cate Smith Todd, Kathleen Lynch, Susanna Loeb, Scott McConnell, Lydia Carlis.

Many preschool agencies nationwide continue to experience closures and/or conversions to virtual or hybrid instruction due to the ongoing COVID-19 pandemic. Despite the importance of understanding young children’s learning and development during the COVID emergency, limited knowledge exists on adaptable practices of assessing young children during the pandemic. We detail practices used to assess learning in 336 Head Start children across four states during three different time periods in the 2020-21 school year, using adaptation of traditionally in-person assessments of early numeracy, early literacy, and executive functioning. In doing so, we distill early lessons for the field from the application of a novel, virtual assessment method with the early childhood population. The paper describes adaptations of assessment administration for virtual implementation and incorporation of feedback into continued virtual delivery of assessments. Applications and limitations in broader contexts are discussed.

More →

Kathleen Lynch, Monica Lee, Susanna Loeb.

The COVID-19 pandemic’s impact on preschool children’s school readiness skills remains understudied. This research investigates whether exposure to in-person (versus virtual) Head Start preschool predicted children’s early numeracy, literacy, and executive function outcomes during a pandemic-affected school year, using a novel virtual assessment methodology. Study children (N = 336; mean age = 51 months; 46% Hispanic; 36% Black Non-Hispanic; 52% female) experienced low in-person preschool exposure compared to national pre-pandemic norms. However, study children experienced gains during the pandemic-affected year of 0.08 SD in executive function, 0.34 SD in print knowledge, and 0.49-0.75 in early numeracy skills. For two of the three early numeracy domains measured, spring test score outcomes were stronger among children who attended more in-person preschool.

More →

Biraj Bisht, Zachary LeClair, Susanna Loeb, Min Sun.

Paraeducators perform multiple roles in U.S. classrooms, including among others preparing classroom activities, working with students individually and in small groups, supporting individualized programming for students with disabilities, managing classroom behavior, and engaging with parents and communities. Yet, little research provides insights into this key group of educators. This study combines an analysis of national administrative data to describe the paraeducator labor market with a systematic review of collective bargaining agreements and other job-defining documents in ten case-study districts. We find a large and expanding labor market of paraeducators, far more diverse along ethnic and racial lines than certified teachers but with far lower wages, fewer performance incentives, less professional development, and fewer opportunities for advancement within the profession.

More →

Carly D. Robinson, Susanna Loeb.

Research consistently demonstrates that tutoring interventions have substantial positive effects on student learning. As a result, tutoring has emerged as a promising strategy for addressing COVID-related learning loss and affording greater educational opportunities for students living in poverty. The effectiveness of tutoring programs, however, varies greatly, and these variations may drive differential gains in student learning. Therefore, determining the program characteristics that do and do not drive positive student outcomes will be key to providing guidance for policymakers and practitioners who want to implement high-impact tutoring at scale. Our goal is to highlight the programs, characteristics, and conditions that evidence suggests make for effective tutoring and to create an evidence-based framework for delivering and evaluating tutoring interventions. In addition, we identify promising questions for future research.

More →

Heather C. Hill, Zid Mancenido, Susanna Loeb.

Despite calls for more evidence regarding the effectiveness of teacher education practices, causal research in the field remains rare. One reason is that we lack designs and measurement approaches that appropriately meet the challenges of causal inference in the context of teacher education programs. This article provides a framework for how to fill this gap. We first outline the difficulties of doing causal research in teacher education. We then describe a set of replicable practices for developing measures of key teaching outcomes, and propose causal research designs suited to the needs of the field. Finally, we identify community-wide initiatives that are necessary to advance effectiveness research in teacher education at scale.

More →

Jing Liu, Susanna Loeb, Ying Shi.

Classroom teachers in the US are absent on average approximately six percent of a school year. Despite the prevalence of teacher absences, surprisingly little research has assessed the key source of replacement instruction: substitute teachers. Using detailed administrative and survey data from a large urban school district, we document the prevalence, predictors, and variation of substitute coverage across schools. Less advantaged schools systematically exhibit lower rates of substitute coverage compared with peer institutions. Observed school, teacher, and absence characteristics account for only part of this school variation. In contrast, substitute teachers’ preferences for specific schools, mainly driven by student behavior and support from teachers and school administrators, explain a sizable share of the unequal distribution of coverage rates above and beyond standard measures in administrative data.

More →

Cassandra M. D. Hart, Brian Jacob, Susanna Loeb.

A common rationale for offering online courses in K-12 schools is that they allow students to take courses not offered at their schools; however, there has been little research on how online courses are used to expand curricular options when operating at scale. We assess the extent to which students and schools use online courses for this purpose by analyzing statewide, student-course level data from high school students in Florida, which has the largest virtual sector in the nation. We introduce a “novel course” framework to address this question. We define a virtual course as “novel” if it is only available to a student virtually, not face-to-face through their own home high school. We find that 7% of high school students in 2013-14 enroll in novel online courses. Novel courses were more commonly used by higher-achieving students, in rural schools, and in schools with relatively few Advanced Placement/International Baccalaureate offerings.

More →