Search EdWorkingPapers

Search for EdWorkingPapers here by author, title, or keywords.

Educator preparation, professional development, performance and evaluation

Jessalynn James, Matthew A. Kraft, John Papay.

We explore the dynamics of competitive search in the K-12 public education sector. Using data from Boston Public Schools, we document how teacher labor supply varies substantially by position types, schools, and the timing of job postings. We find that early-posted positions are more likely to be filled and end up securing new hires that are better-qualified, more-effective, and more likely to remain at a school. In contrast, the number of applicants to a position is largely unassociated with hire quality, suggesting that schools may struggle to identify and select the best candidates even when there is a large pool of qualified applicants. Our findings point to substantial unrealized potential for improving teacher hiring.

More →


Kathryn Gonzalez, Kathleen Lynch, Heather C. Hill.

Despite growing evidence that classroom interventions in science, technology, engineering, and mathematics (STEM) can increase student achievement, there is little evidence regarding how these interventions affect teachers themselves and whether these changes predict student learning. We present results from a meta-analysis of 37 experimental studies of preK-12 STEM professional learning and curricular interventions, seeking to understand how STEM classroom interventions affect teacher knowledge and classroom instruction, and how these impacts relate to intervention impacts on student achievement. Compared with control group teachers, teachers who participated in STEM classroom interventions experienced improvements in content and pedagogical content knowledge and classroom instruction, with a pooled average impact estimate of +0.56 standard deviations. Programs with larger impacts on teacher practice yielded larger effects on student achievement, on average. Findings highlight the positive effects of STEM instructional interventions on teachers, and shed light on potential teacher-level mechanisms via which these programs influence student learning.

More →


Vladimir Kogan, Brandon Genetin, Joyce Chen, Alan Kalish.

Student surveys are widely used to evaluate university teaching and increasingly adopted at the K-12 level, although there remains considerable debate about what they measure. Much disagreement focuses on the well-documented correlation between student grades and their evaluations of instructors. Using individual-level data from 19,000 evaluations of 700 course sections at a flagship public university, we leverage both within-course and within-student variation to rule out popular explanations for this correlation. Specifically, we show that the relationship cannot be explained by instructional quality, workload, grading stringency, or student sorting into courses. Instead, student grade satisfaction -- regardless of the underlying cause of the grades -- appears to be an important driver of course evaluations. We also present results from a randomized intervention with potential to reduce the magnitude of the association by reminding students to focus on relevant teaching and learning considerations and by increasing the salience of the stakes attached to evaluations for instructor careers. However, these prove ineffective in muting the relationship between grades and student scores.
 

More →


Derek Rury.

To study beliefs about ability and STEM major choice, I conduct a field experiment where I provide students with information that they are above average in their top fields of study. I find that STEM students are more likely to switch out of their major and that non-STEM students fail to switch into STEM at the same rates as other fields. I also find that learning you are above average in your top field of study increases STEM major choice by almost a third, as STEM students appear more like to persist and non-STEM students increase their switching into STEM fields.

More →


Aaron Phipps.

Using administrative data from D.C. Public Schools, I use exogenous variation in the presence and intensity of teacher monitoring to show it significantly improves student test scores and reduces suspensions. Uniquely, my setting allows me to separately identify the effect of pre-evaluation monitoring from post-evaluation feedback. Monitoring's effect is strongest among teachers with a large incentive to increase student test scores. As tests approach, unmonitored teachers sacrifice higher-level learning, classroom management, and student engagement, even though these pedagogical tasks are among the most effective. One possible explanation is teachers ``teach to the test'' as a risk mitigation strategy, even if it is less effective on average. This is supported by showing teaching to the test has a smaller effect on student test score variance than other teaching approaches. These results illustrate the importance of monitoring in contexts where teachers have the strongest incentive to deviate from pedagogically sound practices.

More →


Sam Sims, Harry Fletcher-Wood, Alison O’Mara-Eves, Sarah Cottingham, Claire Stansfield, Josh Goodrich, Jo Van Herwegen, Jake Anders.

Multiple meta-analyses have now documented small positive effects of teacher professional development (PD) on pupil test scores. However, the field lacks any validated explanatory account of what differentiates more from less effective in-service training. As a result, researchers have little in the way of advice for those tasked with designing or commissioning better PD. We set out to remedy this by developing a new theory of effective PD based on combinations of causally active components targeted at developing teachers’ insights, goals, techniques, and practice. We test two important implications of the theory using a systematic review and meta-analysis of 104 randomized controlled trials, finding qualified support for our framework. While further research is required to test and refine the theory, we argue that it presents an important step forward in being able to offer actionable advice to those responsible for improving teacher PD.

More →


Meghan Comstock, Kenneth A. Shores, Camila Polanco, Erica Litke, Kirsten Lee Hill, Laura M. Desimone.

As states and districts expand their goals for equitable mathematics instruction to focus on cultural responsiveness and rigor, it is critical to understand how teachers integrate multiple teaching approaches. Drawing on survey data from a larger study of professional learning, we use mixture modeling to identify seven unique ways that middle school mathematics teachers integrate ambitious, traditional, and culturally responsive (CR) mathematics instruction. The resulting typology is driven almost exclusively by variation in CR teaching. About half of teachers reported rarely engaging in CR teaching. Teachers who emphasized CR teaching tended to be teachers of color and have high CR teaching self-efficacy. Findings suggest that tailoring teacher development to how teachers blend multiple approaches may best support equitable mathematics instruction.

More →


David Blazar, Doug McNamara, Genine Blue.

While teacher coaching is an attractive alternative to one-size-fits-all professional development, the need for a large number of highly skilled coaches raises potential challenges for scalability and sustainability. Collaborating with a national teacher training organization, our study uses administrative records to estimate the degree of heterogeneity in coach effectiveness at improving teachers’ instructional practice, and specific characteristics of coaches that explain these differences. We find substantial variability in effectiveness across individual coaches. The magnitude of the coach-level variation (0.2 to 0.35 standard deviations) is close to the full effect of coaching programs, as identified in other research. We also find that coach-teacher race/ethnicity-matching predicts changes in teacher practice, suggesting that the relational component of coaching is key to success.

More →


David Blazar.

There is broad consensus across academic disciplines that access to same-race/ethnicity teachers is a critical resource for supporting the educational experiences and outcomes of Black, Hispanic, and other students of color. While theoretical and qualitative lines of inquiry further describe a set of teacher mindsets and practices aligned to “culturally responsive teaching” as likely mechanisms for these effects, to date there is no causal evidence on this topic. In experimental data where upper-elementary teachers were randomly assigned to classes, I find large effects upwards of 0.45 standard deviations of teachers of color on the short- and longer-term social-emotional, academic, and behavioral outcomes of their students. These average effects are explained in part by teachers’ growth mindset beliefs that student intelligence is malleable rather than fixed, interpersonal relationships with students and families, time spent planning for and differentiating instruction for individual students’ needs, and the extent to which teachers lead well-organized classrooms in which student (mis)behavior is addressed productively without creating a negative classroom climate.

More →


Biraj Bisht, Zachary LeClair, Susanna Loeb, Min Sun.

Paraeducators perform multiple roles in U.S. classrooms, including among others preparing classroom activities, working with students individually and in small groups, supporting individualized programming for students with disabilities, managing classroom behavior, and engaging with parents and communities. Yet, little research provides insights into this key group of educators. This study combines an analysis of national administrative data to describe the paraeducator labor market with a systematic review of collective bargaining agreements and other job-defining documents in ten case-study districts. We find a large and expanding labor market of paraeducators, far more diverse along ethnic and racial lines than certified teachers but with far lower wages, fewer performance incentives, less professional development, and fewer opportunities for advancement within the profession.

More →