Instructional Quality Assessment Tool (Tekkumru-Kisa et al., 2020)
Category: Teacher and Leader Development
We present one of the first Generalizability studies of non-test measures of teaching effectiveness administered by practitioners in a middle-income country. The reliability of observations varies widely (from 0 to 0.75 on a 0-1 scale) and depends upon their context (whether they are conducted during training or on the job) and rater assignment configurations. The reliability of surveys varies substantially, coinciding with a change in which students were sampled across occasions. Our estimates are comparable to the reliability from research in highincome countries, but the variation within our estimates and between them and those from individual studies suggests that practitioners should conduct their own reliability analyses. We offer guidance on leveraging such analyses to improve the reliability of their measures.