I thank my colleague Chris Lusk for bringing this paper by Uttl, White and Gonzalez to my attention.
Many universities and polytechnics acquire Student Evaluation data on courses and teacher quality at the end of a course. There are different ways this can be done – here at The University of Waikato students are asked (online) a series of questions about the course and teaching of the course to which they can ‘Agree strongly’, ‘Agree’, ‘Disagree’, or ‘Disagree strongly’. They can also provide free-response feedback on the course. It is the quality of these student evaluations that is the subject of the research of Uttl et al.
They present a large meta-analysis (i.e. a combined analysis of lots of peoples’ data) on the extent to which the scores that university students give to their teachers in end-of-semester evaluations relate to independent measures of teaching quality. In other words: “What can we learn about a teacher’s effectiveness from the scores their students give them?” This paper comes up with a simple answer: Nothing. There is no relationship between other measures of quality of teaching and how students evaluate the quality of teaching.
Many universities, including The University of Waikato where I teach and whose computer, office and time I am using to write this post, use Student Evaluations to inform decision making regarding teaching (e.g. which staff member teachers which paper) and also staff advancement (promotion). To be fair, when it comes to using evaluation of teaching for promotion, the Student Evaluation scores are not the only evidence used (every teacher can include whatever evidence they deem appropriate) but they are used. This meta-analysis says, unequivocally, that they are worthless for assessing how effective a teacher is.
The paper closes with a clear statement that universities which value student learning should abandon use of student evaluation scores, and a (presumably tongue-in-cheek) statement that universities that value students’ perceptions might wish to increase their use and fire staff who don’t meet the highest possible rating.
Food for thought for us teachers…
Uttl, B., White, C. A., & Gonzalez, D. W. (2016). Meta-analysis of faculty’s teaching effectiveness: Student evaluation of teacher ratings and student learning are not related. Studies in Educational Evaluation 54, 22-42.