Message sent to all Spring 2021 Instructional Staff on January 18, 2021.
Dear Faculty and Instructional Staff,
In 2017, Lehigh moved to online administration of course evaluations. National data as well as Lehigh data indicate that online course evaluations often suffer from low response rates. Faculty have raised concerns about a number of important implications of low response rates, including having too few responses to report and skewing the results in ways that are not always predictable. The good news is that there is growing empirical support for a set of practices to increase participation rates for online course evaluations.
None of this is to say that course evaluations are the only or even the most important means of evaluating the quality of instruction, and higher response rates do not negate other issues such as biases that can shape evaluations. General guidance on the interpretation of course evaluations can be found here: https://oirsa.lehigh.edu/sites/oirsa.lehigh.edu/files/Guidance%20for%20i...
More information on how different biases operate in course evaluations can be found here: https://advance.cc.lehigh.edu/research-biases-faculty-evaluation
https://www.cambridge.org/core/journals/ps-political-science-and-politic...
Course evaluations are but one means, and an imperfect means of evaluating the quality of instruction. You should consider adopting or asking your department to adopt other methods for demonstrating excellence in teaching. In the meantime, the attached document offers suggestions for how to improve student response rates and thus the utility of course evaluations for you.
Getting the Most Out of Online Course Evaluations
There are steps supported by empirical research that you can take to get the most out of online course evaluations and to increase participation rates. Because there is mixed support at best for “incentive” style methods for increasing response rates, we do not recommend those. Some actions that we do recommend that you can try before the semester, during the semester, and at the time of administering evaluations are below.
Actions you can take before or early in the semester include:
- Design custom questions that will be useful to you and that students will see as relevant to the course. There is a process to implement those questions in a given semester. You will receive an email from OIRSA approximately 4-6 weeks prior to the end of each semester articulating the timeline and process for adding custom questions through their office.
- Include on the syllabus and/or discuss with students early in the semester the importance of their feedback to you. Tell them how you have used course feedback in the past and assure them you will carefully consider their feedback.
- Explain why response rates for course evaluations matter.
Actions that you can take during the semester include:
- Seek anonymous informal formative feedback from students at various points along the way. Discuss the feedback with your class and act on the feedback where appropriate.
- Demonstrate to students that you are interested in improving the classroom when something does not go as well as you had hoped.
At the time of course evaluations:
- Remind students of how biases can find their way into course evaluations. Research has found that reminding others that biases exists can act to blunt its effect.
- Give the students some in-class time (~15 minutes) to complete the evaluations. Alert students ahead of time that they should have a laptop, tablet, or smartphone with them to complete the evaluation. This works best at the start of the class period.
- If you cannot give students in-class time, send reminders to them as a class. You can monitor the response rate through the tracker in the Evaluation Kit.
- Incorporate some of the recommendations here (e.g. how you will use the feedback, why response rate matters).
- Reiterate how you will use their feedback. “Future students will thank you!”
- If you have done mid-semester evaluations and made adjustments from the feedback, remind them that you have done so.
- Encourage responses to open-ended questions.
- Remind them that the data from course evaluations are anonymous. You will not receive the results until well after grades are entered and you will not know who wrote what.
Nathan Urban, Provost and Senior Vice President for Academic Affairs
Jackie Krasas, Deputy Provost for Faculty Affairs
Yenny Anderson, Vice Provost for Institutional Research & Strategic Analytics
Greg Reihman, Vice Provost for Library and Technology Services