Evaluation Process

You are here

Course and Instructor data extraction and preparation

The course and instructor data for a given semester are extracted from Banner a few weeks after the start of classes.

In the Fall and Spring semesters, these data are prepared by OIRSA and reviewed by each department with the help of the department coordinators. The departments help identify anything that needs to be changed, such as changes in instructor, crosslistings not identified in Banner, short courses not yet in the system, etc. Departments help identify instructors who are in a coordinator role for a course and should not be evaluated for it, as well as courses that are not suited for end-of-term evaluations due to their nature or structure (e.g. dissertations, private music lessons, thesis, research, independent study, fieldwork, etc.). Note that since the Fall 2018 semester, courses with fewer than 5 students enrolled are automatically excluded from the evaluation process.

TA and Practicum Evaluation Forms

Graduate students serving as teaching assistants can also be evaluated using a separate evaluation form specifically for TAs (see Core Questions section). Department coordinators are encouraged to include TAs in the information they send OIRSA if the TA evaluation form is relevant to the TA responsibilities. If TAs believe that their duties are relevant to the TA evaluation form but they are uncertain whether they will be included in the course evaluation process, they should reach out to their department coordinator by the middle of March in the Spring Semester or the middle of October in the Fall Semester to ensure that they are included in the information used by OIRSA to administer the evaluations.

Since Spring 2019, Practicum courses are also evaluated using a different evaluation form (see Core Questions section). The departments specifically identify these courses to OIRSA to make sure the proper form is used.

Student data extraction

The data for all students enrolled in courses marked for evaluation (who will receive an evaluation form) are extracted from Banner towards the end of the semester, after the last day for students to drop courses with a “W”.

Evaluation window

In the Fall and Spring semesters, course evaluations are administered during the last week of classes, opening on Saturday at 12:00AM and closing on Friday at 11:59PM. However, instructors have the option of choosing one of three 72-hour window options during that week, if they prefer a short evaluation window.  To do so, instructors have to send their window preference to OIRSA before a term-specific date, usually about a month before the start of the evaluations.

For courses that do not follow the regular semester schedule, such as is the case for a number of summer courses, course evaluations are administered during the last few days of class. The length of the evaluation window is dependent on the length of the course.

All evaluation windows close before final exams.

Evaluation questions

All course evaluations use a set of core questions: https://oirsa.lehigh.edu/core-questions. As discussed above, TAs and instructors of Practicum courses are evaluated using a slightly different set of core questions.

In the Fall and Spring semesters, instructors are able to request custom questions in addition to the set of core questions being asked. The custom questions can be single-choice, multiple-choice, or open-ended text. These custom questions have to be sent to OIRSA before a term-specific date, usually about a month before the start of the evaluations.

Administering the evaluations

All the course, instructor and student data are uploaded into EvaluationKit, a third-party course evaluation software tool. The evaluation window is an attribute of the course and therefore is also provided to EvaluationKit.

For each course, at the start of the evaluation window, the tool generates an email to the students enrolled in this course letting them know that the course evaluation form is open, and inviting them to log in and complete the evaluation. When a student logs in, they see all the evaluation forms that are open at that moment, for the courses they are enrolled in.

The tool also sends reminders to non-completers 2, 4 and 6 days after the start of the evaluation window.

Student evaluations are captured in EvaluationKit and the system tracks who has and has not completed the evaluations. However, the individual responses are de-identified and confidential. These evaluations are then summarized at the course level, and are made available to instructors shortly after final grades are submitted.

 

Last updated: February 20, 2020