Event Re-Cap: Midterm Evaluations of Teaching, Challenges and Opportunities

The first presentation was from Dr. Janet Giltrow, Senior Associate Dean for the Faculty of Arts, a Faculty that Dr. Giltrow described as “an enthusiastic supporter from the start” of the MEoT project. Dr. Giltrow gave an introduction to the topic of MEoTs by situating them within a broader, conceptual framework of teaching and learning. With an academic background in Genre Theory, Dr. Giltrow described how she is interested in teaching students to think and speak confidently in in the discursive framework of different genres. Generally, work produced as part of a genre is produced for an audience—the work of one person becomes subject to critical response by the many. Dr. Giltrow proposed viewing the dialogue that takes place within the classroom as a genre, and with this in mind asked: what are the patterns and flows of critical discourse about the classroom genre?

Her response to this question: traditionally, within the classroom, there is a “single track rhetorical trajectory,” with little sense of critical exchange unfolding. Often, one person—the instructor—does most of the talking, while the many—the students—are silent until it comes time to write an exam, when the simply say back what has already been said. This lack of critical exchange is traditionally echoed in the genres of the term paper and the end of term Student Evaluation of Teaching: in the former, the instructor responds to the work of the students, but this response is rarely heard or discussed again; in the latter, the students get a chance to critically evaluate their instructors, but the manner in which students can respond is limited largely to numerical ratings, which are accumulated into averages and receive minimal critical follow-up.

Dr. Giltrow then turned finally to the genre of MEoTs: MEoTs, she argued, are a rhetorical situation in sharp contrast to the genres of traditional classroom discourse. Because MEoTs allow instructors to collect feedback and to have follow-up discussions with their students, they provide an opportunity for interactive, ongoing dialogue that makes teaching and learning themselves into objects of critical inquiry. By allowing students this opportunity to engage critically in a more organic way with less institutionally mandated structures, MEoTs aid students in their maturation into responsible, contributing members of the community.

Dr. Giltrow closed by saying that, for a genre theorist like herself, the MEoT is a very promising genre indeed.

The second presentation was given by Dr. Simon Bates, CTLT’s Academic Director and an instructor in UBC’s Physics department. Dr. Bates began by surveying the background of the CTLT/AMS MEoT project and the work that has been done over the last couple of years.

Dr. Bates knew all about MEoTs from his previous position at a university in the UK, and these immediately came to mind as a solution to a problem identified by UBC’s Student Evaluations of Teaching (SEoT) committee—that student response rates to end of term SEoTs have been declining. Research has demonstrated that participating in MEoTs helps students to SEoTs more seriously, and Dr. Bates highlighted several of their obvious benefits that may contribute to this: they are a mechanism for reducing distance between instructors and their students, and they allow instructors a chance to explain the pedagogical rationale behind classroom practices, which often remain hidden from students.

The presentation then turned to a discussion of the pilot MEoT project that was conducted in 2012/2013, about which a report was subsequently published. The pilot included follow-up surveys sent to participating instructors and students, to allow them to provide feedback on their experiences. Most respondents indicated that MEoT results prompted fruitful in-class discussion, which, echoing what Dr. Giltrow had said earlier, shows that MEoTs can be a significant departure from usual, unilateral frameworks of classroom communication. Most instructors also said that students provided thoughtful, useful feedback, as well as that, though most feedback received was not surprising to them, the most important aspect was simply giving students an opportunity to voice this feedback.

Dr. Bates then briefly highlighted how CTLT and the AMS have been developing and refining resources for instructors on conducting MEoTs. Throughout this process, it has been important to emphasize that implementation of MEoTs is flexible, without one, mandated structure. Instead, the goal has been to identify key principles that should underlie all MEoTs, while leaving the particulars of the process to each instructor’s unique needs. Once again, Dr. Bates reiterated that the most important of these principles is that MEoTs should be a tool for sparking two-way dialogue.

Dr. Bates then turned to describing some of his own revelatory experiences as an instructor who has been incorporating MEoTs into his courses for years. At UBC, Dr. Bates teaches a first-year, introductory Physics course of around 250 students. In the past, he had always administered MEoTs on paper. However, noticing that students often use mobile devices in class (both for teaching and learning purposes and as distractions), he began to wonder if he could leverage students’ comfort with these devices. He thus tried using class time to conduct MEoTs on Typeform, a customizable, mobile-friendly survey tool. The results of this experiment were eye-opening: Dr. Bates received over a 90% response rate to his survey, and, with three minutes to respond, students provided around 8,000 words of feedback—approximately two to three times the amount they had provided in previous, paper-based evaluations. Not only was this a staggering amount of feedback, it proved to be a very rich set of data—and, unlike with paper-based evaluations, the task of sorting and tabulating it did not prove onerous.

The last presenter was Dr. Michael Griffin, a professor in UBC’s departments of Philosophy and Classical, Near Eastern, and Religious Studies. Dr. Griffin shared his experiences conducting MEoTs in two of his cross-listed Philosophy and Classical Studies courses, both of which are lecture courses that he tries as much as possible to make interactive. For MEoTs in these classes, Dr. Griffin copied a few questions from CTLT’s Formbuilder tool into an online survey platform, and asked students to take the survey outside of class time. As he had recently tried to incorporate more student participation into the course, he asked his students about pacing and balance between lecture and discussion, as well as about the usefulness of the on course website.

Dr. Griffin described how, after conducting a MEoT in his first course, he used the results to slightly tweak the structure of his second course. As part of his presentation, he displayed charts with results from each evaluation. These showed evidence that MEoTs have positively impacted his students’ experiences of these courses: after making changes to his second course in response to MEoT feedback in the first, he conducted a MEoT with the same questions in the latter, and results were even more positive. To further the point that MEoTs can have an observable positive impact on students’ perceptions of a course, Dr. Griffin also displayed some of the open-ended comments he received on his end of term Student Evaluations of Teaching. Several comments he received actually referenced changes that had been made to the course because of MEoT results, with students indicating they were appreciative of these.

Finally, Dr. Griffin posed a few questions to those present, soliciting suggestions for how to improve his use of MEoTs in the future. One of these was regarding response rates—while the overall MEoT process was a very beneficial one for Dr. Griffin’s classes, only about one third of his students responded. Dr. Griffin himself suggested that perhaps response rates could be improved by setting aside a short time in class for students to fill out evaluations, expressing interest in using a format like Typeform as described by Dr. Bates. The sentiment was echoed both by Dr. Bates, who said that using class time illustrates to students that instructors value student feedback; Dr. Giltrow agreed, saying that using class time highlights the process’s rhetorical significance by making students and instructors part of the same “shared consciousness” about a course.

Overall, those present seemed to agree that the name of the project, Midterm Evaluations of Teaching, is misleading in a way that could be detrimental to both instructor and student buy-in. This is because MEoTs, as has been emphasized in messaging about the project, do not need to be restricted solely to evaluating teaching. Rather, based on instructor needs, MEoTs can also be used to evaluate things like course content and classroom environment, as well as to ask students to evaluate their own and their peers’ participation in a course. Labelling this tool as an “evaluation of teaching” may imply a much more restricted use; it could increase instructor resistance by making the evaluations seem much more confrontational or critical, and could increase student resistance by sounding too much like the mandated end of term evaluations.

Several suggestions for improving this name were brainstormed. These included the following:

• Remove the word “midterm” to avoid association with midterm examinations; consider replacing with something like “mid-semester.”
• Remove the word “teaching” to ensure the name is not misinterpreted under a narrow scope; or, adapt to “teaching and learning” to ensure the broad scope is captured.
• Remove the word “evaluation” to avoid sounding confrontational, and to avoid association with end of term evaluations; consider replacing with something like “feedback”

On the subject of instructor buy-in, the group considered many challenges having to do with instructors’ perceptions of MEoTs—either that they are not valuable tools, or that the entire amount of time needed for creating an evaluation, soliciting feedback, sorting feedback, and making changes to a course is prohibitive. In general, the response to these types of challenges was that there should be clear and straightforward resources and guidelines provided to instructor participants. These should explain the integrity and utility of MEoTs, as well as resources available for instructors who desire more guidance. Several discussion groups also said that it is important to have faculty members within each Faculty and department that can act as champions for the project and who can be formal or informal sources for info about it.

It was also pointed out that some instructors who might otherwise like to conduct MEoTs face issues of trepidation based on the perception that soliciting feedback about a course is frightening or makes instructors vulnerable; or, because they have conducted such evaluations in the past and have received negative or concerning comments to which they did not know how to react. The first of these concerns highlights the need to make sure that messaging to instructors explains that evaluations do not need to be restricted to unilateral evaluations of an instructor’s teaching, but can be a robust way to solicit feedback on all aspects of a teaching and learning environment. Regarding the latter concern, it was noted that instructor resources should include suggestions for how to handle situations like these—for example, instructions on how to seek out mentorship from a colleague or educational developer.

The group also identified challenges to achieving enthusiastic student buy-in. Given that students are sometimes used to being passive consumers of their education, it was noted that they may not understand why they, as students, are asked to give feedback to instructors, who are meant to be the professional educators. Another challenge raised is that students at UBC may suffer from “survey fatigue”—they are often asked to provide feedback through a variety of different surveys, but may feel that they haven’t seen tangible change based on the feedback they have provided; if students feel frustrated that their voices are never being heard, they may not be enthusiastic about participating in yet another call for feedback.

Various mechanisms were suggested for ensuring that students understand the merits of participating in MEoTs, as well as understand that their feedback on MEoTs can have a real impact on their courses. Above all, two were stressed: before conducting MEoTs, instructors should make sure to lay out their purpose and expectations clearly; and, when conducting follow-up discussion about MEoT results, instructors should ensure that they clearly explain what they intend to do with these results. Echoing Dr. Bates presentation, groups also agreed that student participation can be increased by administering MEoTs during class time, online, and in a way that students are comfortable with—such as on a mobile-friendly platform.