Toward the end of every semester, students at the University of Indianapolis find themselves filling out course evaluations either on paper or online. Sometimes they are written by the professor; other times they are the standard IDEA form issued by the IDEA Center in Kansas. Students may not think that their feedback is valuable, but Dean of the College of Arts and Sciences Jennifer Drake said student evaluations are one of the measures looked at when evaluating faculty members annually, that the evaluations go into a faculty member’s promotion and tenure dossier to be reviewed by the promotion and tenure committee and that the evaluations can positively or negatively impact a faculty member’s salary.
In addition to course evaluations, UIndy looks at other data points when evaluating professors, such as quality of syllabi, activities used in the course, student learning outcomes and peer evaluations from other faculty, Drake said.
Faculty are required to administer evaluations for a certain number of courses each year, depending on the level of their position and which program they are in, Drake said. Faculty can choose to administer their evaluations online or on paper. They are supposed to fill out a master form, noting the primary learning objectives for that course, so what is being evaluated is clear. Then, faculty members are supposed to hand the forms out or make them accessible to students and leave the classroom. When students have completed their evaluations, someone other than the faculty member takes the forms to the proper person, depending on the department. This process ensures that the students’ comments are anonymous and the faculty member does not change or compromise the data, Drake said. If a faculty member does stay in the classroom or turns in the forms himself or herself, Drake said that students should report that to the department chair, so the chair is aware of the possible compromised data and break in procedure and can decide how to handle it.
After the forms are collected, they are sent to the IDEA Center to be processed. The center returns results for each faculty member at UIndy, who is able to see how he or she ranked in relation to other professors in the IDEA Center’s database, whether he or she is in the top 10 percent, next highest 20 percent, average, lower 20 percent or lowest 10 percent of faculty. In addition, the faculty member sees how he or she ranks compared to other professors in the same discipline, Drake said.
“Sometimes there are interesting patterns,” Drake said. “For example, math faculty often have lower IDEA scores if they’re teaching 100-level general education math courses that students don’t like, whereas faculty who are teaching 400-level electives in the major, students are much more likely to like those courses. You can see really skewed data based not on the ability of the faculty member but on the type of course it is. So that’s important and we can take all of that into account.”
Faculty members also learn how their quality of their teaching and their course rank on a scale of one to five. Drake said that sometimes those numbers are different, but other times they match up. She said that most faculty members review their results from the IDEA Center and the students’ comments carefully, and these provide an open door for conversation with department chairs and the deans of the various schools on campus.
“It’s an opportunity for conversation. Sometimes it’s, ‘Wow, students are loving what you’re doing, Tell me about it. I’d love to hear,’” Drake said. “If someone is struggling . . . it does give you an opportunity for conversation. Sometimes somebody is just an inexperienced teacher. They’re a new hire; they didn’t have a lot of experience teaching in grad school; it’s their first year, and they’re figuring it out. Sometimes something else is going on, maybe in that person’s personal life, and it gives you an opportunity to mentor and support faculty who are struggling.”
Drake said that consistently low evaluations are rare at UIndy. Deans and department chairs are not worried about faculty who simply have low course evaluations in one class or during one semester. Consistently low course evaluations with no improvement have, on occasion, resulted in an adjunct faculty member being let go, according to Drake, but it is rare. Most of the time, department chairs and deans work with professors to create a plan to improve scores and teaching ability.
“If we see scores consistently below that middle range in those two lower categories [of the IDEA Center results], then we’re concerned, and that’s when . . . we have a conversation, you know: ‘Let’s talk about why you’re not doing well in your classes. Do we have other data points that say you’re not doing well? Are your students actually learning, even though they’re rating you low? Are there things you can do to help them continue to learn but rate you less low?’ [If] mentoring is needed and if improvement doesn’t happen, there are a variety of choices available, depending on what kind of faculty line someone occupies,” Drake said. “But it’s been my policy that you don’t just pull the trigger if someone gets a bad evaluation or even a bad semester. You always come in and provide support and training.”
The reasons why consistently low evaluations are rare is because faculty work to improve and the way UIndy as a teaching school hires its faculty, according to Drake.
“I will say that, in part, it’s how we hire. When we’re hiring, we’re a teaching-focused institution, so [during] the hiring process . . . we talk a lot about teaching, and we share that teaching is our primary value because we’re a teaching-intensive institution,” Drake said. “So we tend to attract faculty members who are excited about teaching, who come here because they want to teach and develop ongoing relationships with students and serve as mentors. If someone wants to do only research, they probably wouldn’t come here or see us as a fit. We attract faculty who are committed teachers, so that’s why we don’t see a lot of problems.”
Drake said that UIndy is currently reevaluating its course evaluation process. A faculty subcommittee has been formed to look over the process and the forms and decide what to do next. According to Associate Professor of History and Political Science Doug Woodwell, who is on the subcommittee, the reevaluation is long overdue. UIndy switched to the IDEA form from faculty written evaluations shortly before he started working at the university about 12 years ago. The idea was to use it until faculty could figure out a better method.
The subcommittee was formed in November and began meeting in December of 2016. The overall goal of the subcommittee is to decide whether or not to continue using the IDEA form or move to a different method of evaluation and then identify that method. The subcommittee will get faculty approval before moving forward with any changes, Woodwell said. According to Woodwell, that much of the subcommittee’s time is currently spent trying to identify the goal of the evaluations.
“First of all, this is why it’s such a big project: What is the goal in the first place? Is it to best reflect teaching effectiveness?” Woodwell asked. “In order to figure out if you want to move to something else, you have to figure out what your goals are. Is it [this evaluation method] more important as an instrument by which we get assessed in terms of our pay raises, our promotion and tenure, and what role does it play? So that’s something totally different than a tool for self-assessment, for becoming a better teacher. Other goals that are important [include] getting students to care about the material and their field. I mean, that’s important. If that’s your goal, maybe you’re looking for something different.”
According to both Drake and Woodwell, a lot of the problems with the IDEA form stem from what it measures, how the numbers are crunched and how the data are interpreted overall. Woodwell said that one of his problems with the form is that it measures the popularity of the professor more than teaching effectiveness. He said that professors have actually altered their courses in order to keep student evaluations higher, sacrificing student learning outcomes in order to get the good marks that affect their pay and tenure. Along those same lines, Drake said that she worries about course evaluations used as a way to punish professors and that she has seen that happen.
The subcommittee did a survey of faculty to see what faculty members thought of the IDEA form. According to Woodwell, approval was low.
“We did a survey of over 100 people; we got over 100 faculty responses. If you look at percent approval of how IDEA [forms] work, it’s only about 33 percent. There’s a middle ground where it’s like that neither approve nor disapprove . . . but actual just approve or strongly approve of what’s going on and how it affects their teaching, it’s only about one third,” Woodwell said. “In terms of usefulness, the quantitative sections, with the bubbles—one third. Two thirds of professors say that the free response from students has been useful. So what you would find from me and other people is that the free response from the back, that nobody probably fills out, is actually what we tend to regard as the most useful feedback.”
Overall, no matter the form, Drake said that it is important for students to take their time when filling out course evaluations.
“It’s an important moment, where you have voice as a student…. It is high stakes, and other folks are looking at those besides your faculty member. And you can have a positive or negative impact on peoples’ careers. It’s pretty significant,” Drake said. “Thoughtfulness is really important [when filling out evaluations]. This is not just a throwaway thing. Your faculty really do want to know how to improve. These forms are important data points in faculty evaluation.”
