I’ve had a query from a reader wanting to know about studies of bias in student evaluations. Looking through past posts, I’ve found this on evaluations of fat female lectures. But I know there’s a lot of other stuff out there. Please let us know about it, with respect to any groups likely to be negatively stigmatised.
13 thoughts on “Bias in student evaluations”
Comments are closed.
Feminist Philosophers 
Michael Huemer has a useful — if somewhat dated — critical review of student evaluations here:
http://home.sprynet.com/~owl1/sef.htm
no matter what your field, if you a fat woman your less intelegent, more likely to make mistakes, and more likely to take excessive time off from the job, add over 40 and your invisable as well. I hear this all the time, that along with, well if she was not stupid, why did she let herself get that way ?
A friend, very overweight 300lb plus, is smart, comes to work on time, is efficient, but is passed over for promotion time and time again, and no she does not deal with the public. Whats wrong with her, she’s fat.
yes that’s sarcasm folks, just making sure you got that bit.
I’m not in Philosophy, but I have vaguely heard about groups like hiring committees having diversity seminars where part of what they talk about is how to ignore bias in student evaluations. I’m not part of those groups so I don’t really know the details, but the impression I got was that the idea is you can’t stop students from writing implicitly sexist things, but you can train the people reading the evaluations to discount those types of comments. I sure hope they actually do a good job with that training.
Personally, I have gotten overtly sexist comments before, like dirty comments from guys, and critiques of my looks or clothing. But reading through the previous thread you mentioned made me realize that there’s another comment I often get that I hadn’t thought of as being sexist, but maybe it is – “she’s kind of a know-it-all”. Do guys ever get that comment? I mean, honestly, I’m TAing your class – do you want me to not know all the material?
If you search under “evaluations” there are a couple of other related posts. One is on the shocking fact that some students admit to lying on evaluation to reward or punish profs. Another is relevant to LGT profs.
To answer the question #4 wrote about guys getting the “know-it-all” comment: I’ve never gotten that sort of comment on student evaluations, or any variation of it that I can recall. But there’s a male version of it that I have seen and heard about from others. The comment goes something like this: “Professor Dude expects us to know as much as him about philosophy! We’re only beginners! etc.”. The student comments are generally not as well formed as that, but that’s the gist of it.
My guess is that it’s just a more deferential version of the student comments being reported by women – a version that presupposes additional male authority.
The very best scholars I know, including some really fantastic feminist and l/g/b/t scholars of education, continue to insist to me that evidence of bias in students’ evaluations of professors is mixed. They don’t say the evidence is INconclusive, but they do say it is not easy to interpret as decisive; generalizations are so hard when the metrics across institutions and years are not the same.
Having said all that, I adore the sentence from this recent article: “In fact, students’ ratings of
professors show little, if any, correlation with objective measures of what students learn”. From here: http://www.stjohns.edu/media/3/15809021162c4c7abc99c9b0134c8049.pdf
@profbigk: Funny, I’ve heard precisely the opposite – that the evidence is pretty clear. I haven’t read the papers, though; it looks like I’ll have to.
@um: I’m male, and I’ve never gotten “know it all” or anything similar, and if anyone would, it’d be me (not because I know it all, but because I’m a pedant even in everyday life…). This, to me, smacks of students who are on some level (even subconsciously) offended by the idea of a knowledgeable woman. Or, more precisely, a knowledgeable woman who has the authority to call them out on their lack of knowledge…
My female colleagues have spoken of evaluations with crude comments and even drawings on them. I’ve never received anything like that. It’s disgusting what these kids think they can get away with saying to female instructors.
I happen to have a bunch of data I scraped from ratemyprofessors.com a while ago sitting around in my database. To check Landon’s contention […] I graphed reported easiness versus reported quality, if students really do have a special hatred for women teachers that know their stuff you would expect the curves to be different (i.e. Women getting more of a penalty for being harder), but they are basically identical.
See http://i.imgur.com/yQlCU.png
[lightly edited in order to keep to our policies]
@6 – Do you think those are meant to be the same thing? To me, “know-it-all” reads as “talked down to us too much”, while “expected us to know as much as him” reads as “didn’t talk down to us enough so we didn’t understand what he was talking about”. But maybe they are both just supposed to mean “knew their stuff and made me feel stupid”. Anyway, I’m still curious if that general category of complaints are the kinds of things like people might feel about any professor, but might only complain about in writing about female professors….
@8 – I agree, “know-it-all” smacks of complaints about uppity women and stuff like that. I’ve only gotten a really crude comment once, but it made me really wonder if anyone else was reading these before they got handed back to me. I’d almost rather get crude comments – they hurt my feelings, but its a lot easier to be confident that anyone reading my evaluations would discount a crude comment as clearly sexist and not my fault, while more subtle complaints could be more of a problem.
Another time I got a complaint about my clothing not being professional enough, which I’m pretty sure came from a female student.
On the whole most of the complaints I get don’t seem sexist, but they do show that the students have no understanding of what is reasonable to expect from their TA. I’ve had students who complained that I wouldn’t stay after class to help them when I told them I had to teach another class right after, students who complain on my forms about things the professors did that I had no control over, etc. So even when they aren’t sexist, I think the complaints on those course evals have little connection to reality.
One problem is that usually the students get only between 5 and 10 minutes to write their comments. It might be better if they had a chance to think them through, and maybe submit them online at home, rather than at the end of the last class while they’re worried about finals.
When I filled out evals in England one year (graduate program) the evaluations said something like “evaluate your professor on the following questions, keeping in mind that you may be in their position in a few years”. I can’t remember the exact phrasing but it was something like that. Interesting approach (though of course makes more sense for grad students than undergrads).
@um I am also a female graduate student. I’ve never received a “know-it-all” comment, but I did have a student once write that I was “condescending” on an evaluation. I think this might amount to the same thing.
I’ve also encountered the problem of students not understanding what role the TA plays in the classroom. I’ve gotten my fair share of “the tests are too hard,” “the pace is too fast,” “homework instructions are unclear” in situations where I had no control over those issues.
I worry a lot about how these statements will be perceived on the job market. In particular the “condescending” comment worries me along with several comments from a different semester along the lines of “she never knew when things were scheduled or what was happening.” The lead professor for this class repeatedly told me one date for tests/upcoming assignments and the students a different one. I guess my student evaluations run the gambit of female stereotypes from “uppity” to “ditzy.”
“I worry a lot about how these statements will be perceived on the job market.”
The way things are done at my school, its unclear anyone in the job market will ever see them. The department has the right to read our evals, I think, but it seems like no one does. I do think they might see the evaluations we get from the professors, though.
So I’m curious, how is it done at your school? At what stage of the job market would you expect someone to see negative evals from your students?
Some schools assemble the teaching dossiers of their PhD candidates on the market, and some of those that do include every scantron, every comment. Schools that assemble teaching portfolios may do so quite differently from each other, and some pride themselves on a very complete package including every evaluation! Other schools just hand all the material over to the grad student and leave it up to them what to include in application files. I’ve talked many times to job candidates who worry about what to include with respect to evaluation materials in their job applications, and my suggestion is to take control of the contents of the dossier if you can. If you actually have quite a lot of irrelevant, nasty, odd comments in your evaluations, then don’t pass along the comments if you don’t have to.
TT-jobs are trickier; at St. Mary’s College of MD we were required to have all our classes evaluated every term, so everyone knew that the comment material was available, and generally expected to see it at reviews. But then the data set is so huge that, frankly, a few irrelevant/nasty/odd comments were considered par for the course and outliers worth ignoring. My concern with evaluations continues to be that the results should be considered part of a body of data and not, after one course, seen as a data set in itself. You know what I mean? I’m saying, scholars at every level from grad school to full professors have exclaimed over what it means that so-and-so has a particular outcome in a particular course, and I keep trying to encourage the alternative perception that it is not meaningful to look at the results of a single course with a handful of scantrons. The dossier must be considered as a whole: multiple courses, peer observation, student narrative evaluations, and self-reflections.