Dealing with gender/topic biases in teaching evaluations

A reader solicits practical strategies for facilitating the sensible institutional interpretation of student evaluations of teaching, given the empirically well-founded worry (as was noted on this blog recently) that such evaluations express a substantial bias against women instructors.

Hello wonderful community of feminist philosophers, I’m hoping that you can help me with a problem that is not just mine but is one that so many of us share. This is the problem of teaching evaluations. Teaching evaluations as a method of assessing teaching leave much to be desired. However, their use becomes even more problematic or worrisome when (as in my case) they are used as one of three main criteria for annual departmental evaluations and promotion.

There is good evidence to show that anonymous course/teaching evaluations are biased against women and a number of other underrepresented groups. Most recently, there is this study. But in addition to evaluations being generally biased against women, I’m facing the additional issue: namely, in all of my courses I include a good deal of feminist and critical race theory. Having recently read my course evaluations, I noticed that a good number of my students reacted negatively to this material. For example, there were many comments that spoke to the “problem” of so much feminist philosophy, about how I’m trying to “indoctrinate them,” and about how if they didn’t simply agree with my (feminist) positions then I would give them low grades. Of course, all of these claims are false but nonetheless I am worried about their presence. It seems that on the basis of the content of my courses (in addition to the gender bias), my evaluations are importantly lower than those of others (and for reasons that have nothing to do with my actual teaching abilities).

So I’m wondering whether and how people in other departments have dealt with this problem. I’m pretty certain that my institution (big, public university) is committed to keeping them, so abolition is not on the table at this point. Still, I wonder if there is any way to take into account these known biases so that certain groups of people are not systematically disadvantaged. Have any departments tried other methods of assessing teaching either instead of or in addition to the required ones? Even though my university probably isn’t going to stop using teaching evaluations any time soon, it is possible that my department might be persuaded to use a different method of assessing teaching when it comes to departmental annual merit reviews (or at the very least, supplementing the university required teaching evaluations with some other methods).

Looking forward to hearing your thoughts on this matter.

New Study: Gender Effect in Research Funding

In PNAS:

We examined the application and review materials of three calls (n = 2,823) of a prestigious grant for personal research funding in a na- tional full population of early career scientists awarded by the Nether- lands Organization for Scientific Research (NWO). Results showed evidence of gender bias in application evaluations and success rates, as well as in language use in instructions and evaluation sheets. Male applicants received significantly more competitive “quality of re- searcher” evaluations (but not “quality of proposal” evaluations) and had significantly higher application success rates than female ap- plicants. Gender disparities were most prevalent in scientific disciplines with the highest number of applications and with equal gender dis- tribution among the applicants (i.e., life sciences and social sciences). Moreover, content analyses of the instructional and evaluation mate- rials revealed the use of gendered language favoring male applicants. Overall, our data reveal a 4% “loss” of women during the grant re- view procedure, and illustrate the perpetuation of the funding gap, which contributes to the underrepresentation of women in academia.

Hypervisible Invisibility and Race

H/T TF

Post by Olivia Cole at HuffPo.

“Khadijah Costley White shared stories from her childhood in a piece for the Washington Post, in which she described the distress and pain of growing up black and female in school. She describes the punishment she received for her frustration, and it mirrors what I witnessed in my own schooling: black girls facing bias and neglect, who are then punished for their human response. What was interpreted as a cry for help from me was interpreted as “bad black girls” from my peers. It is in this way that black children are both hypervisible and invisible: in one way, the black girls at my school’s behavior was hypervisible; subject to heavy policing and punishment, gossiped about by the faculty. But in another way, these girls were entirely invisible: the causes for their behavior going unexplored and unconsidered, their cries for help unheard.”

“A white kid at the pool party — the one who recorded the incident in McKinney, in fact — said the following: “Everyone who was getting put on the ground was black, Mexican, Arabic. [The cop] didn’t even look at me. It was kind of like I was invisible.”

“This is what it means to be white in America. To be visible for the good and invisible for the bad. We are on every TV screen, every magazine cover for our achievements, but when we riot after a basketball game or a Pumpkin Festival, we’re slid quietly to the bottom of the deck, or gently sat by the side of the road and without cuffs after engaging in a shootout with police.”

There is a poem by Ai that I quote often, in which she writes:

“what can I say, except I’ve heard

the poor have no children, just small people.”

On reasons for diversifying the profession

In response to Eugene Sun Park’s article on why he left philosophy, Brian Leiter writes:

“What I still do not believe is that we should add Asian philosophers, or African-American philosophers, to the curriculum in order to “encourage” (on some misguided theory) minorities to enroll in philosophy courses.”

I agree. We certainly should not add anyone to the curriculum on the basis of a misguided theory. But knowing all that we know about stereotype and implicit bias, we have very well-supported theories in favour of adding demographic diversity to our syllabi. Knowledge of these theories tells us that our selections for syllabi are very likely influenced by implicit biases which it make it more likely that we will select white men. It also tells us that demographically diversifying our syllabi is no mere marketing ploy, but rather something which is likely to have real effects on the attitudes not just of students from underrepresented groups, but also on those of other students.* Those who are making such suggestions are not acting as “identity politics police”, as Leiter would have it. We are carefully examining the evidence, and working to improve our profession. Eugene Park’s testimony is a further piece of evidence (albeit anecdotal) that these suggestions are on the right track.

*For a summary of some of this, see my “Implicit Bias, Stereotype Threat and Women in Philosophy”, downloadable at the lower right, here.

Sexism at Science Journal Nature

A pretty striking statement about the underrepresentation of women from the Editors at Nature. A cause for cautious optimism? Might have been nice if they’d said more about what those ‘unconscious factors’ are, but the resulting heuristic is still a promising one:

We believe that in commissioning articles or in thinking about who is doing interesting or relevant work, for all of the social factors already mentioned, and possibly for psychological reasons too, men most readily come to editorial minds. The September paper speculated about an unconscious assumption that women are less competent than men. A moment’s reflection about past and present female colleagues should lead most researchers to correct any such assumption.

We therefore believe that there is a need for every editor to work through a conscious loop before proceeding with commissioning: to ask themselves, “Who are the five women I could ask?”

Thanks JI!

Books and bookmen (mostly)

Here’s an article from Salon by Laura Miller about the “literature gender gap”, and I don’t quite know what to make of it. The standfirst is pretty blunt: “women are under-represented in literary publishing because men aren’t interested in what they have to say”. Really?

There are some data in the article, and some speculation. Here are the data, which are partly derived from what Ruth Franklin says here in the New Republic:

1. In literary publications, the majority of contributions are by men;
2. In literary publications, the majority of reviews are by men;
3. In literary publications, the majority of books reviewed are by men;
4. The majority of books published are by men;
5. Women read and buy far more books than men.

Now, (1) and (2) are depressing but familiar findings, mentioned at the beginning but not really addressed in the rest of the Miller article. (4) is her main focus. It is cited as a reasonable explanation of (3), and the question then is, why does (4) happen when (5) is the case? Wouldn’t one expect women to want to read books by women, and thus, wouldn’t one expect the book trade to publish at least as many books by women as by men?

Using a mixture of anecdotal and survey evidence, Miller then says that women in fact tend to read books by men and women more or less equally, while men tend to read far more books by men than by women. Thus, a publisher can be reasonably sure of selling books by men to both men and women, but a book by a woman is more of a gamble, since only half the potential market is at all likely to buy it. So publishers, being risk-averse, mostly publish books by men. Therefore, the problem — the reason why publishers mostly publish books by men — is that men are not interested in what women have to say.

There is something fishy about this argument, I’m sure. But what, exactly? If, for example, women buy far more books than men, why does publishing a book by a woman represent a significantly higher risk? That seems rather weak reasoning.

Most speculatively, I wonder if there’s a difference between two claims: men are not interested in what women say (publish), and men are not interested in what women have to say (would like to publish). Based entirely on my own reading of novels and reviews of them, it would seem that women tend to publish more in the way of realist, domestic novels, in which truths of modern life are revealed indirectly by the study of some set of protagonists. Men do publish some of these, but are far more likely to publish Big Novels with Big Ideas and all sorts of stylistic, formal, technical innovations (I’m talking, by the way, about literary fiction, in some loose sense that contrasts with commercial fiction and genre fiction — and I’m assuming that the points I’m making would apply also to literary non-fiction and poetry).

That’s all very generalised, of course, but bear with me. It may be the case that men are more likely to read big abstract novels, and to not much care for the domestic stuff; and that the audience for the domestic stuff is thus mostly women. Again, pure anecdote seems to support this; women I know read both sorts of novel, men tend to just or mostly read the abstract stuff. There is a lot to be said about why these differences in taste emerge, and I won’t go into that here. The point I was wondering about is this: do women publish more domestic novels because they want to, or because they are encouraged to? That is, are men not interested in what women have to say, or what they do say? Because it seems to me quite unlikely that there are far fewer women than men who are able and keen to write big, abstract, technically clever books. So, in slight contrast with the conclusion that Miller draws, I would be tempted to say that men are uninterested in what women do say, but not in what they have to say.

This is about male readers, obviously. The remaining question is why women are perhaps discouraged from writing the kind of stuff men might like to read. Here, perhaps, we might in fact end up blaming men’s bias. As Franklin mentions in the New Republic, the first step to a book contract is often publication of something short in a journal of some sort; and women write just one third of such publications; and the ‘gatekeepers’ tend to be men. So it could be that the blame should be apportioned, not so much to the curious male reader browsing in the bookshop, but more to the men at journals, and publishing houses, who select which women get to the bookshop, and who perhaps tend to favour more stereotypically feminine subject matter from women writers. Perhaps these places should adopt an anonymous review system? Or do they already do that?

None of this seems quite satisfactory, though. As Miller says, the problem seems to be peculiarly entrenched. I suspect that what we have is a complex pattern of biases and imbalances that reinforce mutually, to the extent that it’s hard to single out one group, or one bias, and say that that is the cause of literary gender gaps. But I’d be interested in what other people think. Can we blame, for example, publishers, or readers? Do men really, actually, ignore books simply on the basis of the author’s name? Is any of the speculation about tastes and so on that I’ve indulged in accurate?

(Thanks, M and S!)

Lost women of science

A fascinating article:

Yet my re-examination of the Royal Society archives during this 350th birthday year has thrown new and unexpected light on the lost women of science. I have tracked down a series of letters, documents and rare publications that begin to fit together to suggest a very different network of support and understanding between the sexes. It emerges that women had a far more fruitful, if sometimes conflicted, relationship with the Royal Society than has previously been supposed.

I was a little worried when I read the following– afraid the author would go on to say that women had special insights because of their relational thinking, or care, or something like that:

Indeed, the Royal Society archives suggest something so fundamental that it may require a subtle revision of the standard history of science in Britain. This is the previously unsuspected degree to which women were a catalyst in the early discussion of the social role of science. More even than their male colleagues, they had a gift for imagining the human impact of scientific discovery, both exploring and questioning it.

But then he linked it instead to their exclusion from the Royal Society– a very interesting example for standpoint theorists:

Precisely by being excluded from the fellowship of the society, they saw the life of science in a wider world. They raised questions about its duties and its moral responsibilities, its promise and its menace, in ways we can appreciate far more fully today.

Gender Bias in Theater?

The NY Times reports on a year long study conducted by a Princeton economics student, Emily Glassberg Sands, whose work has been vetted by a number of excellent economists.  The study looked at the fact that many more plays produced are written by men than by women.  These are the conclusions reached:

1.  There are many more men than women playwrights and the men are more prolific.  Given that, men’s and women’s plays are produced at the same rate.

2.  A study, which sent a play to theater directors and literary managers around the country with the name variously a male or female one, indicated that men judge men and women equally, but women are more likely to favor the male writer.

3.  Women’s plays, when produced, are commercially significantly more successful, but they do not get any longer run time than the less successful men’s plays.  (A case of there being higher standards for woman to get treatment equal to a man.)

4.  “Plays that feature women — which are more commonly written by women — are also less likely to be produced”.  (One person’s quoted reaction said that the female characters were less likable; it was not possible to tell if this was supported by the research or an interpretation of it.)

Let me start off the comments.

I have heard the equivalent of 1 holds for the sciences; I am not sure about other fields, but I wouldn’t be surprised if it is true in most or all male  dominated fields.  The data in this case was drawn from a directory that depends  on self-reporting, so part of the difference could start with what is reported.  One reader comments that women are still socialized to believe that they will be wives and mothers in a way that puts other things second.  I am inclined to think rate of productivity may be influenced by having a sense of being a part of the community, having expectations that what one writes will be taken seriously, and so on.

2 is certainly unwelcome, but a reminder that women are equally likely to have implicit biases; if we simply assume we  are more fair, we may well be less fair.

Now, please, do join in and tell us what you think!

Gosh, what do you think he’s suggesting about Arabs?

People are remarking on the fact that McCain has recently defended Obama against some of his supporters’ accusations at rallies: there’s been so much hatred expressed, that surely it’s good to start speaking out against it. Surely…. Here’s the defense:

A woman at the town hall asks softly: “I’ve heard that Sen. Obama is an
Arab.”

McCain quickly cuts the woman off.

“No, ma’am. He’s a decent family man and citizen,” McCain says. “He’s
not. Thank you.”

Oh yeah, that’s a big improvement. No hating there. Nope.

Words and Able-ism

A post on Feministe led me (via Takenji) to this very informative post arguing that certain terms commonly used in a negative way on political blogs are offensive:

You know, when you throw around words about mental illness, like crazy, psycho or psychotic, frootbat, and nutjob, you’re mocking disability. You’re spitting in the face of everyone who suffers from a mental illness. You’re equating horrible behavior with mental illness. Stop it.

(Another excellent post is here, from Wheelchair Dancer.) This reminded me of Shelley’s argument that terms like ‘double-blind’ are offensive (see the comments here). But it also reminded me of one of the things I promised to discuss eventually– a talk at the recent SWIP conference by Jackie Leach Scully. Part of Scully’s discussion was about the many metaphors based on bodily abilities. Her focus was on the different ways that these metaphors may be understood by people whose bodies work in different ways. Scully, for example, is profoundly deaf, and reported that she spent many years misunderstanding the phrase “I hear what you’re saying”. It’s meant to convey a fairly deep level of understanding, but for Scully, hearing is about piecing something together very uncertainly from fragmentary clues– leading to a very different understanding of the metaphor. She listed many other such metaphors: “stand on your own two feet”; “stable”, and so on, noting each time how the metaphor might be understood by people with various different sorts of bodies. I thought of both the very widespread feminist discussion of ‘silencing’. Interestingly, Scully explicitly did not want to argue that all metaphors like these were offensive, despite the fact that they present being able to stand and being stable as positive, and being unable to speak audibly as negative (and equivalent to being unable to communicate). Instead, her take was that the experience of one’s own body is so fundamental that basing metaphors on it is inevitable; but that we should be aware of the potential for miscommunication when we do this. It’s perfectly compatible with this thought, of course, to find some particular such metaphors offensive, and I imagine that she does, though this wasn’t her focus. How to distinguish between the offensive and non-offensive metaphors would then become an important issue. (Some who I’ve discussed this with suggest that it’s easy: Just pay attention to what disabled people say on the topic. But since disabled people– like members of all other groups!– disagree with each other, there’s no such easy short-cut.)

You may well react to this– as I confess that I did, initially– by getting defensive insisting there’s no malicious intent behind use of these terms and that trying to expunge them is simply too much to ask. However, that’s how opponents of feminist linguistic reforms feel about, for example, the supposedly gender neutral ‘he’ or the insistence on classifying women as ‘Miss’ or ‘Mrs’. So I think it’s very much worth taking these worries seriously.

What do you think? (A note: for some reason, discussions of this topic on other blogs have shown a particular tendency to get heated. So please make an extra effort to observe our standard BE NICE rule.)