Haslanger and Refereeing Procedures

Note: This post has been edited in response to Shelley’s comment.

There’s been a lot of lively discussion online lately regarding Sally Haslanger’s excellent paper on women and minorities in philosophy.  Here I’m going to focus on her discussion’s implications for editing and refereeing procedures.  (If you’d like a discussion of the importance of diversity in both faculty and students, go here.  If you’d like a discussion of gendered metaphors, styles, and behaviour in philosophy, go here. For general discussion and extracts from an interview with Haslanger, go here.) Haslanger draws on Virginia Valian‘s work on gender schemas, which we’ve often discussed on this blog.  Valian argues that the vast majority of us, whatever our conscious beliefs, subscribe to unconscious gender (and other) schemas that influence our judgments.  The experimental data she offers are very convincing and, even to me, surprising.  Haslanger suggests, very plausibly, that our schemas for philosopher and woman are likely to clash.  This may causes us, even if we have very explicit strong beliefs in favour gender/sex equality, to judge female philosophers more harshly than male philosophers.  The effects may be small, but (a) small effects can make a big difference in a very competitive situation; and (b) small effects accumulate and reinforce each other, and over the course of a career they can make a huge difference. This suggests to me that journal refereeing and editing should be anonymous to both referee and editor.  That way unconscious gender schemas will have no chance to influence judgments.  Nor will knowledge that the author is un-famous, or has a foreign name, or is from a non-prestigious institution.  Anonymous refereeing is currently fairly widely practiced in philosophy, although some top journals make anonymity an optional choice for the author.  Anonymous editing is rare. Why is anonymous editing important? Because at many top journals, editors reject 70-90% of papers without even sending them to be refereed. (Anecdotal evidence seems to suggest that an individual editor’s attitude toward such things as promoting the work of unknowns can make a huge difference at this stage.) So this seems to me a no-brainer. We should all strive for anonymous refereeing and editing. It’s the best way to assure that biases, conscious or unconscious, do not play a role in decisions. My understanding is that this is the norm in the sciences. (I’d love to hear from scientist readers about whether this is true.)

In the discussion over at TAR, however, few agree with me on this. (1) Brian Weatherson, an editor, argues that he needs to know sex so that he can practice a helpful affirmative action in favour of women. (He also seems pretty confident that he doesn’t have unconscious biases based simply on knowledge of sex.) His thought, drawing on other claims of Sally’s, is that women may be more likely to employ e.g. a non-aggressive writing style, and this may unfairly disadvantage them. If he knows they are women, he can correct for any bias he may have against non-aggressive writing. My first thought was that this could be done by a final non-anonymous stage after the anonymous stages of refereeing. But then I thought: if you think non-aggressive writing is good, and you suspect you are biased against it, shouldn’t you be extra careful about your judgments regarding all non-aggressive writing, regardless of sex? (2) Baber argued (as others have in previous discussions) that anonymous-refereeing is on its way out due to posting of papers on the internet: the idea seems to be that we’ve all read them all already, so we know who they’re by. Maybe I’m just bad at keeping up with the literature, but this seems hugely overstated. And we’re especially unlikely to have read the work of unknown people, for whom anonymous refereeing is most important.

I think what most troubles me about all this resistance to anonymous refereeing is that everyone seems so sure that they’ve overcome their own biases. The whole point of Valian’s work is that our conscious beliefs don’t tell us whether we’ve done this. I’ve spent a lot of my career on feminism, and I don’t assume I’ve overcome even my gender/sex biases. (And one interpretation of the tests JJ referred us to is that I haven’t, as I note in my comments there.) No scientist would dream of saying “I don’t need to do [so-called] double-blind experiments– I know I am free of interpretation biases”. Why do philosophers make and accept these claims of bias-freedom so easily?

32 thoughts on “Haslanger and Refereeing Procedures

  1. I haven’t read the paper yet (thanks for the link!) but I’m wondering whether or not blind refereeing and editing would really be all that effective. Beyond even women using non-aggressive writing styles (and I’m not sure I really buy that that’s so much a woman thing), software exists which can accurately “guess” an author’s gender. Surely we’re on some level able to make similar guesses, and surely those guesses inform any gender bias we may have as much as a name or circled “F”. I don’t think that such a policy would really stand in the way of equal treatment, I’m just not sure that it would make much of a difference. Or, I’m not sure the case has been made here…

  2. My impression was that such software is actually pretty poor when dealing with something like academic writing– I’ve heard somewhere that all analytic philosophers just get called ‘male’ (is this right?) because the conventions of the genre swamp whatever gender differences the software looks for. Of course, it’s entirely possible that there are lots of other cues that academics will pick up on but software won’t. However, I’ve frequently been told that there was a huge rise in women’s university degree results when anonymous marking was introduced to the UK. This, if true, suggests that even if anonymity isn’t perfect it could still make a difference.

  3. My sense is that the situation is a bit better for women in the top-20 U.K. Philosophy Depts. But maybe that’s just wishful thinking (!) Does anyone know of any stats around on it? (If it is better in the U.K context, it would be interesting to know why.)

    Also, one stat in particular that might be worth knowing is the ratio of PhD’s to jobs by gender: what are the odds of getting a job after PhD broken down by gender. If it’s close to equal, that might bode well for the future – in the U.S. and the U.K. If not…well…

  4. Whoops – maybe that comment should have been in the more general ‘Women in Philosophy” section, rather than in this section on refereeing. Apologies !

  5. AC, I think these issues are all closely connected.

    Something I just discovered is that one’s word processing program can put one’s name and affiliation in among a document’s properties and that’s very easy to access. In my word program it’s accessed through “properties” under “edit.” You can just change it to “anon.”

    Let me mention something that I experienced recently at some conferences, because I think it illustrates what can happen to a woman’s work. I was giving a pretty different take on a central issue (actually, very different take), and at the first conference it provoked what could aptly be called “extreme hostility.” As in, I was pretty much put outside the discourse of the conference. At the next conference, similar material, very similar reaction – “you think your little stuff can show MAJOR MALE philosopher is wrong” (sneer, sneer) until a STAR MALE philosopher got up and basically said he loved it. Would others have been as positive later as they were had he not gotten up? I have no idea, but until he gave his talk, I was thinking of leaving. I mean, there’s a limit to how many times one should have to tolerate that.

    Should I add that I’m pessimistic about getting it into a journal?

  6. Hi Jender,

    I’m one of those who think the solution (alright, that’s far too optimistic: a step in the right direction) might be to abandon blind refereeing. But I certainly don’t think because I think I’ve overcome my own biases, and I didn’t read anyone else in the discussion at tar as claiming that.

    I think I have plenty of unconscious biases. I just think that I have a better shot of overcoming them, or compensating for them, if I’m actively conscious of the fact that I should. And I’m going to be actively conscious of that fact if I’m told that the paper I’m refereeing is by a woman. If I thought I was bias free I wouldn’t think any of this was an issue; but I do.

    It’s an empirical question whether knowing the gender of the author would help a referee overcome or compensate for bias, of course. But I find it a prima facie plausible claim; it’d be worth investigating.

  7. OK, thanks, Ross, now I see what you’re saying, and you’re right that there are interesting and important empirical issues here, and that we don’t know the facts. But here’s my worry. You and I, and hopefully increasingly many others, have read Sally’s paper, and thought about this stuff, and want to struggle to overcome our biases. But not everybody is like this. And the procedures that we as a field accept should be ones that also work when an editor doesn’t think bias is even worth thinking about– and I do know non-blind editors like this. And your procedure– if you’re right about it– only works for those who are willing to think about and work on their biases. I’d love it if everyone was like that, but they simply aren’t. It’s perhaps worth noting also that there are lots and lots of biases worth correcting against– including esp. ones favouring the well-known but who knows how many others. It would be very difficult to bring oneself to awareness of all these potential biases and correct against them, I’d think. Fortunately, there’s a dead simple procedure available to deal with all of them: anonymity. (Actually, this doesn’t deal with the stylistic biases, but I’m not convinced affirmative action is the right way to deal with them either.)

  8. While I’m very pleased that Sally Haslanger’s paper is being widely discussed on various blogs, I am dismayed that so much of this discussion is ableist, that is, discriminates against disabled people. Unfortunately, this ableism conditions much of the Feminist Philosophers blog entry above, is made explicit close to a dozen times in the entry, and is caricatured in a graphic at the top of the entry. I am referring to the use of the term “blind” to allegedly signify a form of anonymous reviewing. The putative explanation generally given for this use of the term “blind” is that this is a form of refereeing in which the reviewers cannot “see” the name(s) of the author or authors. Not seeing is blindness. Right? Wrong. That is not what is really going on here. What is actually going on is that blindness is being metaphorically equated with not knowing, with not having knowledge, or not having knowledge of something: blindness is not knowing, blindness is ignorance, blind people cannot be knowers. Consider the expressed purpose of this form of refereeing. According to the ideals of this form of refereeing, it would not be any more acceptable to have the name of an author whispered in a reviewer’s ear than it would be to have someone give her a peek at it. As the Feminist Philosophers entry itself notes: “journal refereeing and editing should be done blind (with neither referee nor editor knowing the name of the author)”. (Notice that there is a presumption that all reviewers in philosophy will be sighted people. Blind people can’t be philosophers! That goes against the schemas of who is a philosopher and what blind people can do!)

    Sally Haslanger and I have in fact begun to engage in a discussion about claims she makes in the paper. We are beginning to discuss how many of the claims she makes in regards to women and racial minorities apply to the hostility that disabled philosophers confront, how many of the claims about work on gender and race apply to work on disability, and how ableist and disablist metaphors abound in philosophy. Unfortunately, the latter seem to appear now and again on the Feminist Philosophers blog, though I have yet to read anything about disability oppression here! I guess the occasion that most readily comes to mind appeared in a Feminist Philosophers piece about Virginia Wolff which criticized her classism in a very uncritically self-referential way: the term “blind” was used in that piece to signify ignorance, that is, Wolff’s ignorance of forms of oppression other than gender oppression. I struggled with myself about whether or not to challenge that post (I receive the Feminist Philosophers email). In the end, I decided to simply delete it. I regret that decision now: perhaps if I had pressed the point, opened up a discussion, this discriminatory (ableist) use of the term “blind” would not have been reproduced so many times on this blog and others by feminist philosophers who are very politically astute in many regards, but seem uninformed about the oppression of disabled people.

  9. It’s perhaps worth noting also that there are lots and lots of biases worth correcting against– including esp. ones favouring the well-known but who knows how many others. It would be very difficult to bring oneself to awareness of all these potential biases and correct against them, I’d think. Fortunately, there’s a dead simple procedure available to deal with all of them: anonymity.

    This just sounds totally right to me. I haven’t read yet the discussion over at TAR, but I guess that an appropriately generalized “disclousure” option should then also have to include info about ethnicity, sexual orientation, reputation, and whoknowswhatnot.

  10. Weatherson’s pseudo-affirmative action procedure strikes me as well meaning but bizarre. The idea of noting an author’s gender and then mentally compensating for supposed bias seems so to me so obviously imprecise as to be unviable. If anything, I would worry about Weatherson favoring female authors to too great an extent, if such a policy were implemented. The whole scheme seems precarious. (Moreover, the idea that a person can internally manage his own biases strikes me as self-stultifying.)

  11. Thanks, Shelley, for pointing that out. I was completely unaware of what I was doing. Which, in a way, demonstrates the difficulty of overcoming all of one’s biases through sheer force of will. I’ve certainly tried to do so, but I obviously haven’t succeeded.

  12. Re: the software that “guesses” gender, there was some discussion on Brit’s blog (also related to Haslanger) about the Gender Genie. (See comments).

    Men who were analytic philosophers somehow got categorized as women, and vice versa. At least on that anecdotal evidence, I am suspicious that we can guess an author’s gender accurately. Are there any studies on this, for academic literature?

  13. I think I have plenty of unconscious biases. I just think that I have a better shot of overcoming them, or compensating for them, if I’m actively conscious of the fact that I should. And I’m going to be actively conscious of that fact if I’m told that the paper I’m refereeing is by a woman.

    I’m with Coffman, on this. I can appreciate wanting to be aware of your own biases and correcting for them, but, if nothing else, I think that the potential risks involved should one fail to fight one’s biases is worth making assesssments anonymous. The problem I see with telling you the sex of the person whose paper you’re refereeing, Ross, is that, if you’re unable, for some reason, to compensate for your biases, we’ve just given you information that you probably wouldn’t have had, and we’ve made your bias a factor in the refereeing. If you hadn’t been told the sex of the other person, you wouldn’t really have that information. You’d be forced to referee the writing for the sake of the writing.

    And if you’re concerned about refereeing “feminine” writing styles more harshly than “masculine” styles, that’s something that is going to be a factor regardless of the sex of the person doing the writing, and is something you should be working to compensate for or correct regardless of the sex of the writer, isn’t it?

    In other words, as I see it, giving you information about the sex of the writer only increases the risk that you’ll referee unfairly, by giving you that extra information and raising the possibility of sex bias. By hiding the sex of the writer, we decrease that possibility, and do nothing to prevent you from working to overcome your prejudices. The risk to a writer is just reduced in that case, should your attempts to overcome bias fail.

  14. It should be possible to set up a little self-test, like the IAT, enabling editors (and others) to self-check for name-recognition biases (not just sex/gender, but ethnicity and other cues). Those with moderate or severe biases should *certainly* not try to compensate solo — and perhaps shouldn’t be editing at all (or have I left room for too few editors?)
    Biased responses to linguistic style might also be checked out, if some differences in academic (philosophical) style could really be confirmed and if they admit of pretty systematic translations back and forth.
    We shouldn’t think of any self-test as an alternative to anonymous reviewing, though. It would be wiser for those in editorial positions to look only at anonymous texts, and to enlist the support and feedback of others to correct for biased responses to style. I’m not clear on how the details might go, however…

  15. Re my last comment: I just flew through most of the staff pages and, one or two exceptions aside, the permanent female staff numbers are pretty abysmal. Birkbeck’s number of women at both junior and senior levels is especially impressive. And I guess even if the PhD/job ratios are close to equal, that aint gonna make much of a difference to the number of women in the profession if the grad. student numbers aren’t at least close to equal too, which in most places, it seems, it’s not. (My ‘sense’ was that it was.) Just goes to show that one’s ‘sense’ of the way things are doesn’t always track the truth; wishful thinking can be a powerful thing.

  16. Yes, wishful thinking is powerful. Another powerful influence is that we’re so used to having not many female philosophers in a room that, say, 4 out of 10 looks like “lots of women”, where if 4 out of 10 were men we’d be shocked by how few men there were. There’s name for this, but I can’t remember what it is!

  17. Thanks for the link to the paper – it has been quite enlightening. I understand stereotype threat all too well. As a high achiever in undergraduate psychology and philosophy programs, I felt substantial perceived pressure to conform to the mainstream ideologies of the departments I was in. Furthermore, at the time, I understood analytical philosophy to be an individualistic and highly competitive pursuit and, in retrospect, worked hard at keeping a stiff upper lip (unfortunately, this resulted in my being relatively ignored, I think). Though I graduated top of my class in philosophy, there was little to no support/advice given for further education.

    I often wondered whether I ought to have reached out for help more often, or whether I should have tried harder to conform to people’s expectations of me (I am female, lesbian, and of Asian background). I wondered if I had been too defensive in my dealings with others (after years of having had “otherness” thrust upon me). In the end, I decided to pursue other avenues and continued to read up on feminist philosophy on the side, even though it is my true passion…

  18. […] Academia and credibility September 24, 2007 Filed under: Journals, bias, feminist philosophy, gender, human rights, science, silencing, women in philosophy — JJ @ 11:35 pm Jender’s discussion of Rachel McKinney’s wonderful post on “grey rape” led me to McKinney’s site and a great discussion, with informative links, about underrepresentation in philosophy and engineering. And through those links to important information about women’s publishing in ‘top’ philosophy journals. And then back to Jender. […]

  19. I admire some people’s tenacity and optimism that academic philosophy can be fixed. However, as it stands, given the amount of aggression and mental violence that women and minorities wil be forced to endure in the discipline, how can we in good faith encourage them to join?? I would like to read a good argument as to why women and minorities should be encouraged to pursue a career in academic philosophy when they have, seemingly, numerous better options. I think an undergraduate degree in philosophy is very valuable as a stepping stone ….but a doctorate??? What are the arguments? I would like to know this as I would to know how to responsibly advise my philosophy majors.

  20. Read this and found it very depressing with the inability of the male posters to engage in a real dialogue. I have been reading “Does Femnism Discriminate Against Men,” a debate between W. Farrell and S. Sterba. Feminist philosopher Sterba does an excellent job. How can we encourage male philosophers to read this? If male philosophers don’t stop being sexist, philosophy departments will continue to be uninhabitable for women. Buy copies for the staff room and say that they were free from the publisher? I’m thinking of sending some to philosophy departments in my city.

  21. Anon, I’m not very optimistic about changing people. I think there are some effective strategies for reaching individuals, but the best ones cost money – that is, one bribes them to do a thorough departmental self-examination or involves institutional pressure.

    Now that we have blogging that is making these problems so much more visible, perhaps we’ll be able to put pressure on the APA.

  22. PS to Anon: I like and appreciate your comment about the inability of male philosophers to engage in real dialogue. It is the height of arrogance for them to expect us to respond constructively to them, as they pretty clearly do.

  23. A quick note, before short-fused commenters (who may not be reading carefully) jump in: The full context makes clear that JJ is not referring to all male philosophers in the comment above. Also, I’m going put in a pro-active plea right now that we not delve deeply into revisiting nasty comments and working exactly what went wrong in threads that got heated. That seems to me very likely to be unconstructive.

  24. Thanks, Jender. I forgot about the potential audience.

    I think it would be great if we could find a way to encourage helpful dialogue from the beginning, but it may be that a prevalent argumentative style of our discipline can count against it.

    That is to say that many people in philosophy (not necessarily identified as male or female) put forward positions with the expectation that you either accept them or you have to refute them. That seems to be a very different model from one that aims at reaching some mutual understanding at least of one another’s concerns, priorities, views.

    In addition, blog communication can make things worse, because one misses out the cues about what is meant that are not carried by the words.

    In fact, I’m inclined to think the “accept or refute” approach may be more practiced by the men, but it certainly is not done so exclusively.

  25. Yes, I am all for encouraging a helpful dialogue with those who are willing. And yes some male philosophers, like John Sterba, are willing and wonderfully able. I think it is helpful to try to bring out and support more feminist male voices. I think feminist male philosophers who visit blog spaces such as these should feel most welcome in making comments. I know it can be hard for male philosophers to make a stand against sexism (just as it requires fierceness from female philosophers). … I do think it is interesting though that Jender felt a need to rush to point out that JJ wasn’t thinking of all male philosophers when she said that male philosophers are arrogant. I think that one could take for granted that she did not mean all but I can understand Jender not wanting there to be a flare-up in communication. I do think that it is important to acknowledge and accept that most male philosophers are not femininst friendly, meaning, a large percent, and so to be knowledgeable about the real situation that feminist philosophers face. Male philosophers need to be given much more pressure to change. I am finding that public knowledge about the horrible sexism raging in philosophy departments is very poor. If this sexism is exposed more male philosophers will be forced to change. For instance, I recently joined a choir that has a very sexist male philosopher. People were asking about my profession and I was very honest about the sexism I have long faced. The male philosopher became very uncomfortable and aggressive. However, no one doubted my account; and others have become less friendly and supportive toward him.

  26. […] Re-visiting this incident, I sat down to formulate this blog, which I’d entitled ‘making the classroom more inclusive, interrogating the blind spots in our teaching’, hoping that I could emphasise how easy it was to forget the perspective of the reader, the student, the learner, even in our desire to be inclusive.  Until I thought about it, and realised the itself the word ‘blind spot’ carries connotations of able-ism, associating ‘blind’ metaphorically with ignorance, an accusation which, when highlighted, is appalling (see for example this Feminist Philosophers blogpost on refereeing procedures). […]

Comments are closed.