There’s a nice summary here of several studies of anonymization in peer review. The summary itself comes from a study showing that 74-90% of reviews (reviewers were invited to guess author identities) contained no correct guesses of author identity.
reviewers with author information were 1.76x more likely to recommend acceptance of papers from famous authors, and 1.67x more likely to recommend acceptance of papers from top institutions…when reviewers knew author identities, review scores for papers with male-first authors were 19% higher, and for papers with female-first authors 4% lower.
They really shouldn’t be used for hiring and promotion. They’re biased in many ways– including gender, race, and difficulty of topic. And they often contain abuse.
Jules Holroyd, Sarah-Jane Leslie and Sophie Stammers on Radio 4’s Analysis programme!
Here’s a link for the radio show.
And here’s a short accompanying article.
Why did you start investigating this issue?
A postdoctoral fellow in my lab pointed out that the preliminary speaker list for an international neuroimmunology conference included only 13 female speakers out of 93 total. I contacted the conference organizers, and they responded that there weren’t enough accomplished female neuroscientists at senior ranks to invite. So I thought, “That’s a hypothesis that I can test.”
Read on, to find out how she indeed did test this hypothesis, and to find what seems to make a difference.
But that headline doesn’t even scratch the surface of how interesting this study is. Erin Hengel examined papers by economists in top journals. She found:
- Women’s papers took longer from submission to publication
- Women’s abstracts were more readable than men’s (employing standard measures of readability)
- Women’s papers improved in readability than men’s, during the transition from draft to final published version.
- Women’s abstracts’ readability continued to improve steadily throughout their careers, while men’s did not– leading to a very large gap in readability for senior women.
Hengel suggests that this may offer us a partial explanation for the often-noted productivity gap between men and women. If women are revising their papers more, and spending longer bringing them up to a higher standard, they are likely to publish more slowly. After considering several explanations, she concludes that the most likely one is that referees are tougher on women’s work than on men’s.
Thanks, L, for letting me know about this study!