There’s a nice summary here of several studies of anonymization in peer review. The summary itself comes from a study showing that 74-90% of reviews (reviewers were invited to guess author identities) contained no correct guesses of author identity.
reviewers with author information were 1.76x more likely to recommend acceptance of papers from famous authors, and 1.67x more likely to recommend acceptance of papers from top institutions…when reviewers knew author identities, review scores for papers with male-first authors were 19% higher, and for papers with female-first authors 4% lower.
They really shouldn’t be used for hiring and promotion. They’re biased in many ways– including gender, race, and difficulty of topic. And they often contain abuse.
According to this:
- Women publish less, but their papers are more readable.
- Women apply for fewer grants, but their applications do better.
- Women doctors see fewer patients, but their patients are less likely to die.
- Women realtors show fewer properties, but get higher prices for them.
Suggested solution: women know they will be held to higher standards, so take longer and therefore do better (but slower) work. Read more here.
Jules Holroyd, Sarah-Jane Leslie and Sophie Stammers on Radio 4’s Analysis programme!
Here’s a link for the radio show.
And here’s a short accompanying article.
A new study on law school internship hiring has yielded interesting and dismaying results regarding the influence of both social class and gender on hiring. A c.v. study found that call back rates for men track class indicators, with men having c.v.s indicating lower class origins markedly disfavored relative to men with markers for higher class origins. The beneficial effects of higher class origins disappear for women, however, and women with markers for higher class origins received the lower callback rates than their lower class peers. A follow up study suggests that these women were perceived as the greatest “flight risk”:
Attorneys cited “family” as a primary reason these women would leave. Parenting strategies vary between social classes, and the intensive style of mothering that is more popular among the affluent was seen as conflicting with the “all or nothing” nature of work as a Big Law associate. One female attorney we interviewed described this negative view of higher-class women, which she observed while working on her firm’s hiring committee. The perception, she said, was that higher-class women do not need a job because they “have enough money,” are “married to somebody rich,” or are “going to end up being a helicopter mom.” This commitment penalty that higher-class women faced negated any advantages they received on account of their social class.
The study itself is disturbing in multiple ways, not least because the class penalties emerge in response to what would otherwise be laudatory information (e.g., working as a peer-mentor for first generation college students) or benign information (e.g., liking country music or sports with low cost). Moreover, the benefits accrued to higher class men relative to both women and their lower class male peers were dramatic: The higher class man “had a callback rate more than four times of other applicants and received more invitations to interview than all other applicants in our study combined.”
But that headline doesn’t even scratch the surface of how interesting this study is. Erin Hengel examined papers by economists in top journals. She found:
- Women’s papers took longer from submission to publication
- Women’s abstracts were more readable than men’s (employing standard measures of readability)
- Women’s papers improved in readability than men’s, during the transition from draft to final published version.
- Women’s abstracts’ readability continued to improve steadily throughout their careers, while men’s did not– leading to a very large gap in readability for senior women.
Hengel suggests that this may offer us a partial explanation for the often-noted productivity gap between men and women. If women are revising their papers more, and spending longer bringing them up to a higher standard, they are likely to publish more slowly. After considering several explanations, she concludes that the most likely one is that referees are tougher on women’s work than on men’s.
Thanks, L, for letting me know about this study!
I imagine many of our readers were shocked by a much-hyped study which purported to show lack of racial bias in police shootings. You can read a very important critique of it here.
First, there is extensive evidence (including in the datasets Fryer considers) of large racial disparities in who gets stopped by police, even controlling for differences in crime rates (perhaps especially under policies like New York City’s “Stop-and-Frisk”). Because of this, the “hit rate”—or the percent of times a stop ends with a confirmation of wrong-doing—is often higher for whites than blacks. Even if police pulled the trigger without “bias,” this disparity in stops would produce vastly unequal death rates.
This means that when we start the analysis by looking at encounters with police, we have already washed away some of the relevant racial bias. The unique data on police-citizen encounters Fryer relies on from Houston allows him in effect to “control” for the propensity to come into contact with the police in the first place. This is likely part of the reason he finds no evidence of bias in lethal interactions, while others have shown substantial racial disparities. For example, in a 2015 Plos One article, Cody T. Ross estimates that black Americans’ probability of being shot by the police is 3 times the rate for whites—and the disparity goes up to more than 20 in some counties. Similar community-level disparities that are unexplained by differences in crime rates emerge from a recent report from the Center for Policing Equity.