Uncategorized

Fighting implicit bias while sleeping.

jennysaul9 Comments

The study, published in Science, began with two Pavlovian-style conditioning exercises designed to counter race and gender biases. In the first, participants were shown female faces with words linked to maths or science and in the second, black faces appeared with pleasant words.

During the tasks, two distinctive sounds were played – one that came to be strongly associated with the gender pairs and the other with the race pairs.

Following the training, participants took a 90 minute nap and once they entered a deep sleep, without their knowledge, one of the sounds was played repeatedly.

After the counter-bias training exercise, and before the nap, people’s bias tended to have fallen, but without the extra cues during sleep, their level of bias had almost recovered to baseline after the nap. However, when participants were played the sound cues during sleep, their bias scores reduced by a further 56% compared to their pre-sleep score. Their scores remained reduced by around 20% compared to their initial baseline when the participants were tested one week later.

I am filing this one under “important if true”.

9 thoughts on “Fighting implicit bias while sleeping.”

  1. (This part is a Facebook comment I made on this story last night.) It’s a low-power study (n = 38) of young white people, and the effect 6-10 days after the experiment is small. There are other red flags, including bad plots (Fig. S1 is better than anything in the paper, and even that’s a bad plot) and bad reporting of statistical tests (and possibly inappropriate, depending on exactly which comparison you think is important). Given all of the problems with experimental psychology over the last decade or so, I wouldn’t expect this to hold up in a more rigorous study.

    (This part I’m adding here.) Here’s a good, non-technical discussion of the problems with studies like this: http://www.slate.com/articles/health_and_science/science/2013/07/statistics_and_psychology_multiple_comparisons_give_spurious_results.single.html

    And here’s a more technical version by the same author: http://www.stat.columbia.edu/~gelman/research/unpublished/p_hacking.pdf

    Here’s an excellent critique of the kinds of plots used in this study: http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1002128

    I haven’t had too many interactions with folks interested in experimental philosophy, but based on those interactions it seems like (1) x-phi folks are modeling their methods explicitly on these kinds of studies in psychology, while (2) not being aware of or responsive to the epistemic crisis surrounding these methods in psychology. So I would encourage x-phi folks to check out the links above.

  3. Dear Dan,

    Based on my extended interactions with lots of people who do experimental philosophy, it does not seem to me that they are “modeling their methods explicitly on these kinds of studies in psychology,” nor does it seem to me that they are generally unaware of the problems associated with some recent psychological science.

    I seriously doubt the value of generalizations about “x-phi folks” based on infrequent “infrequent interactions with folks interested in experimental philosophy.” In every field, some people will do better and worse research. And while I expect that you could find examples of work in experimental philosophy that fail along one or another of the dimensions you indicate, you could also find examples that don’t. More generally, critical commentary on an entire field of research — especially when accompanied by mention of “crisis” — should probably occur only in the context of reasonably large, representative sample of research from the field.

  4. Hi John,

    It’s not hard to find papers by proponents of experimental philosophy who explicitly connect their work to experimental psychology, e.g., Nadelhoffer and Nahmias: http://www.tandfonline.com/doi/abs/10.1080/13869790701305921,
    or Alexander and Weinberg: http://onlinelibrary.wiley.com/doi/10.1111/j.1747-9991.2006.00048.x/abstract

    Likewise, it’s not hard to find examples of papers by (researchers whom I take to be) high-profile experimental philosophers with the kinds of weaknesses that I found in the Science study — bad plots, bad reporting of tests, no assessment of power and what appears to be low power given the observed effect, etc. For example, it seems like many of the studies reported in the Buckwalter and Stich “Gender and Philosophical Intuitions” paper have some of these weaknesses: http://papers.ssrn.com/sol3/Papers.cfm?abstract_id=1683066

    Again, this isn’t a corner of philosophy that I’m very familiar with, and I recognize that the Buckwalter and Stich paper in particular is from 2010 — ages in the development of a new field — and was subject to serious criticism when it was first circulated. But maybe you can understand why I’ve gotten the impression that I have. In any case, if you could recommend some recent discussions in experimental philosophy that speak to these kinds of concerns, I’d be interested in reading them.

  5. Hi Dan,

    It’s not hard to understand how someone can get an impression of one sort or another. What I have a harder time understanding, however, is why someone admittedly unfamiliar with a field of research would, in response to a post unrelated to that field and on a blog with a completely different focus, characterize the field as suffering from an “epistemic crisis.”

    It’s easy to find fault with just about any piece of scientific research, and just about every field can and should strive to improve its methodology and standard practices. I welcome and appreciate the desire to add critical perspective on work in experimental philosophy. It would be great if future interventions occurred more constructively and in a venue where their merits in relation to specific research could be assessed.

  6. Hi John,

    I’m sorry if my first comment was ambiguous. I intended to apply “epistemic crisis” only to experimental psychology, and not to experimental philosophy, in part because the topic of the original post was an experimental psych study and in part because I’m more familiar with the state of things in experimental psych. “Crisis” is used regularly by experimental psychologists to describe the situation in their own field; for example, see this special issue of Perspectives on Psychological Science: http://pps.sagepub.com/content/7/6.toc

    My claims about experimental philosophy in this thread are meant to be tentative, and I’m sorry if poor phrasing or word choice suggested otherwise. Again, if there are methodological discussions in experimental philosophy that you would recommend, I’d be interested in reading them.

  7. Hi Dan,

    I too don’t have particular high credence on this recent study. I think any intervention claiming to reduce implicit bias should go through a rigorous replication process before entering meaningful discussion. Calvin Lai has done an amazing job leading this effort: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2155175 .

    Like John Turri, I too take issue with the claim that experimental philosophers are unaware of the issues in psychology. In fact, some experimental philosophers may be in a better position because they have thought about philosophy of science and statistics in relation to experimentation. See, e.g., Edouard Machery on interpreting null results: http://philpapers.org/rec/MACPAN . I gave a talk last year at the UK X-Phi meeting on Repligate and related methodological issues. Here is a handout from the talk: http://liao.shen-yi.org/secret/XPhiUK.pdf . As you can see, there is a long history of awareness of replicability’s importance and have already responded to it… even before the crisis happened; see, e.g., the replication page that Knobe and Mott maintains and discussions that happened on the Experimental Philosophy blog. Like John Turri, I find it a bit incredible to claim that “[x-phi folks] not being aware of or responsive to the epistemic crisis surrounding these methods in psychology” when a cursory glance shows this to be far from the truth.

  9. Hi Shen-yi,

    Thanks very much for those links, and for your follow-up blog post. Those are exactly the kinds of discussions I wasn’t aware of.

Comments are closed.