Stereotype threat research called into question

There is apparently a literature review forthcoming which argues that studies purporting to show female underperformance at maths due to stereotype threat are flawed. Unfortunately, the article is not yet out, and none of the articles I can find bothers to interview any defenders of stereotype threat theory for a response to the criticisms. In the absence of more information, I just don’t know how to assess claims that the studies are flawed. The more specific bits quoted don’t impress, though:

“We were surprised the researchers did not subject males to the same experimental manipulations as female participants,” Geary said.

“It is reasonable to think that men also would not do well if told ‘men normally do worse on this test’ right before they take the test. When we adjusted the findings based on this and other statistical factors, we found little to no significant stereotype theory effect.”

The studies I’ve seen don’t involve saying anything like this to women. It’s not needed, since women begin from the presumption that a test of maths ability will be one that women do worse on. There *are* studies which compare their performance in the absence of gender claims with their performance after having been told that men and women do *equally well* on this test. And their performance improves. I’d want to know whether studies like this one are also subject to criticism. (They may well be: perhaps there are statistical errors.)

Thanks, R.

Do let us know in comments if responses to this have come out, or if the full article is available.

18 thoughts on “Stereotype threat research called into question

  1. That is a very strange quote. It sounds like Geary is describing research concerning exactly how stereotype threat works, which is certainly interesting, even more so if the evidence suggests it works differently than previously believed. But it doesn’t sound like his research is at all about it not existing, so it is highly suspicious that he seems to want to encourage that sensationalist interpretation. Very curious to see the full study.

  2. While I have not looked at the literature on stereotype threat in depth, my understanding is that the findings are much more intricate than is usually acknowledged – particularly when stereotype threat is used to explain various gender differences (as does, e.g., Fine).

    In particular, the effect seems to depend on various moderators, and there is no consensus about the mechanism underlying the phenomenon. On the other hand, the metanalyses and lit reviews I looked at didn’t challenge its reality.

    It would be great if someone looked at the literature in detail.

  3. Why should we think we’ll get stereotype threat by telling someone they won’t do well and/or they beliong to a group that won’t do well. Men do well often enough and they know it. One does not get a stereotype adopted by a little lecture that seems contrary to known fact.

    Part of some article I read used as a male example of stereotype the supposed fact that white men’s backetball game declines when they play with black guys, expecially if someone says anything about jumping. “White guys can’t jump” is a stereotype.

  4. A 6 minute YouTube video on the study can be found by searching its title (“Can stereotype threat explain the gender gap in mathematics performance and achievement?”).

  5. Honestly, 2 min and thirty seconds into the video I think my blood pressure went sky high. I don’t know anyone who says stereotype threat is an explanation for the cited disparity. It is one element, a different matter. And we’ve discussed several times the canard that though on average men and women are about equal in math, only (or almost only) men get the really high scores and awards.

    If someone manages to get through the whole thing and doesn’t see more blunders like this, I’d love to hear. But it sounds just like the sort of reactionary stuff that serious scientists manage to get past peer review. As though their peers already think what they’re saying is pretty obviously right. Could peer review be biased!?!

    Gosh, I am being harsh. Let me know if it gets better, please.

  6. thanks Jender, seems to me interesting to see what *they* want to/will come up with.

    on a sidenote, meh, sounds like a biased interpretation to me (*what-about-teh-menz*), imo because e.g. in a soc. kyriarchical/heteronormative/androcentric/etc.-system there simply is no soc. un-biased research/science
    (i.e. *white-cis-male* being the umbrella-default-setting for soc. peer-reviews e.a.)
    reminds me of Cordelia Fine’s book (“Delusions of Gender”) where she also addresses this and cites/meta-analyzes studies concerning stereotype-threat (yeah i know “its in the/her book”)

    > Could peer review be biased!?! <
    // rofl //

  7. In haste — in case this hasn’t been noticed:

    It’s already *old news* that when you make the threat *explicity* then the effect is almost reversed for *some* tasks, because you arouse what psychologists call “reactance” (the desire to resist a perceived social pressure, supposedly). Studies show, for example, that women do “worse” in bargaining scenarios under standand stereotype threat (passive exposure to stereotype priming), but BETTER (or at least not worse, sorry for vague memory) when they are explicitly told that they’re expected not to do well at bargaining *because* they’re women.

    There was a great paper at the last FEAST conference by Nathifa Greene on related matters.

    I’ll try to revisit the discussion when I have more time to digest everything.

  8. I agree with Anne in post 3. Surely such a little speech only has the power to reinforce existing stereotypes.

  9. Elf, do you recall if there were any commonalities to the sorts of tasks where explicit stereotyping drives ‘reactance’? Is it a general point, or are there certain types of tasks where it seems to happen regularly?

  10. “The studies I’ve seen don’t involve saying anything like this to women.”

    Hm, well here is one study where participants were told there were gender differences, although not which kind of difference:

    “participants were told that the test had shown gender differences in the past—a characterization that explicitly evoked the stereotype about women’s math ability.”

    “We assumed that telling participants that there were gender differences would lead them to believe
    that men did better than women. Of course, this conclusion is not inevitable, but all participants in this
    condition when asked informally reported this to be their interpretation.”

    Stereotype Threat and Women’s Math Performance Original Research Article
    Journal of Experimental Social Psychology, Volume 35, Issue 1, January 1999, Pages 4-28
    Steven J. Spencer, Claude M. Steele, Diane M. Quinn

    And this one talks about 2 studies in which women read scientific theories saying gender differences in math are experimental vs. genetic:

    “We investigated whether stereotype threat is affected by accounts for the origins of stereotypes. In two studies, women who read of genetic causes of sex differences performed worse on math tests than those who read of experiential causes.”

    Exposure to Scientific Theories Affects Women’s Math Performance
    Ilan Dar-Nimrod and Steven J. Heine
    Science 20 October 2006: 314 (5798), 435. [DOI:10.1126/science.1131100]

    So these two examples both seem to be saying something like “women normally do worse on this test” even though it is implied and not explicit. And since it is implied, particularly in the first case, the criticism that males were not subjected to the same conditions would seem to miss the mark.

    I will be interested to see this literature review.

  11. Matt (and all),

    My rough memory comes by way of Nathifa Greene’s conference presentation — any error to be ascribed to me, not to her. But one suggested framing of the difference was that activities requiring real embodied “fluency” were not so easy to handle with the kind of willful stereotype-resistance that would go with reactance. (But there may be a threshold of fluency — say, Jackie Robinson’s — that enables grace even under enormous pressure. Nathifa (again, if I remember) was particularly interested in the difference between interactions in which individuals become self-conscious of stereotype in a weakening way (intimidated/distracted) and those in which they become self-conscious in a strengthening way (inspired to set a deliberate example, “show them what I/we are made of” etc.).

    Here’s the study of negotiations and implicit/explicit stereotype threat that I mentioned in my prior post, with abstract below:
    >The authors examined how gender stereotypes affect negotiation performance. Men outperformed women when the negotiation was perceived as diagnostic of ability (Experiment 1) or the negotiation was linked to gender-specific traits (Experiment 2), suggesting the threat of negative stereotype confirmation hurt women’s performance relative to men. The authors hypothesized that men and women confirm gender stereotypes when they are activated implicitly, but when stereotypes are explicitly activated, people exhibit stereotype reactance, or the tendency to behave in a manner inconsistent with a stereotype. Experiment 3 confirmed this hypothesis. In Experiment 4, the authors examined the cognitive processes involved in stereotype reactance and the conditions under which cooperative behaviors between men and women can be promoted at the bargaining table (by activating a shared identity that transcends gender). (PsycINFO Database Record (c) 2011 APA, all rights reserved)

    There’s a whole variety of publications on this theme; no one study seems poised to “debunk” stereotype threat as such, but each may be in a position to tweak our understanding of what’s going on, when, and how.

  12. “Do let us know in comments if responses to this have come out, or if the full article is available.” Yes, it’s available online through my library. Probably through other libraries too.

  13. Thanks, Jean! I hadn’t realised it was available pre-publication. I hope to have time to read more closely soon, but so far I’ve been struck by how much weight is put on the fact that subsequent studies are not exact replications of the original, because they have different procedures, different sorts of subject pools, etc. This seems very weak evidence against the claim, and shouldn’t be viewed as a *failure* to replicate, especially since my understanding is that *exact* replications will rarely get published. Those designing the subsequent studies weren’t trying to replicate and failing, they were trying to explore some other aspect of things.

    I do wonder if the press has over-hyped what the researchers are saying, though. Some of what they are saying is that the press has often over-hyped the stereotype threat research, claiming that stereotype is the only factor playing a role in the maths gender differences. This is clearly true but not a criticism of the research. Some of it is that they think we need people to do exact replications of the studies before we should accept their claims because “extraordinary evidence” is needed. I don’t know whether this is true or not, but it’s much weaker than the “stereotype threat explanation debunked” stuff we’re reading in the press.

    Gotta run…

  14. They also discuss studies in which women’s performance is shown to be affected by whether they’re in a threat-provoking situation. They note that these studies often don’t include male controls, and so one can’t draw conclusions about gender differences from the studies. True enough, but one can conclude that women’s performance is affected by whether or not they’re in a threat-provoking situation. So I don’t see this as undermining the idea that stereotype threat affects women’s performance in maths. And if that’s accepted, it would be really surprising if it didn’t play *some* role in the gender gap. (Obviously, one can’t conclude that it fully explains the gender gap.)

  15. I find this article hard to understand, maybe just because I don’t have enough background in statistics. It looks to me like one of their major claims is about the way half the replicating studies use “adjusted” math scores. They say “The essence of the problem is that the mathematics score is the outcome of interest, and adjusting for preexisting differences on this or a very similar outcome creates confounds…” See page 4. This may be a powerful point, but I’m not sure.

  16. I can’t think what universe they live in. They seem to think all these researchers think stereotype threat is the sole explanation for differences in performance. See below.

    Altogether, we hope that our review will encourage researchers
    to put more effort in testing the basic hypothesis that the gender
    gap in mathematics performance can be explained as a stereotype
    threat, put more effort in a clear characterization of data, and last,
    but not least, be more careful with characterizing the current state
    of knowledge in the scientific literature and with discussions with
    the popular press.

    With regard to the last part, let me note that some at least of what they cite as exaggerated claims do not seem such to me. Further, they compare strong statements in an abstract with outright fraud. All in all, it does look like clutching at straws by someone who has argued that there are inherited biological differences, as one of them has.

  17. thanks for posting this. Myself and a colleague are in the process of writing a short departmental presentation on evidence for IB/Stereotype Threat and its likely impact in philosophy.

    I’ve already got a solid grasp on the IB side of things but I only really know the basics with stereotype threat. If anyone has any reading suggestions for the evidence for Stereotype Threat that would be great though!

    Currently my plan is to look through the two meta-analyses cited on and put the latest one into google scholar and see who cites it. I will also, obviously, have a look through Jenny Saul’s paper again to remind myself of the stuff in there.

Comments are closed.