2 thoughts on “No, bias against women in science not disproven

  1. I think this article is unnecessarily condescending and, at a point, unfair to W&C’s paper. Specifically their second point (“The superstar problem”) doesn’t take into account the fourth experiment in W&C paper, which replicates their finding in the first experiment with CV’s where no mention is made of how talented the hypothetical candidates are.

  2. I don’t understand what makes the article great.

    Some of the following might just be repeating what Edouard Machery said on Facebook, and some of this might just be repeating what I said when this Ceci & Williams article came out: https://feministphilosophers.wordpress.com/2015/04/14/new-study-shows-preference-for-women/#comment-139180

    Now, onto the Williams and Smith article in The Chronicle that was linked…

    Re (1): This isn’t a *methodological* critique. If the authors truly believe this is a *methodological* critique (as when they counted five), there is a serious problem with what they count as methodology. I think it’d be helpful to separate Ceci & Williams and this particular paper that they authored. I think they’ve authored some problematic popular pieces (i.e. I don’t think bias against women is a myth), but that should not count against this particular paper which was scrutinized far more than typical. (I think they had an unusual number of referees and an independent stat audit.)

    Re (2): It is plausible to claim that C&W’s effect is limited to superstars, since they only tested for superstars. It is an interesting hypothesis that the effect would not be found with superstars, using the same design. But it’s not enough to armchair this interesting hypothesis. One has to actually test it. More follow-up studies would be great. At the same time, it’s important to note that to point out a *potential* restriction of robustness — one that has not been proven — is not a methodological critique in the sense of they did a bad experiment. No one study will conclusively establish any claim without any confounds. Often, there are multiple reasonable choices in designing any particular study. BTW, I would like to see this perfectly reasonable recognition of a study’s limitation to be systematically applied to other studies in this vicinity.

    Re (3): Again, this is a plausible alternative hypothesis of the results. Again, however, it is only an armchair speculation, and not something that has actually been proven. Indeed, Machery pointed out that the supplement materials included where C&W tested for this hypothesis and did not find support for it. The social desirability hypothesis is also incompatible with the actuarial data that C&W cite. (These points were also already pointed out in that comment thread on the Brownstein post.) BTW, Again, I’d like to see this perfectly reasonable recognition of a study’s limitation to be systematically applied to other studies in this vicinity.

    Re (4): It is incredibly bizarre to claim that a demographic variable like gender of the study respondent should be treated as a separate condition. (BTW, I don’t think in past studies, e.g. Steinpreis et al 1999, this demographic variable has been shown to make a difference.) I agree C&W can do more to test the independent contributions of different variables and their interactions, but that also wasn’t the question they were asking. FWIW I thought this was the best methodological critique they made, which definitely calls for follow-up studies.

    Re (5): I absolutely agree that C&W’s claim of lifestyle choice is silly. But this isn’t a *methodological* critique. Indeed, it shows that even someone who believes that women are generally discriminated in academia (like me) can still try to take on board this C&W study to an extent. (Of course, again, no single study should persuade us too much.) It’s not enough to know that there exists gender discrimination in academia; one wants to know where intervention can be made most effectively. C&W suggests it’s not at the hiring stage. Suppose you believe that, you can still believe that women are generally discriminated due to a number of other reasons, such as inadequate parental leave and general social gender norms which make it difficult for women to have kids and strong academic careers. It’s perfectly reasonable to point out this alternate possibility in the grand scheme of things without it being a *methodological* critique of this C&W study making a limited point.

    tl;dr: I think the authors are confusing their assessment of C&W (and their rhetoric) with their assessment of this study; only some of their critiques are actually methodological; and even those simply call for more follow-up studies (which is completely standard in social scientific inquiry) rather than for dismissing this data point (as “mud”) altogether.

    Those critiques aside, I’m in complete agreement with Williams and Smith on this paragraph:

    “Why smart people say silly things. Why did so many smart people — both those who did the study and those who read it — overlook these serious methodological issues? It’s called confirmation bias. People tend not to interrogate findings that confirm what they already believe.”

Comments are closed.