Help! Assessing the Gendered Conference Campaign

There seems to be anecdotal evidence that the gendered conference campaign (see pages above) has led to more women being invited as keynote speakers at conferences. How might one get a quantitative assessment of its effectiveness?

That might look easy, but it looks hard to me. Presumably we should start by counting and comparing. But that supposes determining what to count, and that’s less clear. Of course, it would be conferences and invited speakers, but how might one select groups of conferences to be enlighteningly compared?

It would be great to get some suggestions!

17 thoughts on “Help! Assessing the Gendered Conference Campaign

  1. Rather than trying to lump all philosophy conferences and taking a percentage (which might not give useful data due to variations in the number/types of conferences included in the sample), why not focus on 20 or 30 annual conferences that have been around for a long time? The thought would be that you could run a study comparing this year to prior years for each individual conference. The result would be something similar to “same store sales” statistics in the retail economy.

  2. I guess I am not sure I understand – how would we know what role if any the GCC had played from doing Matt’s comparative test?

    This wouldn’t give a percentage of any kind, nor would it catch everyone since not everyone is conscious of their motives or willing to fess up to them, but couldn’t someone actually ask conference organizers if the campaign had made any difference to how they design speaker lists, and ask for some details? In other words, it seems to me useful qualitative data may be more obtainable than useful quantitative data here.

  3. The Classical Model of Science (11-13 January 2007, Amsterdam): Invited speakers (5 people): 0% female; 100% male; Scientific committee (11 people): 18% female 82% male

    The Classical Model of Science II (3-5 August 2011, Amsterdam) Invited speakers (8 people): 50% female; 50% male; Scientific committee (26 people): 50% female; 50% male.

    I was chief organiser in both cases: it did make an impact. :)

  4. I used to run the London Logic and Metaphysics Forum 2007-2011. From 2007-2009 we had 19 speakers only one of which was a woman, so that’s 5%. We did invite one other female speaker twice, but she couldn’t come either time. During 2009-2011 we made an effort to invite female speakers as a result of the gendered conference campaign. In those two years, 4 of 15 speakers were women, so that’s 27%. So a definite improvement. I’m not sure what I think would be an ideal percentage given the fact that women make up around about 20% of the profession, and I didn’t set an explcit target.

    I am also putting together an edited volume and have been consciously trying to maintain a respectable gender balance. This has been difficult, however. One problem that I’ve experienced is that some of the women I have invited have refused, partly on the grounds that there are actively discouraged from contributing to edited volumes. These were yound philosophers at leading US faculties.

  5. Rebecca, your comment reminds me that I want to recognize explicitly that there’s been a lot of feminist action on blogs in the last 6 or 7 years. Still, if one took a group of annual conferences from 2006 and 2007 along with another from 2010 and 2011, we might see something of a trend. Then it’s at least a reasonable hypothesis, I think, that the GCC has contributed to it. That doesn’t preclude other hypotheses being also reasonable, and other factors making contributions.

    Since I’m looking at a fairly technical social science journal, I’m thinking self-reports go into the “anecdotal evidence” category. Without some sort of sampling, I doubt the self-reports would cut much ice, since people do misreport motives, etc, and can even report they’ve changed the way they are acting when they haven’t.

    With a relative small sample, maybe a suggestive trend plus anecdotal evidence is all that’s attainable.

    Arianna Betti and Lee Walters, thanks so much for the data, anecdotal and quantitative.

    LW, I keep hoping to meet you when I visit Somerville, though my visits have been very brief. I’m planning on a longer stay in Trinity Term this year.

  6. It’s a reasonable hypothesis but surely no more than that. There are obvious possible common causes, such as increased awareness of gender inequities in the profession, etc. I am not undercutting the role of the GCC at all – I just don’t want to pretend that that kind of data would be scientific support for that hypothesis in particular.

    I wonder if you count all of qualitative social science as ‘andedote’? There are methods for being systematic about interviewing and interpreting people, of course. And lots of qualitative work operates with a really small sample. I guess I don’t believe in a bright line between anecdote and qualitative data but I am not so ready dismiss it all as anecdotal.

  7. Rebecca, I guess there are different ways to understand your suggestion from above, “but couldn’t someone actually ask conference organizers if the campaign had made any difference to how they design speaker lists, and ask for some details?” If that does describe what produces some of the very good qualitative social science, then I think it is a bit incomplete.

    I don’t know if any good qualitative social science is done by untrained interviewers and observers, but obviously there are significant traps one can get into in trying to elicit information from agents. So I’d expect we’d need to add in that the ‘someone’ had academic training in interviewing people (or formulating written interview questions), which might well also eliminate most of those interested in the GCC’s results.

    In my limited experience, doing something like drawing up some questionaire to be send around is a very lengthy task, if it is to be done well, and then typically only a small percentage of academics respond to such things, thus ensuing that one doesn’t have statistically significant results.

  8. Well I didn’t have all the specifics in mind, but yeah, my idea was that it would likely be more productive to study this qualitatively than quantitatively, and that would require some careful attention to the sample and structured questioning, by someone in a position to do it right.

    Surely one of these many x-philers running around the disciple ought to be able and willing to lend us the right kind of social scientist? :)

  9. I think the x-phils is a great idea, especially if they rent their expertise out, but do see the para I tacked on to my previous comment. It is a lenthy task, and it is extremely difficult to get academics to reply to such things. I was once involved in providing a survey at a profoundly unhappy university, and I think we got a 55 or 60% return, with 89% of them prepared to damn the provost, whcih gives one a sense of the high emotions that produced that apparently remarkable turn out.

  10. I don’t believe that the notion of statistical significance applies to truly qualitative work. It’s a quantitative notion. There’s no control group here, so I don’t even know what statistical significance would mean in this context. The idea would not to be to run a quantitatively assessable survey, but to find people willing to interview some conference organizers about what, if anything, had changed their practices with respect to gender and invitations.

  11. I am not sure. I have certainly seen a situation in which one might want to launch interviews, but have been told that the sample was too small to be statistically significant. I.e., for all we know, the phenomena are just accidental.

  12. I think we should get second, third and fourth opinions from other social scientists on that, Anne! I’ve co-authored with social scientists whose data set was extremely small. Let me ask her about good possible procedures for a qualitative study, because I think Rebecca is right that qualitative is the way to go.

  13. Profbigk, I certainly didn’t say that all small data sets are useless. I instead was contesting the idea that ‘not statistically relevant’ has no application to any qualitative study. I think a really good qualitative look at some conference organizers could be very illuminating about a lot of things. i would love to hear your ideas on this. I think it could be a great project, and if we can fund it, it would be wonderful.

    I suspect that we are confusing or not separating goals here. Behind my initial question is a very specific hypothesis about combating bias. It’s based on neuroscientific work by ann Harvey and Read Montague, and I’m working with a former colleague of mine who is now in Read’s lab. Of course, the idea that there’s a specific mechanism making the GCC effective falls flat without the causal claim that it is having an effect. I’m also, as you know, floating this idea in a proposal to a technical journal, but in fact I floated it today, so the issue is now less urgent.

  14. Anne, I honestly don’t have a clue still how you are using the word ‘statistically significant’ here. I literally can’t imagine what it means for qualitative interviews to be ‘statistically significant’. What are the two sets of results being compared? What is the dependent variable? The whole idea of measuring whether the GCC has a ‘statistically significant’ effect only gets off the ground only if we have or can simulate a control group of conferences whose organizers have not been exposed to the campaign, and no one here has proposed that and I have no idea how you’d do it. And in the qualitative interview case, again, the notion is just a category mistake – we would not be quantitatively measuring effects, so I can’t imagine what the term could possibly mean in this context.

    You say “I was instead contesting the idea that ‘not statistically relevant’ has no application to any qualitative study. I think a really good qualitative look at some conference organizers could be very illuminating”. First of all, you switched from ‘statistically significant’ – a technical term that you used several times that just gets no grip here – to ‘statistically relevant’, which is not a technical term and I don’t know what it means. What is statistical relevance? Relevant to what? What statistics? Second, the second sentence is a non sequitur – just because it illuminates things (I agree) doesn’t mean it tells us anything statistically significant or statistically relevant, whatever that would mean in this context.

Comments are closed.