18 thoughts on “Rutgers climate survey”

  2. So here’s something I’m really curious about, and would love to hear thoughts on from those with experience doing climate surveys. Basically – how, exactly, do you interpret the results, once you’ve got them? More specifically, how do you interpret positive “yeah, everything’s fine” results, if that’s what you end up getting?

    I mean, I can imagine what I’d do if a survey like this yielded answers that suggested there were major problems. My worry is what to do with results that more or less say that everything is fine. For starters, I’d be unsure how to interpret those results, or what they would say about my department. More insidiously, though, I’d be worried that they could be used by higher-ups to say “hey look, it’s all good – we don’t really have problems here” – even if there were real and serious problems

    Let me give some context to these worries, by way of explanation. When I was a grad student, I was in a climate that was very difficult. But I seriously doubt that I’d have reported that on a survey like this. That’s partly because I felt intense pressure not to complain, and to say that everything was fine. But it’s also because so many of the problems in my environment were *completely invisible* to me at the time. I only realized how bad things were in retrospect. At the time, I thought it was normal, and I thought I was being treated exactly how I deserved to be treated.

    None of this should by any means by taken to suggest that I don’t think that women in grad school are capable of accurately reporting their own experiences, or that self-reporting in things like climate surveys isn’t really important. It’s just that I have these lurking worries, based on my own experience, and I’m curious whether others who have thought more about the issue have things to say about it.

  3. I’m wondering what an expert in statistics would say about the distribution of factors asked about. It might be a richer survey if it were set in the context of factors about candidates choices of schools, areas of interest, amount of financial support, grade expectations, career expectations, etc. E.g., a correlation between expectations and answers would be interesting, as would a lack of correlation.

    If I did a survey like this, I’d worry that it would be professional suicide to answer negatively unless I was one of about 200 or more. Of course, that could be a reason for not including more questions, but without more questions, motivations for answer really are unclear.

    What Magicalersatz said too.

    I have never heard anyone address Rutgers hiring practices for women professors. How many women are wives of male professors? Without wanting to raise any questions of quality, I’d like to know who was picked first of the pair.

  5. Does anyone know why they didn’t ask the same questions they asked about race and gender about sexual orientation as well?

  7. Anne– it used to lead to the survey; I managed to download it before the link changed (by which I mean I think the Rutgers people must have removed access to it). I’m disturbed by the fact that questions about sexual orientation (and gender identity) weren’t asked, given that questions about race and sex were. But I’m also disturbed by the fact that I keep asking why not–here, and elsewhere, and –crickets chirping. ???

  10. Wow- I checked this link when it first went up and didn’t see any comments, and then just forgot about it…I’m really sorry about that. Blogging is still a bit weird to me. Thanks for your patience and to Jender for letting me know that there were these comments that needed responding to.

    In previous years, we have had a climate survey written by faculty, but this year we decided it made sense for the graduate students, on the theory that they were closer to the ground, devise one themselves, which they were keen to do. We have some extremely committed graduate students who have really taken on leadership roles re climate — they include Heather Demarest, Lisa Miracchi, and Michael Hicks. Look out for them as they are a force for real change in the profession. Anyhow, the survey posted here is devised wholly by the grad students. We thought it would be good to see how it worked, and judging from the data we got, it worked pretty darn well.

    I don’t really have any authority to speak for the grad student committee which devised this survey, but as a member of our climate committee, which is chaired by Holly Smith and Martha Bolton, I can say the following.
    1. We had a professional statistician from outside the university collate and analyze the results.
    2. The results were reviewed in a three-hour session, 1 hour for the whole dept, 1 hour for just faculty, 1 hour for just grad students with an eye to questions about how to be an ally. All 3 discussions were led by an outside expert (who was very good) brought in to help guide discussion and to interpret the results. Andy Egan was a co-director of the discussions and was great. One of the benefits of having such a protracted meeting was to reaffirm our commitment as a department to being serious about climate issues. It really helps to have a chair who is committed to and supportive of these measures (kudos to Jeff King).
    3. I might mention that this session followed a special session a couple of weeks earlier organized by Susanna Schellenberg for women grad students going on the market for gender-specific issues surrounding going on the market. This was also really useful. And that in another few weeks, we’ll be having a Women’s Community Dinner for anybody interested in what it’s like to be a woman philosopher. So climatey-type things are a fairly regular occurrence in the department.
    4. Back to the survey. The survey yielded some interesting results – I agree wholeheartedly with magicalersatz and others that if the climate is really lousy, a climate survey is likely to get anodyne false positives. That is definitely not the case at Rutgers but I can see that climate surveys might be of very limited use especially in small departments with bad climates. Our graduate students are quite frankly really incredible – highly thoughtful, outspoken, committed to making the climate better, and so I think our climate survey was useful for Rutgers.
    5. Various improvements to the survey came up at the meetings, including the ones suggested in the thread comments. There was a discussion about why questions about sexual orientation weren’t asked, and it seems like it was an unfortunate oversight that will be corrected for next time ’round. We’ll do better next time.
    6. Believe me, issues about hiring more women came up, and it is something on everybody’s radar even without special climate discussions.
    7. One issue that came up in our discussions and about which I personally feel rather strongly is the role of men in changing the climate for women in the profession. I think there is an expectation among women and men that women are supposed to spend their time on climate issues when it seems to me that men should be taking an equal leadership role in these issues. Andy Egan co-led our climate discussion, and Mike Hicks is one of our graduate representatives on our climate committee. My own view is that a key way forward is to get men involved. Also, the women who do the yeoman’s work on this need to be recognized. That’s why I keep dropping names in this post.
    8. On the website. We just switched to our new website literally a few days ago! There are kinks — Jerry Fodor is listed as as Associate Professor — and we are aware that the Climate button has gone missing and all the great data there has been spread out in some weird way. It’s a huge amount of work to get all the bits in place and certain people in the department are working tirelessly to accomplish it, but the Climate button is a top priority. There will be lots of links to the climate data on the final webpage – not just a separate button but it will also appear under the graduate program, etc. We needed it to go live without being perfect because of certain other pressures too boring to mention.
    9. Thanks to everyone for taking the time to offer their comments on the survey – they are all incredibly helpful and we will definitely take note for the next round.

  11. A couple of things to follow up on Ruth’s comments–
    (1) We do worry about the usefulness of the survey if respondents feel that their responses could be used to single them out. So in releasing the results within the department, we’ve been careful to remove any information that could identify anyone. The full results are on record with the climate committee and will allow us to target areas for improvement and track long-term changes.

    It seemed like people felt generally very comfortable answering–we think this both from the sorts of reports we got and the informal discussions we’ve had within the grad student body. Next time we give this, people will have seen how we distribute the results, and hopefully that will increase their expectation of anonymity.

    (2) We’re having a discussion about what sorts of extra questions to add, with the goal of gaining more info about the status of respondents (their year, maybe their grade expectations, finances, school, etc), but we want to insure the anonymity of our graduate students. So there’s an interesting balance we’re trying to strike between maximizing information and maximizing anonymity.

    (3) We’re definitely going to add a section on sexual orientation and gender identity on the next survey. We got a few comments on that on the survey itself and in discussion.

  12. Thanks so much, Ruth and Mike! And thanks also for doing all the work of both making the survey and sharing it. Very few departments are doing this sort of thing and it’s immensely valuable that you’re willing to come and discuss your experiences with it.

    I should also say something about Magical Ersatz’s concerns: One of the problems is quite a general one that there’s no good way around– people may not tell the truth about a very bad environment. And that’s something to be aware of before confidently patting oneself on the back and thinking everything’s perfect. But it doesn’t follow that nothing useful can be learned, and it sounds like they are getting detailed and useful stuff from those surveyed. A further problem is that of not recognising problems as problems. Like you, I would have failed to notice some rather large problems as a student. But as I recall their questions were nicely constructed in such a way that I can see my answers would have revealed problems even if I was too benighted to see them myself.

    Thanks again for being willing to go out on a limb by sharing all this!

  13. Thanks, Jender! I posted our survey in the hope of helping people from other departments, from whom I get emails about this, to have access to yet another template from which to fashion their own.

    I want to underscore agreement with both magicalersatz and Jender: these climate surveys can be totally useless and even harmful especially in small departments with bad climates where speech about climate is probably on ice. If the responses are of a largely uniform, unenthusiastic ‘everything’s fine’, with nobody offering details in open-ended questions that might come at the end of a survey, that is, to my mind, a bad sign, and another way to tackle climate issues has to be found.

  14. All: Thanks for your ideas about ways to improve graduate climate surveys. I’m sorry that you’ve recently had troubles accessing the Rutgers Climate page. As Ruth reports, we’ve just shifted from the old webpage to a new one, and not everything made the transition as well as we hoped. The Climate page will soon become more visible, but in the meantime you can access it from the homepage under “Resources for Graduate Students” and “Resources for Prospective Graduate Students.”

  15. For a new thread– I see why everyone is saying (and I think it’s true) that in a really horrific/icy climate, surveys would be useless. Right. But then what to do, in such a situation… esp. when denial about the hostile work environment is part of/enables the hostile/icy climate?

  16. I am not sure that in an icy climate surveys are useless; the point was that an inexperienced grad student who hadn’t known anything better might not have the capacity to articulate what was wrong, or even see it as an error in the environment. But a good pollster ought to be able to get at questions that might uncover that.

    I can’t tell whether the Rutger’s survey questions were done entirely in house or not, but they should have had outside expert advice on the questions to ask.

    At my university about 12 or so years ago, we had a survey in a very icy climate. The female faculty knew they were not getting the rewards the men were, but I don’t think that anyone had any knowledge of implicit bias. Equally, though we all knew about sexual harassment, the idea of a hostile workplace with demeaning comments was still being worked out. So a lot of faculty were inclined to blame themselves, etc. But the survey got it anyway. There are all sorts of questions that can be asked about expectations, future plans, etc, that can reveal a problem in the present.

  17. Anonymous for this: I think it’s really hard to say things in general about what can be done if it’s clear that a climate survey won’t uncover what needs to be uncovered. What might work depends on specific facts of the situation. I know there are a lot of women with ideas about how to deal with various types of situation and so my first thought is to take your case offline and contact someone experienced who you trust with whom you can share details. If you don’t know of such a person, I can help you get in touch with some awesome folks who will protect your identity and give you some good ideas. Good for you that you are taking a stand and want to do something.

  18. I think doing a survey is very helpful. It gets a conversation going and gives everyone in the department a shared point of reference. I helped to write and run one last year here at CU. We based the survey questions on the Rutgers survey, but expanded a lot of them, including expansions to ask about race, religion, sexuality, and so on. I built the survey in Qualtrics, which I would highly recommend. It’s very professional, easy to use, and gives you some powerful analytic tools.

Comments are closed.