Stuart S. Taylor thinks they might be, as Susan Svrluga reports over at WaPo. I really only have about five minutes to put this post up — so I’ll let readers respond more thoroughly in the comments but, immediately, this part of what Taylor said struck me as something in need of corrective comment:

[T]o resolve any doubt that the respondents were far from representative of the nation’s college students, consider the facts buried in Tables 3-2 and 6-1 of the AAU survey.

These tables indicate that about 2.2 percent of female respondents said they had reported to their schools that they had been penetrated without consent (including rape) since entering college. If extrapolated to the roughly 10 million female college student population nationwide, this would come to about 220,000 student reports to universities alleging forced sex over (to be conservative) five years, or about 44,000 reports per year.

But this would be almost nine times the total number of students (just over 5,000) who reported sexual assaults of any kind to their universities in 2013, the most recent data available, according to the reports that universities must submit to the federal government under the Clery Act.

*You absolutely cannot rely on the numbers reported under the Clery Act if what you want to know is how many sexual assaults are reported to universities and colleges full stop.* Firstly, there’s a question about the extent to which institutions comply with the Clery Act in the first place (hence the push for increased fines as a consequence of violation in the Campus Accountability and Safety Act, and increased scrutiny under the Campus SaVE Act). Secondly, and possibly more significantly in terms of numbers, there is a limit as to which reports of assault need to also be reported under the Clery Act. If an assault happened off-campus, if it was not reported to campus security personnel (e.g., campus police), it may not be reflected in a school’s Clery report — even if it was reported to the university in other ways (e.g., a Title IX office, student disciplinary office, etc.).

### Like this:

Like Loading...

*Related*

Three quick thoughts:

1) those are well-taken points about Clery Act reliability… still, a factor of nine is a huge discrepancy, bigger than would naturally seem to be covered by the factors you list (except, perhaps, very systematic noncompliance). It does seem to at least prima facie support the theory that the very low response rate to the survey creates an overreporting effect. And it is odd, methodologically, that the AAU survey doesn’t engage with the issue (assuming Taylor’s right about that; I haven’t checked).

2) Taylor notes something I’d noticed myself when I read the AAU survey report: that report explicitly calls out the fact that it’s misleading to quote “one in four” or “one in five” statistics, and yet that was exactly the way it sw reported in the press. That’s a bit depressing as an illustration of how the media engages with data.

3) The title of this blog post looks as if “hysterical” is a direct quote, but I don’t think the word appears anywhere in Taylor’s article. (He does refer to an “era of hysteria”.)

Why is there this data collection and analysis problem in the first place? Universities have faculty that are talented in data collection and analysis. Why can’t more universities collect good data on the safety of their own students that are vetted by faculty who will ensure the studies meet the right standards?

Commenter Jason Sullivan writes:

“This piece is actually chock full of factual errors, affecting every single assertion. Here are a couple:

1) That 2.2% number does not appear anywhere. Not only do the two tables mentioned not provide enough information to derive such a number, I can’t reproduce that number even making common mistaken assumptions. Fortunately or unfortunately, the number is irrelevant. These reports include victims who reported to the police department that they were raped by a non-student off-campus. These reports include reports to other agencies that are not required to report to the university. In other words, they include lots and lots of reports that wouldn’t count toward Clery Act reporting. The comparison is simply meaningless.

2) The 23.7% number, as mentioned in this piece, refers to assaults by physical force or incapacitation. The survey did ask about tactics such as “promised rewards,” threats, and absence of ongoing affirmative consent. These tactics are NOT counted as physical force and incapacitation. They are Not included in the 23.7%. Incidents involving only coercion or absence of affirmative consent are reported as separate numbers.”

Taylor says: “Even the AAU acknowledged that the 150,000 students who responded to the electronic questionnaire were more likely to be victims of sexual assault than the 650,000 who ignored it because “non-victims may have been less likely to participate.””

So, it does seem the report engaged with this issue, at least to that extent. But it is also true that Taylor’s response has a number of factual issues.

Two comments:

Surveys are now being conducted constantly on this issue, such as those conducted by Prof Jennifer Freyd at the U of Oregon, that contain none of the alleged errors of the AAU survey. They all conclude the same things about the depth of the problem. No matter how you assess this problem you come up with pretty much the same results.

The backlash against the concern with campus sexual assault, from people like Taylor, strikes me as diversion: suppose your campus had constant racial incidents, or anti-semitic incidents. Would we really be arguing about precisely what percentage of the effected group was subjected to this? Or would we focus on the adequacy of our measures to prevent any of it?

@Cheyney Ryan:

The most serious problem with (drawing population-size conclusions from) the AAU survey seemed to me to be the very low response rate (and the lack of any good reason to assume the decision to respond is uncorrelated with abuse history). Wasn’t that true for the Oregon study too?

(I agree that the exact rate – whether it’s one in five or one in twenty – isn’t really that salient to the policy questions. But that criticism cuts both ways – it wouldn’t be epistemically responsible to highlight very striking figures like “one in five” in making the case for urgent action, and then dismiss criticisms of those numbers as irrelevant.)

Cheyney: the surveys don’t ALL conclude the same thing. Here is a survey done by the University of Kentucky that ends up with 5%. If you read this short piece, you’ll see serious differences in how the survey was conducted that, in my opinion, lend a great deal of credence and legitimacy to its results. https://www.insidehighered.com/news/2015/09/01/survey-5-percent-kentucky-students-assaulted-past-year

Anonymous, note that the Kentucky survey you cite concludes 5% were assaulted *in the past year* — that’s a different question from the number of students who were assaulted during their college career.

Thanks all for your comments! David, I’m more familiar with the Oregon survey than I am with the AAU survey (haven’t had time to read through the report yet) — but from reports on the AAU report, the Oregon one seems much better constructed to me. In order to mitigate self-selection bias, they sent email invitations to participate in the survey to 5,000 randomly selected student email accounts (with some controls to try to get a representative portion of the student body). They offered an incentive for completing the survey (I think it was being entered in a drawing for Amazon gift cards), and most importantly, students did not know the subject of the survey until after they had already signed up to complete it.

@kathryn: a 5% assault rate per year doesn’t mean that over 4 yrs that number for a given person is then 20%. It’s not cumulative in that way. For example, if I had a 5% chance of winning the lottery, my odds would be no greater from year to year. So in year 2, I wouldn’t have a 10% chance–and in year 20 I would not be guaranteed to win with a 100% chance.

Noetika’s remark on the Oregon survey is accurate. There was no possibility of self-selection bias in how it was done. My understanding of the Kentucky survey squares with Kathryn Pogan: the Oregon study, and others, look at experiences over a four year time frame, not one year. To David: personally, I think the suggestion that a response is needed because there is a new massive crisis is mistaken, so I would agree on that. I became involved with this issue 7 or 8 years ago when no studies were being done, and it struck me than that action was needed. I did find by just talking with my students that the problem was much deeper than I realized. Example: the first time I discussed it with a class of mine in the Oregon philosophy dept, four people came to my office hours that week to recount sexual harassment stories in the department. (I eventually left the department because of its attempts to cover up these problems.)

FYI –

Here is an article by Jennifer Freyd, the leading researcher in this area, addressing questions about the AAU survey, especially the issue of response rate:

http://www.huffingtonpost.com/jennifer-j-freyd/examining-denial-tactics-were-victims-overrepresented-in-the-aau-survey-of-sexual-violence-on-college-campuses_b_8216008.html

Hi Anon — I’m not sure why you thought I was suggesting that is how odds work. My point was that the survey Anonymous was citing is asking a different question, and so it wouldn’t be accurate (in the relevant sense) to say that it reached a different conclusion. (As an aside, I don’t think this is relevant to the question at hand, but I don’t think the lottery analogy helps the point: A particular individuals odds of winning the lottery are not cumulative year to year, but the number of times the lottery will be won, is — not relevant, though, obviously, since there is no set number of assaults that take place in a given year)

*Oops — I meant to say there is no set number of assaults that need take place in a given year.

Noetika/Cheyney Ryan re Oregon: thanks, that’s helpful.I’ve now had a (very quick) look at the survey and it does seem much better designed. (Though Cheyney Ryan’s “no possibility of self-selection bias” seems overstated: Jennifer Freyd herself says “Self-selection is nonetheless a factor to consider as a potential threat to representativeness. We will be comparing our sample of participants with the underlying population on demographic variables.”)

Anon @10: “a 5% assault rate per year doesn’t mean that over 4 yrs that number for a given person is then 20%. It’s not cumulative in that way.”

It’s damn near cumulative in that way. Assuming no correlation between your chance of the event happening to you in each year (clearly true for the lottery, but to be fair, contestable for assault), the chance of it happening at some time in N years is 1 – (1-0.05)^N. To first approximation that’s 0.05 N. (That approximation breaks down when 0.05 N gets close to 1 – hence you never get a 100% chance of winning the lottery.

David – Agree, ‘no possibility’ is too strong–anything’s possible. In the Oregon survey, to repeat, respondents agreed to participate before knowing the subject of the survey. Once told, they had the option to opt out; an insignificant number did. So it’s hard to see how the survey could have invited some people and not other from participating, unless there is bias in the whole matter of who agrees to answer surveys. I think the latter is what Freyd remains attentive to, as she should. But as she argues in her piece, there is no a priori reason to think something like self-selection would skew things one way or the other.

@David are you saying that if student x has a 5% chance of being assaulted each year, then by the end of his/her 4 years their chance of an assault is 20%? If a student attends college for 10 years, will they then have a 50% chance?

Let’s say you have a 50% chance of landing on heads when you flip a coin. No matter how many times you flip the coin, you will always ONLY have a 50% shot. Or. let’s say you have a die. Each time you roll it you have a 16% chance of landing on 6. No matter how many times you roll it, you still only have a 16% chance.

Admittedly stats aren’t my strength, so it’s completely possible I’ve misunderstood how to do the numbers.

Anonymous, if you flip a coin 5 times, it’s true that the odds of it landing heads on any try is .5 — but note, the more times you try, the more opportunities you have for it to land heads. If you flip a coin five times, it doesn’t follow from the fact that you had .5 odds on each try that you only have a .5 odds of it having landed heads just one of those five times (think about how if that were true, it would mean you have a .5 odds of it’s landing tails four times out of five).

If you roll a die once, you have a 5/6, i.e. 1 – 1/6, chance of NOT getting 6.

If you roll it twice, you then have a 5/6 x 5/6 chance of not getting 6 on EITHER roll.

If you roll it N times, you have a (5/6) to the power N, (5/6)^N, chance of not getting 6 on ANY of the N rolls.

So if you roll it N times, you have a (1 – (5/6)^N) chance of getting 6 on AT LEAST ONE roll.

More generally, if you do something N times that has a probability x of having a given outcome, and where the probability of getting the outcome on a given run is independent of whether you got it on a previous run, then the probability of getting the outcome at least once is 1 – (1-x)^N.

So far this is all exact. But If Nx is much less than 1, then (1-x)^N is very close to (1-Nx). So to a good approximation 1 – (1-x)^N is 1 – (1 – Nx), or just Nx.

If you have a 5% chance per year of being assaulted, and IF whether you are assaulted in a given year is uncorrelated with whether you were assaulted in the previous year (this is a big assumption for assault, though the analogous assumption is obviously true for coins or dice), then the exact chance of being assaulted at least once after 4 years is 18.6% and 40.1%. The approximation gives 20% and 50%; it gets less accurate as Nx gets larger.

In Noetika’s example, the chance of at least one head in five flips is (1-0.5^5), which is 97%. (The approximation totally fails here, because Nx is quite large.)

There is something I didn’t quite understand in the survey results that I was wondering if others had some insight into.

One of the conditions they examined involved sexual contact under conditions of “being incapacitated because of drugs, alcohol or

being unconscious, asleep or passed out”. My initial reading of being “incapacitated by alcohol” was being unconscious or passed out. But that doesn’t make much sense, given the disjunction.

So, according to the survey, did being incapacitated by alcohol (drugs, etc.) involve being too drunk to give voluntary informed consent? Obviously, there are lots of sexual assaults that occur under such conditions. However, there’s also lots of sexual activity that occurs between drunk students on college campuses that perhaps wouldn’t normally be counted as assault. I guess I’m asking whether “having sex while drunk” was being surveyed, here – and whether those numbers were lumped together with the other stats.

This is making a reply toward Dave Wallace:

“If you roll a die once, you have a 5/6, i.e. 1 – 1/6, chance of NOT getting 6.”

This seems unlikely, given that either you will get a 6 or you will not. The most fundamental division is between sixes and zeroes.

“If you roll it twice, you then have a 5/6 x 5/6 chance of not getting 6 on EITHER roll.

If you roll it N times, you have a (5/6) to the power N, (5/6)^N, chance of not getting 6 on ANY of the N rolls.

So if you roll it N times, you have a (1 – (5/6)^N) chance of getting 6 on AT LEAST ONE roll.”

Same goes for this stuff. (I review these issues in a couple of places. Here: https://sciencedefeated.wordpress.com/2008/11/05/09999-1/ and here: https://sciencedefeated.wordpress.com/2008/11/01/probability/).

“More generally, if you do something N times that has a probability x of having a given outcome, and where the probability of getting the outcome on a given run is independent of whether you got it on a previous run, then the probability of getting the outcome at least once is 1 – (1-x)^N.”

What does it mean to do something “N” times? Which time is such that I do it?

“So far this is all exact.”

….awk

“If you have a 5% chance per year of being assaulted, and IF whether you are assaulted in a given year is uncorrelated with whether you were assaulted in the previous year (this is a big assumption for assault, though the analogous assumption is obviously true for coins or dice), then the exact chance of being assaulted at least once after 4 years is 18.6% and 40.1%. The approximation gives 20% and 50%; it gets less accurate as Nx gets larger.”

I think you’re ignoring the Principle of Compounded Additives here. Is anyone else seeing this? I’m not a formal scientist, but then this isn’t rocket science either?

Cheers,

NS

ajkreider, the question asked if survey respondents have been subject to sexual contact while “….unable to consent or stop what was happening because you were passed out, asleep or incapacitated due to drugs or alcohol”

I take it that one can be past the point of being able to offer consent without having yet passed out, and I take it that one might be unconscious for reasons other than due to the influence of drugs or alcohol, and the question was trying to get at all of the possibilities, but was not asking about sexual contact after having been influenced by drugs or alcohol at all.

I leave it to readers to decide who they want to believe on probability theory: notedscholar’s “math and science defeated” blog or the uniform view of the mathematics community for the last several centuries.

Two quick notes in reply to David Wallace, then I have to get back to work. First, I just want to point out to readers that my blog is called “Science and Math Defeated,” not “math and science defeated.” I assume this was an honest mistake, so I’m not offended!

Second and less importantly, it may be the case that the mathematics community (though I’m dubious that there is such a thing) has had a consensus against me for hundreds of years. But this was also the case with Einstein, Lamarck, Laplace, and Langmuir. So this is just to caution readers against appeals to authorities.

Best,

NS