Mixed blessing? Machine grading

from the NY Times:

EdX, the nonprofit enterprise founded by Harvard and the Massachusetts Institute of Technology to offer courses on the Internet, has just introduced such a system and will make its automated software available free on the Web to any institution that wants to use it. The software uses artificial intelligence to grade student essays and short written answers, freeing professors for other tasks.

To be sure, the artificial intelligence doing the grading is far better than the old computational style. But one might well want to see much more before one lets it grade.

The comments are very interesting because they illustrate a certain dislike of change, I think. The news doesn’t have to lead to doom and gloom.

Athene Donald on unconscious bias

A fantastic post.. The newest bit to me was this one:

The most recent study I came across, entitled The Matilda Effect in Science Communication: An Experiment on Gender Bias in Publication Quality Perceptions and Collaboration Interest looked at the responses of ‘243 young communication scholars’ when asked to rate some (carefully manipulated) conference abstracts. (The Matilda effect was a phrase coined by Margaret Rossiter 20 years ago to describe the systematic underrecognition of women in science.) The abstracts’ topics and authors were varied to see how the readers reacted and to test a series of hypotheses relying on ‘role congruity’. This theory says that a group (or in this case, an abstract) will be positively evaluated when its characteristics are recognized as aligning with that group’s typical social roles. So a paper written by women about a subject ‘appropriate’ to their gender, such as the effect of media on children (remember this was a project involving science communicators), will be more highly rated than one written by women on an ‘inappropriate’ topic such as political communication.

Their hypotheses were largely borne out; on average the papers written by ‘men’ were perceived as of higher quality than those written by ‘women’, and even more so if stereotypically male topics were being written about. The respondents were also more likely to want to collaborate with the males on stereotypically ‘male’ topics and with the women on those topics associated with women. These trends were the same irrespective of the respondents own gender. The differences in evaluations were not large, but as earlier studies have shown, small effects multiply up over time; this is true of salaries and it is true of less tangible attributes such as recognition or collaboration opportunities.

But do check out the whole post. There’s also a great example of some biased letter-writing. (Thanks, J!)