Editorial: Machine testing of writing goes too far

Can a machine grade a writing test? Surprisingly, it’s a subject of some dispute, with dueling studies, impassioned statements and an online petition.

A critical driver is money. As tests are being developed for the new Common Core K-12 educational standards, some see computer scoring as critical in keeping costs manageable. The issue also arises with online university classes.

We’re great fans of saving money, but this one goes too far.

Supporters argue great strides have been made in using artificial intelligence to grade writing, and a study last year from the University of Akron concluded machine scoring was “capable of producing scores similar to human scores for extended-response writing items ...”

Massachusetts Institute of Technology researcher Les Perelman critiqued that study in March of this year, attacking its methodology and complaining that it was not published in a peer-reviewed journal. He is part of a group calling itself Professionals Against Machine Scoring of Student Essays in High-Stakes Assessment, which has launched an online petition opposing machine testing.

Two of the groups developing Common Core tests told Education Week they’re studying the use of machine scoring in their pilot tests and will await those results before making decisions.

A less controversial use of the technology is not for testing, but to give students immediate feedback and allow them to learn by revising their work. It is considered successful at judging structure and grammar, less so for organization, use of evidence, or quality of argument.

Quick response surely has value, although even here the support has a large economic component: Big classes prevent teachers from giving timely, thorough critiques. The machines’ instant feedback helps compensate for that.

Using machines to replace teachers is risky business, whether for high-stakes tests or for learning to write. Our world is increasingly impersonal and automated, but bringing those “efficiencies” into the classroom could have far more long-term and damaging results. Technology can be an important tool for teachers, but it can’t replace them.