Five Reasons Why Computers Should Not Assess Writing

1. Computers Do Not Read With Understanding. By using algorithms based on components like sentence length, sentence structure, word frequency, computers substitute correlation for understanding. We write for the formula in the same way that we write for the scorer for the prompts found in state testing. As Robert B. Shephard commented after a Diane Ravitch piece about computers scoring student writing:

"Consider Noam Chomsky's famous sentence--Colorless green ideas sleep furiously. This would be rated very readable by most readability systems. But it is utter nonsense."

2. Context is Human. Imagine the following passage from Ernest Hemingway's "In Another Country" monitored by the algorithm:

In the fall the war was always there, but we did not go to it any more. It was cold in the fall in Milan and the dark came very early. Then the electric lights came on, and it was pleasant along the streets looking in the windows. There was much game hanging outside the shops, and the snow powdered in the fur of the foxes and the wind blew their tails. The deer hung stiff and heavy and empty, and small birds blew in the wind and the wind turned their feathers. It was a cold fall and the wind came down from the mountains.

The sentence structure is repetitive or, thought another way, uses parallel structure to create a rhythm and craft a tone. Similarly, the use of the conjunction "and" appears seven times in six lines of text; the preposition "in" appears six times, and the noun "wind" appears four times in the span of thirty-three words.

Writing this paragraph above, I tried to think like the computer--as a writing teacher this was unnatural for me. My eye and ear was drawn to the beauty of the wind in the tail of the foxes and the crisp imagery of the setting. For all of the simplicity of the individual words and the frequency with which some appear, does not make any of it an error. The context of the passage takes those individual pieces and moves back, to allow our sensibilities to focus and regard the piece as a whole. No human writing teacher examines and criticizes word for word nor should one.

3. Writing is an Art. Imagine an algorithm written to examine the combination of mechanics used in art, such as brush stroke. Seurat's dots on a canvas might be assessed point by point--frequency, shape, thickness, and color--all could be criticized I suppose. But why would you? If you were teaching someone to paint and to develop their skills we don't burn our sweat over brush strokes. We encourage the artist to learn how to say something with those brush strokes.

Would we try to force every painter to work with the same brush strokes?

4. Confined to the Writing Prompt. While the same issue binds students on state testing, using scoring software muddles the most important part of growing into a good writer--writing about what matters to us. Students can not feed any piece of writing into the algorithm. It must be on the stated topic. By using this system over and over again we are feeding the beast--teaching to the test. And writing remains something that is just done for school or just done for testing.

Being confined to the writing prompt on a weekly basis suffocates any hope of developing a young writer. While, I agree, students need to learn to write to a prompt, a balance must be found. And when students write about subjects not found in the scoring algorithm, what is to be done?

Read them ourselves?

5. The Art of Conferring is Obliterated.
Bad writing is necessary, and we all do it. We have to do a lot of bad writing in order to mine and polish the good stuff. But we will never know what promise a piece has when all we receive from computer-generated scoring is a print out of pre-determined errors.

Georges Seurat - Peasant Woman Seated in the Grass

No conversation takes place.

A student must be able to look at the negative comments and try to figure out where they apply. Nothing is highlighted. No questions are asked of the student.

Ostensibly, the print out would be carried back to the teacher or a peer so that the two might engage in a conference about the piece. In either case, the essay still needs to be read so that a conversation about the computer's tastes can occur. And even if that were the case, what are we talking about? How to mold our writing to satisfy an algorithm? So my piece looks and sounds likes yours?

Worse, my fear is that computer-generated scoring means some may use it to replace ever engaging with students and their writing at all. The tool becomes deemed a time-saver and replaces human skill and compassion. At the risk of offending people, if you are teaching writing and seek ways to avoid reading papers and discussing essays with students then you are in the wrong classroom. If you lean on "time" as the reason why this computerized method of assessing writing then you need to reassess how you spend your time in the classroom.

This is not to say that anyone is a bad teacher, but the earmarks of bad teaching are all over the use of computer-scored writing. The worst possible scenario is that this tool becomes a hoop for students to jump through and all of the necessary and influential steps to becoming a better writer are lost.

Writing becomes a chore for our students. They write to the judge...Big Blue...not to an audience, and never for themselves.

In the process, we all stop reading and we all stop talking.

And when that happens, they stop writing.