Trying to keep up with natural language processing tools

Turnitin responded quickly. Faced with natural language processing tools, it deployed Turnitin's AI writing detection model. One day recently I opened Turnitin through my CMS, and there was the AI detector, ready to go.

This is likely a necessary response by the long-standing plagiarism guardian. "AI" has zoomed past cut-and-paste, and tools like Turnitin are trying to keep up.

As I've said on several occasions, I use Turnitin not so much for its plagiarism-catching talents but because it is integrated into Blackboard and has an easy-to-use commenting interface that includes the ability for a teacher to create a three-minute voice response. (I wish more teachers would use voice commenting features...)

We know its AI checker is not going to be perfect, but I had to relay this brief anecdote about just how far away it appears we are.

A sharp student in my class Language Puzzles and Word Games: Issues in Modern Grammar just submitted her second short paper.

Ah, relief: The AI checker came up 0%. (I expected nothing less, of course!)

A problem, though: Her project, in line with what we're doing in the course, was a comparison of a paper she wrote in another class and how ChatGPT would have responded to a similar prompt. In her paper, she used giant chunks of ChatGPT-generated text to illustrate grammar, usage, and style differences in her work from the natural language generator.

0%. Zero. Zilch. Nada. The AI detector didn't pick up any of it.

(The text she used was not bound by quotation marks, FYI.)

No one is expecting perfection, especially at this stage (and Turnitin includes serious qualifiers about the effectiveness of its tool), but we should remain very aware that digital watchdogs likely won't catch digital text. We have to do it ourselves.

We're in the early stages of a major shift in teaching, and we're collectively engaged with on-the-fly decisions about how we're adjusting to that change. I trust my students, but teachers will invite authorial misrepresentation if we don't develop pedagogies on the front end instead of relying on cross-our-fingers detection after the papers are in.