Process Narration Videos and authentic student writing

Just like everybody else in writing classrooms nowadays, I'm dealing with AI. I like to think I'm mostly working with it, although, my sanguine teaching nature aside, I'm for sure at times working against it.

In several classes, I've overtly invited students to write about and sometimes even for the machine, as in my recent Language Puzzles and Word Games: Issues in Modern Grammar course, which includes this writing project choice (among many): "Using a prompt similar to an assignment from one of your courses, compare the text created by a generative AI chatbot/natural language generator with your own writing specifically in terms of grammar and usage."

In the Fall, I got to talking to an exceptional student, Veronica Medlock, about the drafting process in her writing. After that conversation, she quickly created and sent me a short YouTube video of the drafting changes while she composed, accompanying the video with a voice narration of the process.

It was fascinating.

We're trying to take this show on the road and have sent in a conference proposal about how such "Process Narration Videos," as we're calling them, can help promote and support authenticity in student writing assignments. Students can compose and submit these easy-to-create videos, which are built on the draft stage "mapping/tracking" that is a fundamental component of writing composition tools like Word and Google Docs, with accompanying voice narration.

Why is this so great? Of course, the English and writing teaching and learning literature has been overflowing with discussions about the problems of teaching writing in the--dare I call it?--age of AI. Teaching publications are dedicating focused space to AI (1) and instructors nearly every day receive invitations to professional development opportunities or information about new ed tech tools. Language and literacy teachers at all levels (and, of course, in many, if not all, other academic disciplines, e.g., math and programming) are dealing with the challenges of AI in teaching with a range of attitudes, approaches, tools, and philosophies. 

Some of my colleagues feel the authenticity problem is so dire that they are abandoning process-driven writing approaches and are instead returning to blue books or other non-digital, offline writing approaches. Those who have not given up have taken various often time- and energy-consuming stances and approaches: Requiring increasing amounts of informal writing, carefully scrutinizing the drafting process, leaning heavily on AI checkers in a kind of technological arms race, etc.

But as part of an overall writing pedagogy, Process Narration Videos provide a foundationally sound, pedagogically driven solution to authenticity issues in writing instruction. They reinforce good writing pedagogy, as they are grounded in writing process and ask students to reflect on their writing. They also help students see different stages of their writing, incorporate metathinking, consider rhetorical choices, and embrace informal writing, which of course is often embedded in strong writing processes.  

If our proposal gets accepted, Veronica and I will discuss the compositional background of this approach while focusing on the practical, hands-on way that it can be applied in classrooms across educational levels and curricula.

An important point is that students are often as frustrated by the prospects of other students using AI as their teachers are. So we're aiming to hit a nice balance: While Process Narration Videos can serve a dutiful--and responsible--academic process in discouraging AI-connected plagiarism, they are also simply good teaching and learning practice.

Note:

1) The topic of the most recent issue of CCC [77.1] was, as the Editors’ Introduction stated, “A Dappled, Undisciplined Response to Generative AI."

Narrating a research journey

While we were teaching... lots of questions writing instruction, especially in digital environments.

As I always say, I'm not playing cops and robbers with my students, but we have to be responsible; we have to keep sharpening the way we approach writing assignments. Much like we always should be doing, we've got to use common sense to help students do their best work. AI is just the latest temptation for them not to do so.

Sources are an issue. Since search engines have evolved, there has always been an opportunity for students to come up with wild sources: Where did some of these things come from? 

Aside from the well-established "hallucination" issues that come up when we use AI as a research tool--and remember, that type of search is not what LLMs are primarily designed for or good at--it can also dig up/create material from who knows where.

A way to help combat this is a twist on the annotated bibliography: Ask students, using plain language and perhaps images, to narrate the research path they've taken. I'll frame it like this for students:

Tell me for each source in your bibliography/references/works cited:

1. What tools or apps you used for the search, or, alternatively, who you talked to.
2. What prompts you used to generate your references: Reproduce them verbatim.
3. How much did you each resource when choosing to add it to your resource list?
• This is a shame-free zone: Did you read the abstract? Glance through it for a key word, author, or quote? 
• Reflect on what you did: This will be of interest not just to me but I imagine to you as well.
4. How did you extract information into your project?
• Did you use quotes, paraphrased material, data? 
5. How did you create the citation?

Of course, this is not an exhaustive list, and this is not necessarily something you would use only in OWCs, but because of the text-heavy environment of the OWC, a "research narrative" might be particularly useful there.

On the novelty scale, this approach is pretty tame, but it provides--in what I would argue is a constructive, reflective way--an overt learning strategy for students to describe how they are interacting with AI and search tools to generate the sources and knowledge that form the basis for much of their writing projects.

Writing instructors: Lean into our roots!

As I suppose many writing teachers are, I'm increasingly being asked about teaching writing now that AI is ever-present among us, especially teaching online, whether online writing instruction (OWI)-specific or not. My suggestions/recommendations are strongly based on my belief that we lean back into many of the long-term pedagogical strengths of Writing Studies/Composition and Rhetoric.

First off, it's not all a cosmic struggle vs. plagiarism. While authenticity is tied up with good instruction, we can't let plagiarism become a bugbear that obstructs us or distract us from fundamentally working with students to improve their writing.

We have for a long time had a lot to offer, and in OWI courses, especially asynchronous courses built on writing interactions, we can manifest much of these advantages by using, as I wrote earlier this year, a mix of approaches: 

• Process/drafting. We should continue to create learning environments in which students fundamentally engage in process-driven writing.
• Peer review. Peer-to-peer writing has many advantages, including that it creates authentic audiences for which students are responsible. Again, this means that while some (few) students might feel unabashed about putting an AI (or otherwise inauthentic) writing piece in front of a professor, they may think hard before placing such a piece in front of fellow students. In general, a lot of peer writing helps make the class a semi-public space.
• Informal writing. Students need to have many opportunities to connect writing and thinking. We can say it all we want, but we need to have students see it by giving them ways to use informal writing to build more complex projects.
• Low-stakes writing. Tying in with the above, I'm a believer in grading (almost) everything, but with the caveat that the grading be FLS: frequent and low-stakes.
• Unusual readings. As I wrote a few months ago, assign a "textual brew of unusual texts and discussion prompts." To put it simply, build assignments from texts that are less likely to be part of an LLM's database.
• Metacommentary/Reflection. Remember that us writing teachers are very good at this, with our long-entrenched embrace of portfolios, including for assessment. Last year, I wrote about how, "Metacomments push students to talk about their practice and to defend their choices." 
• Embracing the technology. Part of using metawriting and metathinking is bringing into conscious focus things around us that might otherwise be transparent. While you don't have to center your course (or even assignments) around technology, it is worth making sure technologies are not transparent: Have the students address their use of technology, even in short reflections.

These approaches and suggestions are not just Writing Studies/Composition pedagogy but are specifically how such pedagogies might manifest themselves through OWI. As those of us in the field of OWI have been saying for well over decade, at some point all writing instruction became OWI, and we cannot retreat into blue books because of technological change.  

 OWI is far from dead--long live OWI!

As AI becomes increasingly [S]ubtle

Lately, it's not often I refer back to my 2002 dissertation, "Subtle Technology and Student Writers: The Ambient Influence of Technological Myths on Communicators," but in the diss I had a core idea that's relevance might be re-emerging, a concept I described as Subtle T(echnology); I'll define it here very briefly as "a theoretical transformation digital technology might undergo that leads to complete dependence on these machines." The transformation occurs "when five interdependent traits of digital technology are fully realized: ubiquity, transparency, dependency, interconnectedness, and insubstantiality" (1).

With the explosion of AI use, I think Subtle T provides a valuable lens for viewing this technology, which I could argue has been integrating itself into our activities in ways that meet the traits above.

Of course, I was always a Writing Studies/Comp Rhet researcher and practitioner, so in the diss I described how Subtle T functioned in writing instructional scenarios.

Recently, I had a situation in a course in which I thought a student might have plagiarized language in parts of a project. I didn't think this student had malevolent intent, but some of the writing seemed inauthentic when compared with their other writing, particularly informal writing, in the course.

My course AI policy encouraged productive use of LLMs, so I was more interested in a constructive dialogue than a wrist-slapping--or worse.

When we met, I pointed out passages I felt were inauthentic, and I said while I didn't think it was material deliberately clipped from other sites or sources, it appeared to be AI-generated text. 

We had a good conversation, but the student was nonplussed as I talked about using AI and when eventually asked straight-up if they used AI tools. They told me, equally straight-up, no. The student said while they had indeed used material they found in Google searches to help develop the project, they were "just using Google," not AI.

Well, of course, if you are "just using Google" right now, the first "hit" is an "AI Overview," marked with a "sparkle icon." The student was surprised by this and told me, and after our discussion I believed it, that they didn't think of the results of Google searches as AI, despite the "sparkle icon."

This to me was another good example of how we'll have to keep up in our courses with AI. Many teachers are growing accustomed to handling the results of prompt-based AI use: We feel confident that students who use prompts and submit the subsequent text from LLMs are overtly plagiarizing. But embedded tools?--we'll have to sharpen definitions. Grammarly is one thing. AI built into Google searches is another. 

I almost feel that we might have to broaden our perspective and be careful not only of students using AI but of AI using them: Their common behaviors may be co-opted in ways that are ubiquitous, transparent, dependent, interconnected, and insubstantial. They will notice less and less.

Maybe it's time for me to renew my efforts to get that diss published. 

Note:

1) From page 1 of the diss, which I completed at Temple University. 

Audio still sounds right for student response

I'm a ways away from blogs and other writing I've done about audio--and at times, audiovisual--feedback (see References below).

In practice, though, I've been sticking to it, lately using Turnitin's built-in audio function (Turnitin itself is built into Blackboard at Drexel), which gives you a one-click option for a three-minute audio response to student assignments.

As I've described, there are many advantages to audio response, including comprehensiveness of response, time spent on task (audio response is fast), and psychological load of responding to student writing.

It strikes me that audio also may be more relevant than ever as teachers struggle with AI, because I think among the authenticity/integrity issues that we're running with AI tools into is the way they may depersonalize the student-teacher writing relationship.

When you provide voice commentary, students hear you--I intentionally emphasized both words. Will that make them feel bad, feel sorry, feel reflective about me as their instructor? I don't know. But I do think that it makes them feel something about me, because I'm clearly on the other end.

What does a student who has intentionally and perhaps maliciously cheated think when they hear that raspy voice of mine enthusiastically and often discursively responding to their work? Perhaps they have ice water in their veins, but it seems my chances of receiving authentic writing from them improve.

Perhaps I'm fooling myself. But it doesn't matter anyway, because providing audio or audiovideo response is faster and more thorough. If it personalizes me along the way, all the better.

A few pieces about this I've written:

“The Low-Stakes, Risk-Friendly Message Board Text.” Teaching with Student Texts: Essays Toward an Informed Practice. Eds. Joseph Harris, John Miles, & Charles Paine. Logan, UT: Utah State UP, 2010. 96-107.  

“Cutting Keystrokes, Improving Communications: Response Technologies for Writing Instruction.” California English 15.1 (September 2009): 27-30. 

“Responding to Student Writing with Audio-Visual Feedback.” Writing and the iGeneration: Composition in the Computer-Mediated Classroom. Eds. Terry Carter and Maria A. Clayton. Southlake, TX: Fountainhead Press, 2008. 201-27.

Red flags and white text: A plagiarism checker problem

For catching AI-based plagiarism, Turnitin and other checkers are limited but have their uses. A main issue with these checkers is flawed false positives (e.g., see this review by my colleague Bill Vargo on his blog), but I'm also always concerned with their tendency to default position the student as culprit.

I encountered an interesting situation last term with Turnitin.

In an online first-year course, my students used Jason Ohler's Taming the Beast (a really great text that's more relevant now than ever, BTW) to critically evaluate an everyday technology. They wrote white papers and then op-eds about their technology, and the class culminated (thanks to my colleague Dan Driscoll for the inspiration for this assignment progression) with a third project in which students created a social media campaign about their technology:

You will translate/convert work from your white paper and op-ed into a brief social media campaign (typically, a series or collection of posts). Think about your intended audience, your goal purpose with this campaign, and how to use multimodal strategies to communicate information quickly and efficiently; you may also link your messages to other texts and conversations. Your campaign—again, a collection of posts or the like—should express critical views of technology that you have developed throughout the term; use concepts such as revenge effects, as expressed by Ohler and Tenner, and problem posing, especially this “transformer”: “While most people believe __, a closer look reveals __.” I do hope you have some fun with this—and keep in mind that you’ve been practicing short form writing all term on the class Discussions.

When I began evaluating their final submissions, I was momentarily deflated when the first one had a red flag, stating, in accusatory language: "Hidden text: Attempts to throw off similarity detection." I sought further explanation and clicked to read the following:

Further explanation: Integrity insights to review as a priority. 6727 suspect characters.

What is hidden text?

An attempt to hinder similarity detection by exploiting exclusion mechanisms or artificially inflating the word count. Text is blended into the white background of a document to make it invisible.

But this was one of the top students, and I had met with her several times, so I doubted she cheated on her final project. As soon as I looked at it, I saw the "problem." This assignment was in an unconventional genre, social media posts, so in replicating posts, the student frequently used white text on dark backgrounds. There was no hiding going on at all.

So much for innocent before proven guilty.

I do appreciate Turnitin. As I've said many times, I mainly use it because of its three-minute easy-to-use audio response feature (I've long advocated for such response; see Note below), but it also does help with out-and-out cheaters. However, it has significant limitations, and as this example showed, it often encourages us to assume the worst--and sometimes that may happen when our students are providing their creative best.

Note:

1) I've written several pieces including "Cutting Keystrokes, Improving Communications: Response Technologies for Writing Instruction." California English 15, no. 1 (2009) and “Responding to Student Writing with Audio-Visual Feedback.” Writing and the iGeneration: Composition in the Computer-Mediated Classroom. Eds. Terry Carter and Maria A. Clayton. Southlake, TX: Fountainhead Press, 2008. 201-27.

A mix of approaches in this term's asynch OWC

I'm again teaching an asynchronous first-year online writing course (OWC) this term, and in an environment of increasing AI apps in which teachers not only worry about academic integrity but have more fundamental concerns about what it means to teach writing, I have found myself working from established practices while still refining them to account for the fast-paced changes we're facing.

I'm engaged in a variety of practices to try to maintain a writing-heavy class environment, some of which I've written about individually before:

• I'm again using the "textual brew" of unusual texts and discussion prompts.
• I'm trying to strengthen the relationship between the discussion posts and the major course projects.
• Speaking of discussions, I'm using them in a tried-and-true way to see a great deal of informal student writing: It's mid week 4, and in a class of 20 we're already over 550 posts (including my contributions).
• Drawing on the fundamental nature of low-stakes, informal writing, I'm having students in mini  assignments--right on the discussions!--analyze and revise their own posts; e.g., revising out instances of the verb "to be" in a post they choose.
  
• I've included the stoplight approach to AI that I mentioned last time.
• I'm asking students to include a one-page metacommentary about their use of AI in conjunction with that stoplight approach.
  
• Working with several of my colleagues in a mini teaching circle, I've assigned a fantastic book that helps students take a critical look at technology, Jason Ohler's Taming the Beast.

How's it all going? Talk to me after the quarter is over. However, I'm enjoying it: The challenge is there. These practices have created a productive pressure on my pedagogical approaches.