Narrating a research journey

While we were teaching... lots of questions writing instruction, especially in digital environments.

As I always say, I'm not playing cops and robbers with my students, but we have to be responsible; we have to keep sharpening the way we approach writing assignments. Much like we always should be doing, we've got to use common sense to help students do their best work. AI is just the latest temptation for them not to do so.

Sources are an issue. Since search engines have evolved, there has always been an opportunity for students to come up with wild sources: Where did some of these things come from? 

Aside from the well-established "hallucination" issues that come up when we use AI as a research tool--and remember, that type of search is not what LLMs are primarily designed for or good at--it can also dig up/create material from who knows where.

A way to help combat this is a twist on the annotated bibliography: Ask students, using plain language and perhaps images, to narrate the research path they've taken. I'll frame it like this for students:

Tell me for each source in your bibliography/references/works cited:

1. What tools or apps you used for the search, or, alternatively, who you talked to.
2. What prompts you used to generate your references: Reproduce them verbatim.
3. How much did you each resource when choosing to add it to your resource list?
• This is a shame-free zone: Did you read the abstract? Glance through it for a key word, author, or quote? 
• Reflect on what you did: This will be of interest not just to me but I imagine to you as well.
4. How did you extract information into your project?
• Did you use quotes, paraphrased material, data? 
5. How did you create the citation?

Of course, this is not an exhaustive list, and this is not necessarily something you would use only in OWCs, but because of the text-heavy environment of the OWC, a "research narrative" might be particularly useful there.

On the novelty scale, this approach is pretty tame, but it provides--in what I would argue is a constructive, reflective way--an overt learning strategy for students to describe how they are interacting with AI and search tools to generate the sources and knowledge that form the basis for much of their writing projects.

Writing instructors: Lean into our roots!

As I suppose many writing teachers are, I'm increasingly being asked about teaching writing now that AI is ever-present among us, especially teaching online, whether online writing instruction (OWI)-specific or not. My suggestions/recommendations are strongly based on my belief that we lean back into many of the long-term pedagogical strengths of Writing Studies/Composition and Rhetoric.

First off, it's not all a cosmic struggle vs. plagiarism. While authenticity is tied up with good instruction, we can't let plagiarism become a bugbear that obstructs us or distract us from fundamentally working with students to improve their writing.

We have for a long time had a lot to offer, and in OWI courses, especially asynchronous courses built on writing interactions, we can manifest much of these advantages by using, as I wrote earlier this year, a mix of approaches: 

• Process/drafting. We should continue to create learning environments in which students fundamentally engage in process-driven writing.
• Peer review. Peer-to-peer writing has many advantages, including that it creates authentic audiences for which students are responsible. Again, this means that while some (few) students might feel unabashed about putting an AI (or otherwise inauthentic) writing piece in front of a professor, they may think hard before placing such a piece in front of fellow students. In general, a lot of peer writing helps make the class a semi-public space.
• Informal writing. Students need to have many opportunities to connect writing and thinking. We can say it all we want, but we need to have students see it by giving them ways to use informal writing to build more complex projects.
• Low-stakes writing. Tying in with the above, I'm a believer in grading (almost) everything, but with the caveat that the grading be FLS: frequent and low-stakes.
• Unusual readings. As I wrote a few months ago, assign a "textual brew of unusual texts and discussion prompts." To put it simply, build assignments from texts that are less likely to be part of an LLM's database.
• Metacommentary/Reflection. Remember that us writing teachers are very good at this, with our long-entrenched embrace of portfolios, including for assessment. Last year, I wrote about how, "Metacomments push students to talk about their practice and to defend their choices." 
• Embracing the technology. Part of using metawriting and metathinking is bringing into conscious focus things around us that might otherwise be transparent. While you don't have to center your course (or even assignments) around technology, it is worth making sure technologies are not transparent: Have the students address their use of technology, even in short reflections.

These approaches and suggestions are not just Writing Studies/Composition pedagogy but are specifically how such pedagogies might manifest themselves through OWI. As those of us in the field of OWI have been saying for well over decade, at some point all writing instruction became OWI, and we cannot retreat into blue books because of technological change.  

 OWI is far from dead--long live OWI!

As AI becomes increasingly [S]ubtle

Lately, it's not often I refer back to my 2002 dissertation, "Subtle Technology and Student Writers: The Ambient Influence of Technological Myths on Communicators," but in the diss I had a core idea that's relevance might be re-emerging, a concept I described as Subtle T(echnology); I'll define it here very briefly as "a theoretical transformation digital technology might undergo that leads to complete dependence on these machines." The transformation occurs "when five interdependent traits of digital technology are fully realized: ubiquity, transparency, dependency, interconnectedness, and insubstantiality" (1).

With the explosion of AI use, I think Subtle T provides a valuable lens for viewing this technology, which I could argue has been integrating itself into our activities in ways that meet the traits above.

Of course, I was always a Writing Studies/Comp Rhet researcher and practitioner, so in the diss I described how Subtle T functioned in writing instructional scenarios.

Recently, I had a situation in a course in which I thought a student might have plagiarized language in parts of a project. I didn't think this student had malevolent intent, but some of the writing seemed inauthentic when compared with their other writing, particularly informal writing, in the course.

My course AI policy encouraged productive use of LLMs, so I was more interested in a constructive dialogue than a wrist-slapping--or worse.

When we met, I pointed out passages I felt were inauthentic, and I said while I didn't think it was material deliberately clipped from other sites or sources, it appeared to be AI-generated text. 

We had a good conversation, but the student was nonplussed as I talked about using AI and when eventually asked straight-up if they used AI tools. They told me, equally straight-up, no. The student said while they had indeed used material they found in Google searches to help develop the project, they were "just using Google," not AI.

Well, of course, if you are "just using Google" right now, the first "hit" is an "AI Overview," marked with a "sparkle icon." The student was surprised by this and told me, and after our discussion I believed it, that they didn't think of the results of Google searches as AI, despite the "sparkle icon."

This to me was another good example of how we'll have to keep up in our courses with AI. Many teachers are growing accustomed to handling the results of prompt-based AI use: We feel confident that students who use prompts and submit the subsequent text from LLMs are overtly plagiarizing. But embedded tools?--we'll have to sharpen definitions. Grammarly is one thing. AI built into Google searches is another. 

I almost feel that we might have to broaden our perspective and be careful not only of students using AI but of AI using them: Their common behaviors may be co-opted in ways that are ubiquitous, transparent, dependent, interconnected, and insubstantial. They will notice less and less.

Maybe it's time for me to renew my efforts to get that diss published. 

Note:

1) From page 1 of the diss, which I completed at Temple University. 

Audio still sounds right for student response

I'm a ways away from blogs and other writing I've done about audio--and at times, audiovisual--feedback (see References below).

In practice, though, I've been sticking to it, lately using Turnitin's built-in audio function (Turnitin itself is built into Blackboard at Drexel), which gives you a one-click option for a three-minute audio response to student assignments.

As I've described, there are many advantages to audio response, including comprehensiveness of response, time spent on task (audio response is fast), and psychological load of responding to student writing.

It strikes me that audio also may be more relevant than ever as teachers struggle with AI, because I think among the authenticity/integrity issues that we're running with AI tools into is the way they may depersonalize the student-teacher writing relationship.

When you provide voice commentary, students hear you--I intentionally emphasized both words. Will that make them feel bad, feel sorry, feel reflective about me as their instructor? I don't know. But I do think that it makes them feel something about me, because I'm clearly on the other end.

What does a student who has intentionally and perhaps maliciously cheated think when they hear that raspy voice of mine enthusiastically and often discursively responding to their work? Perhaps they have ice water in their veins, but it seems my chances of receiving authentic writing from them improve.

Perhaps I'm fooling myself. But it doesn't matter anyway, because providing audio or audiovideo response is faster and more thorough. If it personalizes me along the way, all the better.

A few pieces about this I've written:

“The Low-Stakes, Risk-Friendly Message Board Text.” Teaching with Student Texts: Essays Toward an Informed Practice. Eds. Joseph Harris, John Miles, & Charles Paine. Logan, UT: Utah State UP, 2010. 96-107.  

“Cutting Keystrokes, Improving Communications: Response Technologies for Writing Instruction.” California English 15.1 (September 2009): 27-30. 

“Responding to Student Writing with Audio-Visual Feedback.” Writing and the iGeneration: Composition in the Computer-Mediated Classroom. Eds. Terry Carter and Maria A. Clayton. Southlake, TX: Fountainhead Press, 2008. 201-27.

Red flags and white text: A plagiarism checker problem

For catching AI-based plagiarism, Turnitin and other checkers are limited but have their uses. A main issue with these checkers is flawed false positives (e.g., see this review by my colleague Bill Vargo on his blog), but I'm also always concerned with their tendency to default position the student as culprit.

I encountered an interesting situation last term with Turnitin.

In an online first-year course, my students used Jason Ohler's Taming the Beast (a really great text that's more relevant now than ever, BTW) to critically evaluate an everyday technology. They wrote white papers and then op-eds about their technology, and the class culminated (thanks to my colleague Dan Driscoll for the inspiration for this assignment progression) with a third project in which students created a social media campaign about their technology:

You will translate/convert work from your white paper and op-ed into a brief social media campaign (typically, a series or collection of posts). Think about your intended audience, your goal purpose with this campaign, and how to use multimodal strategies to communicate information quickly and efficiently; you may also link your messages to other texts and conversations. Your campaign—again, a collection of posts or the like—should express critical views of technology that you have developed throughout the term; use concepts such as revenge effects, as expressed by Ohler and Tenner, and problem posing, especially this “transformer”: “While most people believe __, a closer look reveals __.” I do hope you have some fun with this—and keep in mind that you’ve been practicing short form writing all term on the class Discussions.

When I began evaluating their final submissions, I was momentarily deflated when the first one had a red flag, stating, in accusatory language: "Hidden text: Attempts to throw off similarity detection." I sought further explanation and clicked to read the following:

Further explanation: Integrity insights to review as a priority. 6727 suspect characters.

What is hidden text?

An attempt to hinder similarity detection by exploiting exclusion mechanisms or artificially inflating the word count. Text is blended into the white background of a document to make it invisible.

But this was one of the top students, and I had met with her several times, so I doubted she cheated on her final project. As soon as I looked at it, I saw the "problem." This assignment was in an unconventional genre, social media posts, so in replicating posts, the student frequently used white text on dark backgrounds. There was no hiding going on at all.

So much for innocent before proven guilty.

I do appreciate Turnitin. As I've said many times, I mainly use it because of its three-minute easy-to-use audio response feature (I've long advocated for such response; see Note below), but it also does help with out-and-out cheaters. However, it has significant limitations, and as this example showed, it often encourages us to assume the worst--and sometimes that may happen when our students are providing their creative best.

Note:

1) I've written several pieces including "Cutting Keystrokes, Improving Communications: Response Technologies for Writing Instruction." California English 15, no. 1 (2009) and “Responding to Student Writing with Audio-Visual Feedback.” Writing and the iGeneration: Composition in the Computer-Mediated Classroom. Eds. Terry Carter and Maria A. Clayton. Southlake, TX: Fountainhead Press, 2008. 201-27.

A mix of approaches in this term's asynch OWC

I'm again teaching an asynchronous first-year online writing course (OWC) this term, and in an environment of increasing AI apps in which teachers not only worry about academic integrity but have more fundamental concerns about what it means to teach writing, I have found myself working from established practices while still refining them to account for the fast-paced changes we're facing.

I'm engaged in a variety of practices to try to maintain a writing-heavy class environment, some of which I've written about individually before:

• I'm again using the "textual brew" of unusual texts and discussion prompts.
• I'm trying to strengthen the relationship between the discussion posts and the major course projects.
• Speaking of discussions, I'm using them in a tried-and-true way to see a great deal of informal student writing: It's mid week 4, and in a class of 20 we're already over 550 posts (including my contributions).
• Drawing on the fundamental nature of low-stakes, informal writing, I'm having students in mini  assignments--right on the discussions!--analyze and revise their own posts; e.g., revising out instances of the verb "to be" in a post they choose.
  
• I've included the stoplight approach to AI that I mentioned last time.
• I'm asking students to include a one-page metacommentary about their use of AI in conjunction with that stoplight approach.
  
• Working with several of my colleagues in a mini teaching circle, I've assigned a fantastic book that helps students take a critical look at technology, Jason Ohler's Taming the Beast.

How's it all going? Talk to me after the quarter is over. However, I'm enjoying it: The challenge is there. These practices have created a productive pressure on my pedagogical approaches.

"Stoplight" approach to AI use in writing assignments

Like you, my Inbox is bombarded with messages about AI. I try to keep up, to stay especially focused on the writing instruction-centric ones, but it seems that only an AI-based triage tool can help me with the deluge...

Teachers do keep productively discussing the challenges of student use of AI, especially in writing instruction. This term, I am team-teaching an Honors seminar Contrasts in AI: Now & Future, part of Drexel's overall Honors College theme this year: Black & White. The class has been a great experience, particularly working with my colleague, a computer and electrical engineering professor.

Our course is about AI, and in addition to many informal assignments such as reading notes and quizzes, we also have three short papers/projects. We talked about this a lot in the summer, and we eventually developed a "stoplight" approach that we thought not only fit well with our course subject matter but also helped us address the challenges of requiring short papers in this current AI educational environment. Lightly edited from the syllabus, here is what we asked of students:

You will write three short, 500- to 1,000-word papers/projects in the course, due Weeks 4, 8, and 11. We’ll discuss each in class when it is assigned, but in general you’ll follow these guidelines:

1. For the theme/thesis of each paper/project, you will develop a contrasting perspective/view of some aspect(s) of AI, building on the course materials and conversations.
2. Each paper/project will be different in its use of AI, based on a “stoplight approach”:
• Project one is RED: You cannot use any AI, including grammar checkers like Grammarly. It may be impossible to avoid some built-in tools in Word and Google Docs.
• Project two is YELLOW: You can use some components of AI to help you develop your ideas and deliver them. You might think of writing on a kind of scale of complexity or a continuum, starting with the word, working through the sentence, then paragraph, then overall concept/purpose. (We missed a few steps in between for sure.) For “Yellow,” please do use AI bots/apps in your project, but only do so up to the paragraph “level.” We ask that you still not use AI for invention, conceptual, or organizational matters, but you should feel free to “partner” with ChatGPT, Copilot, etc. to help you at the sentence- or word-level, perhaps in particular with grammar and mechanics.
• Project three is GREEN: You can freely use AI as a partner to help you write your paper. Note that you cannot just submit the raw text generated by an AI app/tool, but you can certainly go through an evolution of prompts in presenting your final paper.
3. For each paper, you will include at the end a one-page metacommentary about how the use of or the constraints on the use of AI affected your development of your ideas. There are alternative ways you could present this metacommentary that we’ll discuss in class. Note this is different from the AI statement described in the Course Policies below.

So far, we've received the red and yellow papers/projects. This class of bright students have written strong, thought-provoking projects, but, boy, has the hidden gold been in the metacommentaries. 

In these end reflections, students have commented in considerable depth about the constraints of the red and yellow categories, how they navigated what those categories have meant to their writing process, and in some cases how they hadn't realized how much they had become reliant on AI tools in their writing. (Note the metacommentaries are part of the grade, a deliberate move we made so as to reward their thoughtful work.)

I want to reiterate that our class is about AI: Issues surrounding AI are part of the course subject matter. That certainly helped lead us to developing a "medium is the message"-type approach to the assignments.

However, including these detailed metacommentary components has helped us with our writing pedagogy. While plagiarism and authenticity weren't primary targets, the metacommentaries have encouraged authenticity while also providing them with a platform to discuss their own writing and thinking processes, in line with decades of composition work about the value of such writing.

As instructors, we are eager to receive paper/project three next week, not only to see their final thoughts about AI contrasts but to read how the open green category differed for them in their use of AI in their writing.