Friday, July 31, 2015

Observing and evaluating online writing courses


Conducting a course observation/evaluation of an online writing course (OWC), whether peer-to-peer or administrator-to-teacher, presents various challenges. Teaching evaluations in general, or, perhaps it would be more accurate to say the process of conducting teaching observations, can be vexed. You can run into issues ranging from the academic hierarchy to establishing fair criteria

These problems can become more complicated online, especially with OWCs. Evaluators, often administrators, may lack the expertise to "judge" online learning. We have known this about distance learning for a while; Benigno and Trentin said in 2000, “Given the special characteristics of online courses, their quantitative/qualitative evaluation calls for the adoption of specific procedures to assess both the learning process and the participant performance” (1). Our OWI Committee's Position Statement took this problem on, and in a number of Effective Practices we encourage observations specific to OWI:
  • Effective Practice 7.11: Online writing teachers should be evaluated/assessed by a peer or supervisor who has similar training and equal or superior abilities/experience in writing instruction generally and OWI particularly.
  • Effective Practice 7.12: OWI teacher assessment should occur in the setting and modalities that the teacher uses in the online writing course.
  • Effective Practice 7.13: OWI teacher assessment should be engaged as rigorously as—and not more rigorously than—it would be in a similar traditional onsite course.
Quality Matters has a well-regarded review process. But – and I need to talk with my QM certified colleagues and friends more – my understanding is this process and rubric can be challenging to apply to OWCs (although some have made the effort [2]).

Here are some guiding ideas for me when I've observed an OWC or the online component of a hybrid:

"Movie” of the course. First off, my underlying teacher observation strategy, in any modality, is to write a "movie" of the course (I kinda took this from Peter Elbow's "movies of the mind" approach to writing critique  [3]). My letter is just that: Here is what I saw. Ultimately, I’m not the assessor. Someone up the ladder is. I think the idea of a judgment-based evaluation is a problem, so I just say what I see and provide scant editorial commentary. The resulting narrative can be judged by others, and that will help us to avoid stupid teaching observation rubrics that include things like what people are wearing when they teach.

A guided visit. I prefer that the instructor leads me through the course, even if we do that over the phone. Most instructors have been open to providing me with full access, but I'd rather not just wander through unchaperoned. Onsite, the instructor has some control of what I see, and I believe the same should apply online.

The importance of time online. I try to think about the time that both the students and instructor are putting into the work that week. This opens up tricky questions about time and learning/teaching. How much time should someone put into a class? That's tough, and the Carnegie credit is under fire, but once I observed an online course in which the only thing going on one week was that a paper was due. I talked to the instructor about it in terms of an onsite course: What would you be doing all week, sitting in the classroom waiting for students to drop off papers? What would you do with the class time? I try to look at the work of a few students and attempt to get a sense of how long I think it's taking them to read, post, review, etc. Time and space are different online, especially in an asynchronous writing environment.

What does the student see? I try to view the class design and structure from the role of a student, but I don’t obsess over those things. In short, I don’t need pretty, but I do look for function and usability.

There's more to be said about this topic. I have long believed that OWI offers great opportunity for writing instruction. In similar ways, OWCs may offer new ways to observe and understand teaching. In a recent conversation with my colleague Michael Moore from DePaul, we discussed how the “integrity” of observation can be strengthened in an online course, for a simple reason: The whole course is right in front of you. Rather than pretend that a one-shot visit to a classroom one day is good enough for us to say something general about a teacher, we instead can see a broader view of the course and that person's teaching.

Course observations raise good issues that, as I mentioned, I'm going to investigate more over the next few months.

Notes

1) V. Benigno and G. Trentin. The evaluation of online courses. Journal of Computer Assisted Learning. Volume 16, Issue 3, pages 259–270, September 2000.
2) Harkness, S., Soodjinda, D., Hamilton, M., & Bolig, R. (2011, November).  Assessment of a pilot online writing program using the QM Rubric.  Paper presented at the 3rd Annual Quality Matters Conference, Baltimore, MD.
3) In Writing Without Teachers.

Labels: , , , , , , ,

Sunday, May 31, 2015

The Provoker

An aspect of teaching an OWC that I really enjoy is taking on the role of what I call “The Provoker,” a contradictory anti-voice, in the class written discussions.

I’ve discussed this before, but coming off my winter online persuasive writing course, I wanted to provide more detail about a practice while pedagogically fun is I believe also a key part of my students’ writing education in my courses.

The Provoker is a rhetorically edgy, devil’s advocate-type voice in the class discussions. I post using an alias; for some years, I’ve used the alter-ego “Dr. Logoetho.” I always include a few Provoker threads in an OWC. Sometimes I, Prof. Warnock, introduce our “guest” and facilitate his posts; other times I log into the CMS using a guest access so posts don’t actually have my named attached to them. I do make it clear to students that we’re playing a rhetorical game here (only once was a student confused about this). So it’s me – but it’s not.

Dr. Logoetho takes extreme stances:

Dear students in English 102:
I’m tired of hearing everyone complain about the cost of college. Considering how much people benefit earnings-wise over the course of their lifetime based upon the degree they have earned (see http://www.acinet.org/acinet/finaidadvisor/earnings.asp?nodeid=21), I argue that college should cost MORE money than it does now. I would ask if you agree, but how could you not?: I make a very reasonable and logical argument.
Yep, that’s what I think,
Dr. Logoetho

He writes with more than a little cheek, taunting the students: 

Dear students in Prof. Warnock’s English 102 course,
Kind of nice to see you again. I have something to say this week about Wikipedia. I think that Wikipedia is a completely useless site except as entertainment. No one should ever use it for anything of importance or for real research. Wikis are an unreliable way to build true information. I think the positives of Wikipedia are grossly exaggerated and are only promoted by people who have something to gain from Wikipedia or who don’t understand the informational value of the Web!

There. If you want to take me on, at least show a little argument savvy – Prof. Warnock gave you access to some materials this week, I think – and use some evidence.
I doubt you're up to the task,
Dr. Logoetho

With me, the official Professor, out of the picture, I find that students write with verve and passion while also composing solid, interesting arguments. They use evidence. They have to avoid logical fallacies, particularly ad hominem, when dealing with an often disrespectful interlocutor. They have to think through written, and sometimes emotional, argumentation. And they often work together, building off each other to take down Dr. Logoetho.

Especially in a persuasive writing-type class, I want students to have smart, authentic arguments, but a course can be a difficult place for that. Provoker threads allow students to write and argue without worrying about offending classmates or dealing with the authority-laden quagmire of “debating the teacher.” Term after term, I feel Provoker threads bring out some of their best writing.
It’s enjoyable for me too. I’m competitive and debate them head-on. I even get to bust on myself, old “Prof. Warnock.” Here’s Dr. Logoetho replying to their rebuttals to his Wikipedia “argument”; you can see that, throughout, I cite them directly (pseudonyms below) to show the power of their arguments:

SUBJ: Dr. Logoetho mad. Very mad
Dear students of Prof. Warnock,
You all think you’re so smart. Blair asks, “I only want to know how you can decide what source is valid compared to another source.” I’ll tell you: Peer review! Editors! Experts! Let the experts be the gatekeepers. Let’s get scholarly. Pete writes, “Who is to say that the information on this web page is not scholarly as any other?” Well, just about everyone, Pete! Read Praxis about the definition of a scholarly source. As Wendy points out (but in a nicer way than me; everyone is nicer than me), any buffoon can be a Wikipedia editor (in fact, even poor Prof. Warnock is a Wikipedia editor). Then Janet says, “Scholarly articles and sources are questioned all the time,” – name one time this happened! Let’s stop all this nonsense![...]

From the above, as you might imagine, the students usually “defeat” Dr. Logoetho by week’s end. He slinks away, and Prof. Warnock pops in and congratulates them. It’s fun. And I think it’s also good writing instruction.

Labels: , , , , , ,

Tuesday, March 31, 2015

In an OWC, reading is an inherent part of the experience

What do your students read in their writing courses, especially their first-year courses? And why are they reading those things?

As a teacher, I always assign a lot of readings in my courses. In fact, my process of designing a course usually starts with the readings; then I construct the assignments and then move on to the rest of the course.

For a variety of reasons, I’ve started wondering about this approach of using lists of prescribed readings in my courses. Particularly in an online writing course (OWC), is this really what I need to do?

This year, at Drexel we hired Irvin Peckham to direct our First-Year Writing Program. Irvin has brought some different approaches to teaching writing. He's focused on student attitudes toward writing, for one thing, and he also has a scaled-down approach to syllabi, which includes sparer required writings in these first-year courses. I’m oversimplifying his thoughts about this – check out his blog Personal Writing in the Classroom -- but he has led me to consider that in a well-run writing course, students’ earnest reading of each others' texts is plenty of great reading work for the term.

Yep. And this is particularly so in an OWC. When I looked closely at things this way, it struck me that students were doing a considerable amount of reading in my OWC courses. I’ve focused on the amounts of writing they are doing, but because so much of that writing is open to all course participants, they also read a great deal. My colleague Beth Hewett makes this argument in her new book Reading to Learn and Writing to Teach: Literacy Strategies for Online Writing Instruction. She points out that good OWI practices include a strong course culture of reading and re-reading.

Research has backed up that the reading in OWCs is quite different compared to onsite first-year writing courses; June Griffin and Deborah Minter compared the reading in fully online vs. onsite writing courses and found that the reading load online was more than 2.75 times greater. (1)

Let’s not get into the “do they read it all?” question right now, because the assumption that students ever experience everything in any class modality is flawed. (As I always say, in my many experiences observing onsite classes, I’m struck by students who aren't paying a lick of attention even though there’s some stranger in a tie sitting directly behind them.) My point here is that the literacy experience in an OWC is intense and rigorous by its very nature. Students in the course will engage in a lot of engaged reading on discussion boards and through peer reviews, reading that often turns directly into writing. They are casting their eyes over many, many words in powerfully engaged and often transactional ways.

Of course, there’s still room for those texts that I like to assign, but in an OWC, you don’t have to overburden them (or yourself) with structured reading. Really, I should have known. Writing courses are different from other courses because their syllabi emerge through the organic development of the students’ writing within them. Online, this is even more so.

Note:
1) Griffin, June & Deborah Minter. (2013). The rise of the online writing classroom. College Composition and Communication 65(1), 140-61.

Labels: , , , , , ,

Saturday, January 31, 2015

Peer review in OWI: Slowing it down


I’ve written here before (but how time flies since 2008 and even 2012!) about how peer review and OWI intersect so well. I keep returning to this topic, because, for so many of us who teach writing, peer review is the thing: a central part of writing instruction.

Simply put, going online accentuates and highlights that centrality. If you teach an online writing course (OWC) asynchronously, the whole class becomes a peer review. It becomes a workshop, sometimes a cauldron, in which students' ideas come under close examination. Students can continually push each other through a textual dialogue that they, even they, the children of the electronic age they are, are unaccustomed to.

In a way that some may find paradoxical to critics of digitality, an OWC works so well in this way because it is a uniquely slowed-down course experience. Students get to think. They interact with each other in a naturally contemplative environment, one that by its nature does not reward the quickest hand or the sharpest tongue.

For many teachers in any medium, that’s what good peer review is all about. It's not about shuffling papers around a room and asking students to give each other brief comments (leading, too often, to "All you need to is __ and you'll get an A!"). It's about providing students with time, even a few structured minutes, to delve into each others' thoughts made manifest through writing.

I haven't taken video of students working on written discussions in my courses (although, at CCCC 2014 I saw a fascinating presentation by Patricia Portanova of the University of New Hampshire; she used screencasts to investigate how writers are affected by distraction). But it would be interesting to compare students' cognitive and perhaps physical behaviors when responding to a colleague's post to the behaviors when they provide face-to-face responses in a classroom. How much thinking is going on in each situation?

Online, during the routine written conversations they have, students critique each other in detailed, complex ways. They push each other to think and elucidate ideas. They often are complimentary in authentic ways to writing that moves and impresses them. Without the confines of the class space, they can slow down and really think about the piece of writing.

As I've mentioned before, this dynamic can become transparent in an asynchronous OWC. If you are building a dialogue, then all postings become a kind of peer review: Here’s what I have to say about the writing you put before me. Based on my response, you can tell if I thought it was clear, if I thought it was powerful, where I thought it might need more elaboration or even evidence.

My students push each other hundreds of times in this way in the course. I believe it tightens their writing in ways I could never do alone.

Labels: , , , , , ,

Wednesday, November 26, 2014

OWI: Opportunities in medium, genre, volume


I have another blogging life, and this month those lives weirdly intersect, as I'm building here on my post for my blog Virtual Children (which I've been keeping on the Website When Falls the Coliseum, for several years). I posted there about another blog article written by a friend of mine on the Website 11trees; his smart post describes how much he writes in a work day, what he called just an average writing day for a "knowledge worker." In a one-day diary, he calculates he wrote about 2,500 words, saying, "We write more words every day than many college or high school students write in an entire term." 

His post is so smart for many reasons, particularly because it spurs great conversations about genre, medium, and writing volume. In his log, the writer describes all kinds of writing he did during his "writing day." At 5 am he's emailing; he works on PowerPoints and text messages; he drafts notes and other materials in/through word processing programs and different collaboration software. He's composing collaboratively with people around the world, planning, drafting, and putting on finishing touches.

If you teach writing, you have probably thought a lot about how and how much people write. I think the generation we’re educating now interacts with each other through writing more than any generation ever
But our teaching to prepare students for their writing lives often doesn’t account for the rhetorical complexity, the varied genres, and the sheer volume of the writing they will do. In fact, in many cases, writing instruction looks pretty similar to the way it looked many decades ago.
I remember in my earliest moments of interest in OWI that I felt that digital instructional settings opened new avenues for teaching and learning writing. In OWCs, students compose in multi-audience environments that are close to writing they will do in their professional lives -- and these environments are simply inherent parts of the course. Also, it was evident that electronic writing spaces allow teachers to assign lots of informal and exploratory writing, providing many opportunities for students to engage in meta-reflection about their composing processes. 

To me, ed tech tools are not exclusive to online instruction -- I mean, you can use digital tools with any type of course -- but the digital environment of OWI does seem to lead teachers to explore tech tools, so they will find new ways to interact with and even assess student writing that make the whole process more productive. For instance, 11trees is a great example of a tool that makes it easier for teachers to respond to student writing.
As I’ve described before, a brief analysis of my own 10-week, quarter-system classes showed  students were writing some 15,000 words through message boards and electronic journals. I was proud when I found that out and still am; I feel they had a challenging writing experience. But if you think about that number in comparison with the 2,500 daily words of a "knowledge worker,” you realize I might need to significantly up the workload yet if I'm going to responsibly teach writing that prepares students for what they’ll face at the next stage.
By the way, I'm not advocating a kind of mindless kowtowing to the “real world.” I don't want us to dispense of genres/modes like the exploratory essay or writing to learn. But the limitations of "school writing" get exposed in productive ways in online writing courses. The only game in town need not be "papers" that don't resemble the short, high-stakes writing that they do. We can put them in posting and sharing situations that allow us to better teach them what they will do -- and are doing -- in other parts of their lives.
This is a major component of the promise I see of OWI. Perhaps we need to do more in our research to understand not just the kinds but the amount of writing that take place in various professions.

Labels: , , , ,

Friday, September 26, 2014

Blogging about blogs


I’ve been writing this kind-of blog for nearly 10 years (it’s astonishing; I was a young man once), covering many topics, but I’ve never written about, uh, blogs.
 
Time to fix that. A recent conversation on our first-year writing (FYW) listserv at Drexel got me thinking about how I’ve used blogs in teaching, and I contributed a message that included a version of some of the stuff below.
First, I realize I probably haven’t covered this topic because blogs have become almost transparent to me as a teaching practice/approach. I use them for many assignments. As Wikipedia points out, blogs are really just Websites, and you can have students do a lot through the simple platform/genre of the blog. Advantages to me are straightforward; blogs

  • Encourage students to write.
  • Help students heighten audience awareness, especially in terms of multiple audiences, which I think is important.
  • Allow students to explore multimedia.
  • Are easy to share. Having a list of URLs on a simple discussion thread in our CMS makes it easy for me and others in the class to access and share these blogs.
  • Are easy to keep track of as an instructor.
  • Are easy for students to update.
  • Are difficult (maybe impossible) to lose.
  • Encourage post-class continuation of writing work. Many of my students have continued their blogs after our term had ended.
  • Require no paper (they are quite easy to lift).

You won’t need to do much to “teach” students how to create their blogs. They can set up a blog using any blogging application/software quickly. (I once timed a student who was complaining he couldn’t set up his blog. I put him on the clock and it took about a minute; he left with an “Okay, Warnock, you win – this time” attitude.)

I’ve used blogs as a form of journal in first-year writing (FWY) courses for about 15 years. (Lots of people are doing this, even in middle schools.) Mainly, the kind of journals I’m talking about are focused, often prompt-driven. The prompts tend to tie in with writing-about-writing approaches I’m using in the class: Students writing about their own process during a particular project, compiling quotes a la a commonplace book, or even writing down words they learned from our readings. The point is there are not overly personal.

Blogs have been the platform/genre for many other types of projects in my courses. In some courses, I ask them to pursue a focused topic of interest to them throughout the term, blogging several times a week. My students' work -- what can I say? Some of it is now part of the blogosphere. Some of these things are amazing.

I also use blogs a lot for teams. I assign Website projects in both FYW and courses such as Writing in Cyberspace, and students have packaged projects ranging from advocacy sites to class-wide "e-zine" publication through blogs. This appears, among other advantages, as an easy way for them to work together.

During the term, I usually ask students to peer review each other’s blogs for an informal grade. Some teachers worry about privacy, but I don’t do any grading or even commenting in the public space of the blog, although other students use the blog comment functions at times. There is no requirement that students use their names publicly, and I don’t require any personal disclosure as part of their school work.

Labels: , , , ,

Tuesday, August 05, 2014

Online peer review writing groups

In talking with teachers about online writing courses (OWCs), I find myself circling back to some core teaching ideas, and one is that OWI appeals so strongly to me because the OWC becomes, especially when asynchronous forums are at its center, an ongoing peer review. And I sure do believe in peer review.

In OWCs, students are consistently reading, thinking about, and responding to each other’s writing. They are working with each other as writers. It’s serious work, non-surface work. In fact, an oft-encountered problem with “regular” peer review is that students who are not confident or are unsure how to respond turn into grammar guards/editors. You rarely see message board conversations that devolve into that, and when you do, it often looks like flaming.

Some of what makes peer review work so well online is simple: It’s easy to bring students together and share documents digitally. But using easy-to-facilitate groups can become an even more powerful part of the OWC experience. I often conduct my group online peer reviews similarly to how I would other peer reviews in terms of review instructions, but there are two major logistical differences: 1) reviewers are responsible for all drafts in their group, and 2) comments are not just additive; they must be cumulative. In other words, say Ariana posts her document and Alexa reviews it. Drew cannot just come along and review Ariana’s draft without also accounting for Alexa’s comments. The reviewer's writing challenge becomes more complex, and the reviewee must read and account for multiple, perhaps contradictory, comments. They can synthesize such comments and consult me – or the writing center or someone else.

Technology-wise, groups are easy. (I use Blackboard Learn in my courses and find, despite the vitriol often directed at Blackboard, that it mostly does everything I need it to do. That being said, I do think its Group function is not very good, but my understanding, based on a Bb Learn “listening session” I attended recently, is that Bb will soon improve this functionality.) In terms of group creation and maintenance, you can simply let the CMS randomly create the groups for you, or you can purposefully build groups based on various criteria, distributing students in ways you think make sense. Four is a good number of students in a group.

Exchanging documents is easy on message boards. They can simply use attachments. Or they could use Google Docs or something similar. They can respond back to drafts in a post within the group and/or make comments using Track Changes or other annotation software. 

You can also choose whether students remain in the same group all term or whether you reshuffle when a new assignment arises. There are good arguments for both.

The online peer review group is basically a writer's circle, and perhaps students who have a good experience in class will find ways to set up similar circles for their own writing projects.

Labels: , , , , ,