Conducting a course observation/evaluation of an online writing course (OWC), whether peer-to-peer or
administrator-to-teacher, presents various challenges. Teaching evaluations in general, or,
perhaps it would be more accurate to say the process of conducting
teaching observations, can be vexed. You can run into issues ranging from the academic hierarchy to establishing fair criteria
These problems can become more complicated online, especially with OWCs.
Evaluators, often administrators, may lack the expertise to "judge" online learning. We have known this about distance learning for a while; Benigno and Trentin said in
2000, “Given the special characteristics of online courses, their
quantitative/qualitative evaluation calls for the adoption of specific
procedures to assess both the learning process and the participant performance”
(1). Our OWI Committee's Position Statement
took this problem on, and in a number of Effective Practices we encourage observations specific to OWI:
- Effective Practice 7.11: Online writing teachers should be
evaluated/assessed by a peer or supervisor who has similar training and equal
or superior abilities/experience in writing instruction generally and OWI
- Effective Practice 7.12: OWI teacher assessment should occur in the
setting and modalities that the teacher uses in the online writing course.
- Effective Practice 7.13: OWI teacher assessment should be engaged as
rigorously as—and not more rigorously than—it would be in a similar traditional
has a well-regarded review process. But – and I need to talk with my QM certified colleagues and friends more – my understanding is this process
and rubric can be challenging to apply to OWCs (although some have made the effort ).
Here are some guiding ideas for me when I've observed an OWC or the online component of a hybrid:
"Movie” of the course
. First off, my underlying teacher observation strategy, in any modality, is to write a "movie" of the course (I kinda took this from Peter Elbow's "movies of the mind" approach to writing critique ). My letter is just that: Here is what I saw. Ultimately, I’m not the assessor. Someone up the ladder is. I think the idea of a judgment-based evaluation is a problem, so I just say what I see and provide scant editorial commentary. The resulting narrative can be judged by others, and that will help us to avoid stupid teaching observation rubrics that include things like what people are wearing when they teach.
A guided visit.
I prefer that the instructor leads me through the course, even if we do that over the phone. Most
instructors have been open to providing me with full access, but I'd rather not just wander through unchaperoned. Onsite, the instructor has some control of what I see, and I believe the same should apply online.
The importance of time online.
I try to think about the time that both the students and instructor are putting into the work that week. This opens up tricky questions about time and
learning/teaching. How much time should
someone put into a class? That's tough, and the Carnegie credit is under fire, but once I observed an
online course in which the only thing going on one week was that a paper was
due. I talked to the instructor about it in terms of an onsite course:
What would you be doing all week, sitting in the classroom waiting for students
to drop off papers? What would you do with the class time? I try to look at the work of a few students and attempt to get a sense of how long I think it's taking them to read, post, review, etc. Time and space are different online, especially in an asynchronous writing environment.
What does the student see?
I try to view the class design and structure from the role of a student, but
I don’t obsess over those things. In short, I don’t need pretty, but I do look for function and usability.
There's more to be said about this topic. I have long believed that OWI offers great opportunity for writing instruction. In similar ways, OWCs may offer new ways to observe and understand teaching. In a recent conversation with my colleague Michael Moore from DePaul, we
discussed how the “integrity” of observation can be
strengthened in an online course, for a simple reason: The whole course is right in front of you.
Rather than pretend that
a one-shot visit to a classroom one day is good enough for us to say something general about a teacher, we instead can see a broader view of
the course and that person's teaching.
Course observations raise good issues that, as I mentioned, I'm going to investigate more over the next few months.
1) V. Benigno and G. Trentin. The evaluation of online courses. Journal of
Computer Assisted Learning
. Volume 16, Issue 3, pages 259–270, September 2000.
2) Harkness, S., Soodjinda, D., Hamilton, M., & Bolig, R. (2011,
Assessment of a pilot online
writing program using the QM Rubric.
Paper presented at the 3rd Annual Quality Matters Conference, Baltimore,
3) In Writing Without Teachers.
Labels: assessment, CCCC OWI Position Statement, composition, course evaluations, course observations, distance learning, online writing, online writing instruction