Monday, May 07, 2007

Question 21: Overall, the instructor is . . .

Our final exam period is over, and all the blue books are sitting on my desk, just waiting for my just and fair hands to open, read and grade. So are the independent studies, honors supplements, capstone theses and whatever the hell else I agreed to supervise back in January. Our university has a 72 hour grading policy from the point the exam is given so that our "customers," I mean, students, won't have to bite their nails down to the cuticle wondering if they made an A or an A- in Advanced Chem, American Foreign Policy, Spanish or whatever else stands between them and the rest of their life. I can only hope that my students, twenty years from now, will look back at the all-nighters, the Ritalin-fueled study sessions, the panic attacks, the inverted food pyramid diet and everything else that marks the ritual of exam week and wonder why they thought nailing down that 91 instead of an 88 on their Personal Finance final would determine whether they ended up living in a trailer park or driving a Hummer.

For now, our students have the most potent tool available to them in the history of higher education to extract revenge on their professors: the standardized evaluation we administer at the end of every semester. Known in the trade as the Student Evaluation of Course and Teaching Effectiveness Form, these surveys allow students the opportunity to evaluate the course and their professors on everything from "course rigor" to "taking diversity into account" to "course organization" to "incorporating current events." But the only questions that really matter to the administrators who read them are Questions 14 ("Overall, the course is . . .) and 21 ("Overall, the instructor is . . .). The survey choices range from 1 (Poor) to 5 (Superior). We also administer narrative evaluations that allow students to offer comments on the "strengths" and "weaknesses" of their professors. Administrators do not see or read the narrative forms. Those are "private comments" for professors so that we can "incorporate" our students' "suggestions" into our teaching techniques. But that can be challenging. Even after eighteen years of college teaching, I still have no idea how being called an "asshole," "shithead," "headcase," "doochebag [sic]" or "dumbfuck" will help me improve my teaching. On the other hand, a request to "have my children," "smoke a joint sometime," or getting "my private cell phone number" will not help me refine my pedagogical approach to undergraduate instruction.

Can you imagine if professors were permitted to write things like this on student papers and exams, our own instruments to evaluate "student learning effectiveness?"

"D is for dumbfuck -- get your shit together . . . now!"

"A+ (here's my cell number . . . let's talk about the 14th amendment privately sometime)."

"84.5, not bad, but nothing that a little doobie won't cure. I'm dry, but if you're cool come on by and we'll have office hours 'outside.'"

"F -- nice job, doochebag. Could you be any fucking dumber?"

* * * * *
Students also have other outlets in which express their opinions on their professors, the most well-known of which is RateMyProfessors.Com. An entirely self-selected and self-administered survey, students can leave comments about their favorite or least favorite professors and include relevant information such as the professor's "hotness" quotient (you'll have a chili pepper by your name), perceived character flaws ("asshole ") and course difficulty ("no books, no reading, just discussion . . . best professor ever"). You would think this would rank right up there with an Amazon ("Read More About Me") reviewer's take on War and Peace. Think again. Students ask me time and time again, "Have you ever seen your ratings on rate my" I pulled some quotes off my "page" when I created by website last August because I thought the dichotomy of opinions was hilarious. But do I think any of it matters? No. Do I know colleagues who worry about what students have said about them? Yes.

* * * * *
I don't know how other universities treat student evaluations, but my university considers them the most important "measure" of teaching effectiveness. Now matter how respected you might be among your professional peers, how much time you put into your classes or how good a teacher you really are, all that matters for decisions regarding tenure, promotion, raises and teaching loads is the numerical score the students assign to Questions 14 and 21. And most professors understand that the connection between student satisfaction and a positive evaluation of their courses is a good grade. For untenured professors, the student evaluation process can be nerve-wracking. On the one hand, these are scholars steeped in the norms of academic integrity. For most scholars, the thought of fudging results or manipulating questions to achieve a certain outcome is an anathema. But so is the thought of losing their job because of students disgruntled with their grades. For universities like mine that have no corroborative measure of teaching effectiveness (peer observation, invited third parties from outside the university, qualitative assessment), the student evaluations have an importance and impact that are inversely proportional to their ability to assess a professor's ability in the classroom. I have heard my junior colleagues fret about how much work to assign in their classes, or what grades to assign their students or whether to give better grades before the student evaluation is administered and then wait until finals to "correct" the grades by giving more difficult exams or grading the papers more stringently. You would think that a profession whose members are more likely than most to support PBS and NPR because they enjoy programming that is non-market driven would not employ the equivalent of the Nielsen or Arbitron ratings to determine the worth of a professor.

Sometimes, though, the evaluation process doesn't even make sense when it comes out in your favor. For the academic year prior to this one, the five courses I taught were the five highest evaluated courses in my department and for all undergraduate courses taught in the School of Public Affairs. For the "merit" review process, however, I was ranked in the second quintile out of some 45 full-time faculty who teach in my school. Translated, I was the ninth or tenth most "effective" professor in my school, even though I taught the five highest evaluated courses at four different levels ranging from General Education to an upper-level seminar. And when, for the first time in my career, I asked for an explanation from my administrative superiors about why I landed where I did, I received answers ranging from "that wasn't my decision" to an explanation involving a complex formula that might well have beyond the capabilities of a Nobel prize winning statistician. More deflating was the fact that I did not use steroids to enhance my performance. Maybe some other stuff; but anabolic steroids? Absolutely not.

What does it mean to be an "effective" teacher? The benchmark in my school and, as far as I know, the university, is that 80% of your course and instructor questions must be very good (4) and superior (5). That means in a class of 35, no more than seven students can find your class "good," "fair," or "poor" without an eyebrow going up or some external encouragement to work on your teaching. Analyze the problem in these terms: if a professor distributed grades on a normal bell curve in a class of 35, 80% of those students do not receive grades of B or better. Only about nine do. And there is no way that 25 students receiving grades of C or worse are going to evaluate your class as "very good" or "superior." So the implicit bargain is in place.

The evaluation process that we use does not measure course or teaching effectiveness. It is, rather, a crude indication of "consumer satisfaction." We have to explain to students, and sometimes administrators after students complain, why we don't think it is necessary to bring a laptop to class to take notes; why they can't leave class to "take" a call on their cell phone; why we don't email them back at 4.30 a.m. because they emailed us with something "very urgent" at 4.25 a.m.; why we won't email them our class notes or PowerPoint slides; and why they can't sit down to a three-course meal in class when you are trying to teach. We need to put all this stuff on our course syllabus or we will hear how we did not fully inform the student of classroom policies and that we, not them, are at fault.

Universities are very competitive for students and their tuition dollars, especially at a university like mine where approximately 90% of the revenue for operating expenses comes from tuition. When you are almost completely dependent on your students to pay the bills, there is very little incentive to make the students come to you. You have to come to them. That might make for nicer residence halls, better food, luxurious work out facilities and lots of pretty flowers in the spring and fall. But that does not create a condition for truly distinguished teaching and a challenging intellectual environment. And while professors are certainly right to express their frustration with students who don't put as much time into their courses as they should, we would be less than honest with ourselves if we failed to acknowledge that universities, with our complicity, have done more than just a little to create the fix in which we now find ourselves.


jamiej said...

At least you don't give out evaluations during the exam period, before the exam is given when all we really want to do is throw up everything we just crammed into the dreaded blue book and get back outside in the sun and enjoy college. Probably improves your score on #21 pretty significantly.

Maybe the department chair should judge how many students come back to talk to you about life or academics after a class is over, or buy you gifts just because it's funny, or tell friends to take the class. That's what really matters.

Loralee said...

Most students don't even take the evaluations seriously. Professors usually hand them out the last 5 minutes of class, when all anyone wants to do is leave. Sadly, it's the only student feedback to which the university cares to listen. Because what girls in Ug(h) boots and mini skirts have to say should always determine a professor's worth.

For what it's worth, many students feel the same about the evaluations as you do.

rightwingprof said...

Interesting. I get extremely high evaluations from students, and mostly intelligent comments, and I am no touchy-feely buddy type in the classroom. Over my thirty years in the university classroom, I have noticed that faculty who don't take their classes or students seriously end up with students who don't take them seriously, either.