I see from the wiki for BarCampRochester that someone has proposed a session to talk about the online professor rating system. I wish I could go. I'd love to ask some hard questions about these systems generally.
For example, would the people who champion these systems be just as enthusiastic about a publicly accessible "student rating system" that let professors share their opinions about students?
I'm really torn about these rating systems. I understand the desire and the need for them. I remember using a print version of them when I was an undergrad at Michigan. But too often the systems I've seen on the web turn into the worst example of online character assassination. I may not be the best professor there is, but how helpful is it to have a system that lets people write (as I found on one site several years ago) "She should be chasing chickens on a farm, not teaching information technology." Yes, I can laugh at the absurdity of it. But I've seen some that were far worse and more damaging than that. Comments about people's sexual preferences, their physical appearance, and more.
As soon as you allow anonymous free-text commenting, you get the worst of what people have to offer. And unlike in-class evaluations, where you get a full sample of student views--good, bad, and indifferent--on these opt-in systems you tend to get comments only from people with the strongest of opinions, skewing the accuracy.
If these professor rating systems are inevitable, what checks and balances can be put in place to keep them from being overrun with personal attacks? Is it realistic to have content editors? To limit to a preset list of comments? It's not reasonable, I think, to put the burden on the professor to police his or her own evaluations.
What if it were turned around? What if professors could warn each other about problem students--the ones who regularly fall asleep in class, the ones who consistently cause discord in group projects, the ones whose grandmothers have died at least six times since their freshman year? And what if these systems were as publicly available as the professor rating systems? Is that somehow worse? If so, why? (I can think of some reasons, but I think it's a valuable exercise to discuss this.)
Seems to me it should work both ways like that. But the thing about rating students is that you don't get to choose if you get a particular student or not, they choose you. With choice comes the need for more information to make an informed choice. I'm not sure if a student rating system would be valuable enough to merit it.
A couple of things to Ted. One is that the teacher ranking systems are probably also not valuable enough to merit the impact on the professors they rank. There is generally alot of contradictory information/opinion in the rankings. Secondly I wonder if it would be better if faculty could choose to keep a particular student out of their course. I'm not sure if it is really a good idea but I know a lot of teachers who would like that option.
I think students would object to an open list of them because it would "violate privacy" but many of those same students would not be willing to extend that privacy to faculty.
Students should be able to rate fellow students too. You may want to avoid the class with the gunner, or the dolt, or whatever.
Enjoying my summer, I haven't yet made use of a professor-rating system of any kind, and I doubt I will. As you said, they're skewed. It seems more prudent to ask friends or others who've had the professor. That begs the question: are any of these rating systems based on social networking? Do any give rankings based on what friends and trusted others thought of the professor?
Is that even feasible?
I think the thought of ranking students is silly. Students pay to attend college while professor are paid to teach. We shouldn't lose sight of the audience that colleges serve. Without students, colleges (and thus professors) could not exist.
Most often than not, the issues students have with professors come from differences in teaching/learning styles. One student might prefer a teacher who presents PowerPoint slides and lectures, while another might enjoy a more practical or hands-on approach. In-class evaluations are an opportunity for students to provide professors with suggestions for improvement. They are not meant to be reviewed by students when choosing classes. Providing an online evaluation website allows students to share this information with other students. If I want to register for class "foo" with professor "bar", I can look online to see if the professor meets my learning needs.
RIT's "Tiger Reviews" was meant to do just that. It's an open-forum for students to present honest feedback, but more importantly to share that information with their peers. Some concern was raised as to the type of comments that could be posted. The rules are simple: no libelous or threating comments. Beyond that, comments can be deleted if they are found to be of poor taste or contain profanity. I admit that the criteria for judging "poor taste" is pretty subjective. But you can't make a blanket rule that works in all situations. We try to use our best judgment in that case. This website is only open to the RIT community, removing the "prospective student" problem. In order to reduce the number of garbage posts, students are restricted to posting comments on professors/classes that they have taken.
Since its launch in February, we have attained 1,026 posts and only five have been deleted. I think that's pretty strong evidence that the majority of students can handle it. Forums of this nature will exist with or without RIT's participation. By providing our own feature-rich service, we give students an incentive to keep the comments in the family.
Sam, that's a fascinating idea...I'm not aware of a rating system that limits to friends (or friends-of-friends), and that has interesting implications. Of course, that removes the anonymity aspect, but if the ratings are only visible to your friends anonymity may not be an issue. It also reduces the perceived threat to faculty, who are more concerned about administrators reading mean-spirited or inaccurate comments than they are about students.
Bob, I've been away, so haven't had a chance to play with Tiger Reviews. Are faculty able to see their own reviews? What's the criteria for removal if a faculty member challenges the validity of the comment? Is there any way to be sure that people writing reviews actually took a class with that professor? How long are reviews preserved? (For example, would bad reviews of a new professor haunt him or her indefinitely, or do they expire after a time?)
A great discussion.
Rating systems are very young. Anonymity will be bred out of the ones that grow to become dominant.
There is just too much dangerous potential in having unverified anonymous comments that are searchable/findable/slanderous. The walled garden is how we deal with this today (Tiger Reviews), and there's nothing wrong with that for internal use. As we move to trying to verify users across context, the gardens fail us.
Having a voice and having anonymity - I think you only get to pick one.
I've written more over here.
I was out surfing the net and came across this. I just wanted to add my two cents.
My ideas and thoughts may or may not be welcome here, but either way here they are. As a student, I’m on the fence about the professor rating systems. I really don’t think that students should (or do) let them persuade their choice of classes. There are a sampling of students who don’t know on-line rating systems exist, there are students who sign up for a class and have their schedule arranged for them – without the knowledge of what teacher they will be placed with, and there are students who simply don’t choose to look.
I know the rankings exist and I look at them. Some of the feedback seems honest while others seem overly positive or negative. In my own way, I’ve come to the conclusion that students really only take the time to leave feedback if they’ve had a very good or a very bad experience. The teacher will have done something to elicit a response – positive or negative. I find that there are a lot of teachers not listed on the on-line ratings. Is this because they’re new teachers? More than likely it is because they’re your even keel, average teacher. The teacher may have an impact on a student, but not enough to cause the student to take time to rant/gush about their experiences over the last 8-16 weeks.
Also, from a student perspective, professors already rate students. What? Where? Well, professors don’t go out and gab on-line about the flailing futility of a certain student to grasp such a simple subject, but they assign them grades. Honestly, a professor doesn’t have to go out and tell the world that a student is a lazy idiot. The F on the transcript serves as their means of rating, and trust me, it’s more lasting than some website that might not be there tomorrow.
Cheers,