Impartial Rating

Moderation and hosting of bulletin boards is arguably what makes for a good online community. Without it, whether managed by the owner of the service or by the community itself, conversations easily degenerate into spam-fests or flame wars. It is the host, with their carrot that prompts conversations and gently fans the embers of discussion. The Moderator holds the stick and smacks those who behave in an antisocial manner through either a warning or a ban from the system. Whether it's a good model or not is a different debate. If searching for ammunition on such a discussion I recommend this article on H2G2 about the politics of online discussion.

I work for a large corporation that has many, many message boards and 'communities'. They're in a position where they editorially cannot affort to have 'nasty' people being 'nasty' in their public space as this is supposedly a place for all to enjoy. The problem with having good hosting and lots of boards is that you have to pay moderators to trawl through near all the messages to ensure there isn't any 'nastyness' either before or shortly after posts are published to the board (this is known as either 'pre moderated' or 'post moderated').

That is, unless you emply a system such as Slashdot's where, as Clay Shirky eloquently puts it ... [the] core principle, for example, is "No censorship"; anyone should be able to comment in any way on any article. Slashdot's constitution (though it is not called that) specifies only three mechanisms for handling the tension between individual freedom to post irrelevant or offensive material, and the group's desire to be able to find the interesting comments. The first is moderation, a way of convening a jury pool of members in good standing, whose function is to rank those posts by quality. The second is meta-moderation, a way of checking those moderators for bias, as a solution to the "Who will watch the watchers?" problem. And the third is karma, a way of defining who is a member in good standing. These three political concepts, lightweight as they are, allow Slashdot to grow without becoming unusable

However, there are pitfalls to the self moderating system of collaborative filtering. I can't remember who wrote about this, I wish I did so I could link to a far more erudite explaination of the self-fulfilling prophecy syndrome that befalls a self-moderated online discussion. In short, it goes something like this:

Where we have a system that allows a me to view only content that appeals to me (i.e. that other people that I 'rate' have rated said content highly) I will only ever see what I want to see, which is a self fulfilling prophecy. Now, this may seem like a good idea when thinking about filtering out spam and trolls, except when it comes to a situation where a balanced view of the world's take on an issue may be advantageous. If I'm always reading opinions that I agree with, where's the balance in that?

The Corporation cannot afford to keep plunging money into more and more moderators. Especially not when the message boards are becoming more and more popular. So, reactive moderation (a system in which posts are not checked by moderators unless a complaint is made) is phased in on certain boards where the community can be trusted. Moderation costs fall - Hooray!

But, there are still some boards where we can't rely on this system, e.g. in Kid-Safe areas, or where it is considered too legally or editorially risky to have anything defamatory on a space sponsored by The Corporation (regardless of who wrote it), even for a minute. The News site is one of these sites. It is also one of the most popular sites on the web. How do you allow discussion on a site where you can't afford to pre or post moderate? Self moderation and Collaborative filtering are the popular models talked about at the moment. But then you have to think about whether this is healthy in context to the issues raised above. Especially when we're talking about a News division that prides itself on impartiality. Surely, if I don't like someone's viewpoint, I'll rate it poorly and so will people like me. Then I'll see their comments which I agree with... and I'll be happy in my bubble thinking that the world's people all agree with me. Can you have a self moderating system that doesn't fall into this trap? I don't know. I'll let you know if we think of something.