SoylentNews

The tl;dr Version:

• Drastically reduce number of moderators
• Return of meta-moderation
• Allow more moderation with fewer moderators
• Supermoderations to lock posts that need it

So you guys did an amazing job in letting your voices be heard in both the moderate^post and Why Did You Lurk? posts. I've read through every comment, left a couple dozen of my own across both discussions, and have sat here and digested it. The most valuable thing we have is an incredible signal/noise ratio. This is a byproduct of this site being relatively small, and with a highly engaged community. As time goes on, we're going grow; this is a natural part of any website; a web site that is not growing is entering a death spiral.

Any community needs new members to come in as older members either become less active, or leave. Kuro5hin is a great example of what happens if you just completely shut your doors, or create barriers that are too high prevent new members from joining in.

The problem then becomes, given more and more members, can we keep a high signal to noise ratio? I think it's possible.

One thing that always struck me about slashcode's moderation system is that its fairly unique in trying to keep a good signal to noise ratio. Most sites provide a simple discussion system, and doesn't do anything to try and keep the S/N ratio high. Some sites have a +/- system like Reddit or Ars Technica, but these systems seem to mostly encourage groupthink; Ars seems to do it best with its "Controversial" tag, but even then, these systems don't do much to manage S/N, it operates more on the basis of "popularity".

That being said, the system as it exists today doesn't really work. Plenty of comments never leave their starting moderation score, (at least on the other site), get buried under hundreds and hundreds of other comments. During the moderate^post discussion, I was linked to this journal by wjwlsn, talking about seeing far too many downmods. Moderation is supposed to be about raising good comments to visibility, and removing trash; its *not* a system for silencing those you disagree with. Given most comments never leave their starting scores, and Score: 0/-1 comments rarely get viewed, and then upmodded, this compounds an already bad problem. There were two comments on that journal that really stuck out to me:

Re:Downmods

"One good laugh for you is one mod point in the trash for some disagreeable asshole"

So anyone that disagrees w/ you is an asshole?

I don't see the problem, the down mods are there to be used AFAICT. What's wrong w/ modding something you disagree w/ down? There are times when I don't have a dog in the fight so I don't feel like commenting, but if I read something that seems wrong I don't have a problem w/ modding it down.

Human problem

This was often a problem on the other site as well: Express an unpopular or non-PC opinion, be modded into oblivion. Sometimes I put up posts like that just to spark discussion, but a post at -1 doesn't spark anything.

I hope people here will think about it, but it's a basic human problem: We all tend to be lemmings (or sheep, if you prefer), and want to go along with the flock. If we don't stop ourselves, downmodding unpopular opinions is just human nature...

Despite the fact the moderation guidelines explicitly say you don't downmod for being wrong, its obvious this is happening, and even starting to effect discussions here. Other comments (and actions here) show that the problem here is not just limited to one or two people. However, short of manually checking the moderation log and banning people from moderation, at the moment, our options at stopping this sort of behaviour are limited.

The problem is the system is too open to abuse; with the moderation rework, we've also got more modpoints flying per user. Furthermore, the discussion system itself doesn't help any; D1 is very much stuck in 1997, is in drastic need of some modernization, and we've had a ton of requests to bring something like D2 back. JavaScript experience on staff to create a D2 system is somewhat short at the moment, That being said, I think we can improve the situation in the short-term.

In the short term, I think we've got two goals we need to accomplish: making the discussion system more usable, and reworking the moderation system to focus on increasing signal to noise vs. being used as a "wrong/right" system. Let me cover these in turn.

The biggest thing right now is the threaded interface we use by default is kinda clunkily; even "Parent" doesn't work in the way you'd expect it. As a short-term solution, I'm going to introduce a "Hybrid" option, which when a post is short on comments, displays everything as "Nested" does now. This will allow people to easily see comments vs. having to go through the current clickfest required to see replies and reply. At a certain cut-off, the view will change to threaded, which will keep the page manageable. The cutoff will be user modifiable, as well as the option to always use threaded or nested by default if you happen to hate the new behavior.

Furthermore, we're going to add a new option under score, which is called "Average", which is exactly what it sounds like on the tin. As moderators make their way through a discussion, they will elevate the average scores in a discussion which will cause cruft to fall below the average, and drop out of view, which will help to keep the index in nested mode as long as possible until sheer quantity forced it otherwise.

While these two changes will help considerable in reading through various discussions, it doesn't actually help solve the underlying problem of misused moderations. So I'm going to rewrite the moderation algorithm and create a v2. Here's what I'm proposing to rework it.

Moderate & Post
So, after that entire discussion, I found most in favour of loosing the existing conditions. I'm treading carefully on changes here as it may upset the balance too much, but the current XOR methodology is too restrictive. The first change will be very simple.

You can moderate in any discussion you haven't posted in

You can post in a discussion after you've moderated, and your moderations won't be undone.

After posting, you can no longer moderate

My biggest concern with moderate^post was it created a chilling effect due to users reluctance to undo their moderations. This system should allow moderators to join in the discussion without influencing it too heavily. I'm willing to make more changes here, but this is something I want to take baby steps on.

Bringing Back Metamoderation
I've honestly been against M2 (at least as how its currently implemented), but its clear a system to rate moderations is desperately needed. I suspect most here have never seen the original M2 system; it disappeared on the other site some years ago, but it was extremely clunky; you got 10 comments, then a box to mark if you agree/disagree. Those votes then disappeared into the backend, never to be seen again. Not exactly a system designed to encourage repeat use. Furthermore, they never had any impact on posts themselves; bad moderations were never repealed.

Obviously, we can do better than this. My proposal is to tie metamod into the new karma system (which itself will be subject of a second post), and then attaching weights. Bad meta-moderations will get weighted by the M2 scores attached to them; if enough people say that a moderation was bad, that moderation is undone. Good meta-moderations will inform the moderator that they did a good job and influence my next bullet point.

M2 scores will be visible to a user so they can see why they're getting X points, or why they stopped getting any. This system only works with transparency, and that's whats needed.

Reducing Moderator's Workload
The fact is there are a lot of posts that deserve moderation even to +3, but just never get there. Its demoralizing to have posts just sit and never have anything happen. Furthermore, users who have a reputation of posting well should be assumed that they will continue to post well. As such, as part of the moderation reworks, I want users to be able to post higher than just +2, and get to higher and higher scores without needing intervention.

Sliding Scale of Points
With a system in place to weigh people as moderators, your M2 scores will affect how many mod points you get. People that vindictively downmod will (hopefully) get negative M2 modifiers which will undo the damage, AND cause bad moderators to get fewer and fewer points until they stop getting points all together. I need to work out how these weights will work, but in effect, bad moderators will be removed from the pool of potential moderators (as they will be awarded a grand total of zero points).

That being said, some users do change their ways, so M2 weights will (over time) age back to zero, so if someone has a change of heart, and manages to have positive karma, they will slowly begin to get modpoints back after being made ineligible. Everyone should have a chance to redeem themselves, accomplished by allowing moderation bans to expire naturally.

Ending Expiration Of Modpoints
Modpoints (theoretically) expire to prevent people from hoarding them or saving it for their favourite stories. In practice this causes huge amounts of irritation, and doesn't work very well. On the other site, its relatively easily to just have 10 or 20 accounts which can become eligible for modpoints, and allow complete and total hijacking of discussions.

Furthermore, both here, and on the other site, mod points are handed out like candy; there's a *very* good chance you'll have those mod points when your favourite story comes around, so the entire expiration system is pointless. If we can make M2 work properly, then abuses of the moderation system will fix themselves via peer-review, and we don't need to worry about group fanboys trying to moderate dissent out of existence. Under this revised system, modpoints will replenish themselves throughout the day, up to a users modpoint cap. A user who has a cap of 24 will get one modpoint every hour until they've hit their quota.

This should also reduce pressure on people to moderate constantly, and prevent moderator burnout. That being said, for those who just don't want to moderate, they can opt out as always.

Increased Costs to Downmod
The fact of the matter is that despite the abuse, downmoding exists for a legit reason; without a way to reduce "clutter", trolls and flamebait, the index would quickly become unnavigable at low scores. So the question becomes, how do we limit the abuse. M2 will help, but even then, there should be limits to the amount of damage that can be done by any one account. The easy solution is to make downmodding harder than upmodding. The backend already supports variable costs for moderations, so, the question is how much should it cost. The fact of the matter is most people don't like to downvote but it doesn't take a lot to upset the balance.

My thought here is it should be 2*score_of_comment (minimum of 2), which means those who have a good reputation for posting are harder to downmod. This is likely to need adjustment if we started get pelted with ungodly amounts of spam, but until then, I think this is enough to help curtail some of the worst abuses of the moderation system.

Underrated/Overrated
That being said, for every rule, this is our exception. Underrated/Overrated theoretically serve a good purpose, but frequently just aren't used being properly. I want to preserve this functionality, but prevent abuse. I've come up with a couple of ideas on this, but they remain relatively complex. Right now, the best idea I've had is allow Overrated/Underrated to be applied once per comment by any moderator, and the poster isn't dinged any karma for it. That being said, I'm open to suggestions in light of the rest of the reworks on how to keep Underrated/Overrated without allowing rampant abuse

Moderation Eligibility
Right now, basically, if your account is old enough, and you have positive karma, you're eligible to be a moderator. This is a bit too open for my taste; moderators should be vested in our community. What I want to do is change it that there's a cooldown of a few weeks before an account becomes eligible for modpoints, and limit it to high karma accounts (under the current karma system). This will drastically reduce the pool of moderators, but make the system much harder to game, and much less vulnerable to astroturfing.

My thought is if you have to have multiple sockpuppets regularly posting insightful and interesting comments to get enough modpoints to astroturf a discussion, they will be helping to drastically improve the S/N ratio! Metamod will cause those astroturfed moderations to go away once peer review has a chance to review them. It should require real effort to get those first ten moderator points, lest the system be further corrupted by trolls.

With fewer moderators, the system has to work to better and smarter; we need ways to connect moderators to posts that need moderation that may be buried in the index.

Suggest/Report Buttons
Suggest does what it sounds like; it places a comment on a to-be-implemented list of comments that should be moderated; this helps comments and moderators connect. To prevent abuse, only logged in users can suggest comments for moderation. If need be, we'll implement rate limiting as well to limit the number of comments suggested in a period.

Report on the other hand is when someone spots moderation abuse. Despite everything above, its still possible that moderation abuse will still happen. A comment may be controversial enough that it gets pounded to the bottom of the tree. Report calls in the last line of defence, and makes a comment eligible for super-moderation.

Wait, what? Supermoderation? Indeed, this is our last line of defense against abuse. A small subset of users who have very high metamod scores, and high karma will be handpicked by the staff to become supermoderators. Supermoderators will receive a special supermod point which, when applied to a post, allows the supermoderate to send it to Score +4/+5 (moderator choice), and lock it from further moderation.

This is designed to keep controversial posts from being moderated out of existence, and keep discussion, instead of making it vanish into the realm of -1. I realize this is easily abusable, so we'll be keeping a very close eye on supermoderators to make sure there's no abuse going on; if we're lucky, supermoderations will be an extremely rare event, but as time has shown, any system can be gamed. This is to help prevent it.

In Closing
I'm determined to try and keep this community one of extremely high quality. Now, the fact is, I may be trying to solve an unsolvable problem, and perhaps there is a great reason why most forums do not try to filter for signal/noise. That being said, I'm willing to attempt to try and solve or at least drastically improve it. As with all things, this system will constantly be under review, and if need be, be ripped out and replaced should it prove to be untenable with larger amounts of users. I also plan to implement moderation previews which should help make sure you don't misapply moderations.

A lot of this work will tie into the karma reworks I want planned, so keep your eyes peeled out for that post in the next few days.

NCommander adds: *sigh*, after this went up, someone went and linked me to the moderation rework ideas on the wiki. I didn't see that page before (as I don't frequent the wiki unless I'm checking the backend documentation) and wasn't linked in the previous discussion. I'm going through it now to see what I want to incorporate. If you were an author on those pages, please make yourself know, and provide feedback and I'll revise my plans.