So you guys did an amazing job in letting your voices be heard in both the moderate^post and Why Did You Lurk? posts. I've read through every comment, left a couple dozen of my own across both discussions, and have sat here and digested it. The most valuable thing we have is an incredible signal/noise ratio. This is a byproduct of this site being relatively small, and with a highly engaged community. As time goes on, we're going grow; this is a natural part of any website; a web site that is not growing is entering a death spiral.
Any community needs new members to come in as older members either become less active, or leave. Kuro5hin is a great example of what happens if you just completely shut your doors, or create barriers that are too high prevent new members from joining in.
The problem then becomes, given more and more members, can we keep a high signal to noise ratio? I think it's possible.
Re:Downmods
"One good laugh for you is one mod point in the trash for some disagreeable asshole"
So anyone that disagrees w/ you is an asshole?
I don't see the problem, the down mods are there to be used AFAICT. What's wrong w/ modding something you disagree w/ down? There are times when I don't have a dog in the fight so I don't feel like commenting, but if I read something that seems wrong I don't have a problem w/ modding it down.
Human problem
This was often a problem on the other site as well: Express an unpopular or non-PC opinion, be modded into oblivion. Sometimes I put up posts like that just to spark discussion, but a post at -1 doesn't spark anything.
I hope people here will think about it, but it's a basic human problem: We all tend to be lemmings (or sheep, if you prefer), and want to go along with the flock. If we don't stop ourselves, downmodding unpopular opinions is just human nature...
Despite the fact the moderation guidelines explicitly say you don't downmod for being wrong, its obvious this is happening, and even starting to effect discussions here. Other comments (and actions here) show that the problem here is not just limited to one or two people. However, short of manually checking the moderation log and banning people from moderation, at the moment, our options at stopping this sort of behaviour are limited.
The problem is the system is too open to abuse; with the moderation rework, we've also got more modpoints flying per user. Furthermore, the discussion system itself doesn't help any; D1 is very much stuck in 1997, is in drastic need of some modernization, and we've had a ton of requests to bring something like D2 back. JavaScript experience on staff to create a D2 system is somewhat short at the moment, That being said, I think we can improve the situation in the short-term.
In the short term, I think we've got two goals we need to accomplish: making the discussion system more usable, and reworking the moderation system to focus on increasing signal to noise vs. being used as a "wrong/right" system. Let me cover these in turn.
The biggest thing right now is the threaded interface we use by default is kinda clunkily; even "Parent" doesn't work in the way you'd expect it. As a short-term solution, I'm going to introduce a "Hybrid" option, which when a post is short on comments, displays everything as "Nested" does now. This will allow people to easily see comments vs. having to go through the current clickfest required to see replies and reply. At a certain cut-off, the view will change to threaded, which will keep the page manageable. The cutoff will be user modifiable, as well as the option to always use threaded or nested by default if you happen to hate the new behavior.
Furthermore, we're going to add a new option under score, which is called "Average", which is exactly what it sounds like on the tin. As moderators make their way through a discussion, they will elevate the average scores in a discussion which will cause cruft to fall below the average, and drop out of view, which will help to keep the index in nested mode as long as possible until sheer quantity forced it otherwise.
While these two changes will help considerable in reading through various discussions, it doesn't actually help solve the underlying problem of misused moderations. So I'm going to rewrite the moderation algorithm and create a v2. Here's what I'm proposing to rework it.
Moderate & Post
So, after that entire discussion, I found most in favour of loosing the existing conditions. I'm treading carefully on changes here as it may upset the balance too much, but the current XOR methodology is too restrictive. The first change will be very simple.
You can moderate in any discussion you haven't posted in
You can post in a discussion after you've moderated, and your moderations won't be undone.
After posting, you can no longer moderate
My biggest concern with moderate^post was it created a chilling effect due to users reluctance to undo their moderations. This system should allow moderators to join in the discussion without influencing it too heavily. I'm willing to make more changes here, but this is something I want to take baby steps on.
Bringing Back Metamoderation
I've honestly been against M2 (at least as how its currently implemented), but its clear a system to rate moderations is desperately needed. I suspect most here have never seen the original M2 system; it disappeared on the other site some years ago, but it was extremely clunky; you got 10 comments, then a box to mark if you agree/disagree. Those votes then disappeared into the backend, never to be seen again. Not exactly a system designed to encourage repeat use. Furthermore, they never had any impact on posts themselves; bad moderations were never repealed.
Obviously, we can do better than this. My proposal is to tie metamod into the new karma system (which itself will be subject of a second post), and then attaching weights. Bad meta-moderations will get weighted by the M2 scores attached to them; if enough people say that a moderation was bad, that moderation is undone. Good meta-moderations will inform the moderator that they did a good job and influence my next bullet point.
M2 scores will be visible to a user so they can see why they're getting X points, or why they stopped getting any. This system only works with transparency, and that's whats needed.
Reducing Moderator's Workload
The fact is there are a lot of posts that deserve moderation even to +3, but just never get there. Its demoralizing to have posts just sit and never have anything happen. Furthermore, users who have a reputation of posting well should be assumed that they will continue to post well. As such, as part of the moderation reworks, I want users to be able to post higher than just +2, and get to higher and higher scores without needing intervention.
Sliding Scale of Points
With a system in place to weigh people as moderators, your M2 scores will affect how many mod points you get. People that vindictively downmod will (hopefully) get negative M2 modifiers which will undo the damage, AND cause bad moderators to get fewer and fewer points until they stop getting points all together. I need to work out how these weights will work, but in effect, bad moderators will be removed from the pool of potential moderators (as they will be awarded a grand total of zero points).
That being said, some users do change their ways, so M2 weights will (over time) age back to zero, so if someone has a change of heart, and manages to have positive karma, they will slowly begin to get modpoints back after being made ineligible. Everyone should have a chance to redeem themselves, accomplished by allowing moderation bans to expire naturally.
Ending Expiration Of Modpoints
Modpoints (theoretically) expire to prevent people from hoarding them or saving it for their favourite stories. In practice this causes huge amounts of irritation, and doesn't work very well. On the other site, its relatively easily to just have 10 or 20 accounts which can become eligible for modpoints, and allow complete and total hijacking of discussions.
Furthermore, both here, and on the other site, mod points are handed out like candy; there's a *very* good chance you'll have those mod points when your favourite story comes around, so the entire expiration system is pointless. If we can make M2 work properly, then abuses of the moderation system will fix themselves via peer-review, and we don't need to worry about group fanboys trying to moderate dissent out of existence. Under this revised system, modpoints will replenish themselves throughout the day, up to a users modpoint cap. A user who has a cap of 24 will get one modpoint every hour until they've hit their quota.
This should also reduce pressure on people to moderate constantly, and prevent moderator burnout. That being said, for those who just don't want to moderate, they can opt out as always.
Increased Costs to Downmod
The fact of the matter is that despite the abuse, downmoding exists for a legit reason; without a way to reduce "clutter", trolls and flamebait, the index would quickly become unnavigable at low scores. So the question becomes, how do we limit the abuse. M2 will help, but even then, there should be limits to the amount of damage that can be done by any one account. The easy solution is to make downmodding harder than upmodding. The backend already supports variable costs for moderations, so, the question is how much should it cost. The fact of the matter is most people don't like to downvote but it doesn't take a lot to upset the balance.
My thought here is it should be 2*score_of_comment (minimum of 2), which means those who have a good reputation for posting are harder to downmod. This is likely to need adjustment if we started get pelted with ungodly amounts of spam, but until then, I think this is enough to help curtail some of the worst abuses of the moderation system.
Underrated/Overrated
That being said, for every rule, this is our exception. Underrated/Overrated theoretically serve a good purpose, but frequently just aren't used being properly. I want to preserve this functionality, but prevent abuse. I've come up with a couple of ideas on this, but they remain relatively complex. Right now, the best idea I've had is allow Overrated/Underrated to be applied once per comment by any moderator, and the poster isn't dinged any karma for it. That being said, I'm open to suggestions in light of the rest of the reworks on how to keep Underrated/Overrated without allowing rampant abuse
Moderation Eligibility
Right now, basically, if your account is old enough, and you have positive karma, you're eligible to be a moderator. This is a bit too open for my taste; moderators should be vested in our community. What I want to do is change it that there's a cooldown of a few weeks before an account becomes eligible for modpoints, and limit it to high karma accounts (under the current karma system). This will drastically reduce the pool of moderators, but make the system much harder to game, and much less vulnerable to astroturfing.
My thought is if you have to have multiple sockpuppets regularly posting insightful and interesting comments to get enough modpoints to astroturf a discussion, they will be helping to drastically improve the S/N ratio! Metamod will cause those astroturfed moderations to go away once peer review has a chance to review them. It should require real effort to get those first ten moderator points, lest the system be further corrupted by trolls.
With fewer moderators, the system has to work to better and smarter; we need ways to connect moderators to posts that need moderation that may be buried in the index.
Suggest/Report Buttons
Suggest does what it sounds like; it places a comment on a to-be-implemented list of comments that should be moderated; this helps comments and moderators connect. To prevent abuse, only logged in users can suggest comments for moderation. If need be, we'll implement rate limiting as well to limit the number of comments suggested in a period.
Report on the other hand is when someone spots moderation abuse. Despite everything above, its still possible that moderation abuse will still happen. A comment may be controversial enough that it gets pounded to the bottom of the tree. Report calls in the last line of defence, and makes a comment eligible for super-moderation.
Wait, what? Supermoderation?
Indeed, this is our last line of defense against abuse. A small subset of users who have very high metamod scores, and high karma will be handpicked by the staff to become supermoderators. Supermoderators will receive a special supermod point which, when applied to a post, allows the supermoderate to send it to Score +4/+5 (moderator choice), and lock it from further moderation.
This is designed to keep controversial posts from being moderated out of existence, and keep discussion, instead of making it vanish into the realm of -1. I realize this is easily abusable, so we'll be keeping a very close eye on supermoderators to make sure there's no abuse going on; if we're lucky, supermoderations will be an extremely rare event, but as time has shown, any system can be gamed. This is to help prevent it.
In Closing
I'm determined to try and keep this community one of extremely high quality. Now, the fact is, I may be trying to solve an unsolvable problem, and perhaps there is a great reason why most forums do not try to filter for signal/noise. That being said, I'm willing to attempt to try and solve or at least drastically improve it. As with all things, this system will constantly be under review, and if need be, be ripped out and replaced should it prove to be untenable with larger amounts of users. I also plan to implement moderation previews which should help make sure you don't misapply moderations.
A lot of this work will tie into the karma reworks I want planned, so keep your eyes peeled out for that post in the next few days.
NCommander adds: *sigh*, after this went up, someone went and linked me to the moderation rework ideas on the wiki. I didn't see that page before (as I don't frequent the wiki unless I'm checking the backend documentation) and wasn't linked in the previous discussion. I'm going through it now to see what I want to incorporate. If you were an author on those pages, please make yourself know, and provide feedback and I'll revise my plans.
(Score: 2) by NCommander on Tuesday April 08 2014, @01:59AM
So, what do you call Wikipedia Administrators who lock and protect pages?
I won't even have considered supermoderations except for the fact people do abuse the moderation system, and any system that involves users are abusable. Supermoderators will not be able to silence people; just prevent people from being unfairly silenced. What would you have us do? Let everything be community decided? We tried that on Slashdot, and I don't think anyone is going to defend the moderation system as it exists today.
Still always moving
(Score: 3, Insightful) by hemocyanin on Tuesday April 08 2014, @03:40AM
When you say "unfairly silenced" you bring a perspective to the issue different from the community which did the moderating. Which is fine -- it's your site, but you shouldn't then refer to us a community. We become your users and our opinions are subject to being over-ridden by the powers that be, i.e., for all the lip service to "community" -- SN becomes another hierarchical organization. I guess what bugs me the most is the whole feel-good "community" talk -- you hear it all over the place if you hang out with liberals like I do -- so often what people mean by "community" is: "I know best." And that sentiment is expressed perfectly in the concept of appointed super-mods. The only way to do super-mods and respect the community, is to randomly assign them like mod points, but making some random asshole like me king for a day is a whole different can of worms. Still, better than appointed ones, if we really are a community.
(Score: 2) by NCommander on Tuesday April 08 2014, @04:38AM
Give me a realistic alternative then. You argue that everything should be community controlled when in actuality, that's not true; communities choose their leaders to do the best they can; for this site (and other sites on the internet), that's people sticking around. If I do crap, I expect people to take a hike. In every site, you have some people who have more power than others; editors here routine go through the submission queue in an effort to keep high quality posts. We could attach a RNG to the submission queue or to score moderators which would make it "fair" but I'm fairly sure no one will stick around if we do it.
I didn't make this clear on my post, but supermoderators were going to be picked with those from exceptionally high M2 scores; people who have been voted on the community as GOOD moderators, and likely can be trusted with the power a supermoderation provides. Yes, a staff member must ACK them; that's a sanity check so someone can't game the system and earn supermoderator points by sockpuppeting with abandon, but try and tell me how its not fair.
I've gone out of my way to try and describe things at length and communicate with every member on this site. Am I perfect? No, but I am making the effort to take in account all opinions on these things. I could have simply revamped the system with virtually no discussion, or incorporated us without trying to discuss things first, etc. You claim I use the term politically; that's your opinion, but I strongly disagree. The fact that we're having a conversation (and it is a conversation) proves its not me just doing shit and making everyone here eat it. If I started forcing my will on people, both the staff, and the users of this site would leave; it is a two way street, even if you don't wish to see it as such. I wrote up most of my revisions to the mod post due to an article on a journal (my original posts was mostly involving moderate^post, NOT larger reworks).
Maybe I'm wrong, and supermoderation is a horrid idea, but the only complaint I'm hearing is a couple of people will have a power that most won't. Maybe the feature will last all of a week before we collectively decide its a bad idea and it goes away. The fact is, this site already works this way, with people from the community stepping up to be editors and moderating the queue of stuff coming in, editing it, and publishing it. The doors of staff are always recruiting; feel free to come to IRC and make yourself heard.
Still always moving
(Score: 2) by hemocyanin on Tuesday April 08 2014, @05:07AM
First, I think you should revisit the concept of "unfair mods." That's what the super-mod is all about. I've been modded troll plenty of times, both here and on Slashdot, for things I deeply believe. That's OK. It gives me information about the wider generalized opinion of the group of people to whom I'm posting. It doesn't change what I think. It doesn't piss me off. It gives me valuable insight about who I'm dealing with and how I should view things that are modded troll or insightful. That's why I browse at -1 all the time, whether I have mod points or not -- to me, only my opinion counts, but I do like to know what the other dorks around me have gotten suckered into. Anyway, if posts are artificially inflated from their low moderation, I get none of this meta information.
Secondly, as I've mentioned numerous times now, you'll end up with an official SN perspective. This true no matter how you appoint the super-mods.
My suggestion honestly, is to do nothing. Tools exist for moderators to undo what they see as unfair moderation. Anyone who has posted plenty, will have posts go from troll to insightful and vice versa -- that represents the community's collective grading of those posts. The same is true when a post gets modded troll or informative but doesn't reverse direction. What exactly is so wrong with that? It actually express the aggregate opinion of the community. In this sense, there are no unfair-mods and there is no problem to address. There are only mods that one would personally wish didn't turn out the way they did, which tells you something about the people you're dealing with, and gives you insight on how much value to give that aggregate opinion.
(Score: 2) by NCommander on Tuesday April 08 2014, @05:23AM
My concern here is I've got a lot of posts telling me that the system is unfair. Now, that problem is compounded by lack of metamoderation, so it is *possible* that a proper M2 system will be self-correcting. Then again, groupthink might be able to overwhelm the M2 system which was the guiding though on having supermoderation. With Report/Suggest, it might have enough oopfh to self-correct if we can connect moderators to mismoderated comments.
That being said, I'm going to split the difference here. You've made enough good points that I rather err on the side of caution. Supermoderation will get implemented, *but* not enabled out of the box (with a slight exception that I'll do a few supermods on a junk journal entry to make sure it works). If M2 doesn't fix the problem on its own, I'll open the floor to discuss full enablement of the supermoderation system, and the criteria for selecting supermoderators.
Still always moving
(Score: 2) by hemocyanin on Tuesday April 08 2014, @05:31AM
Can't argue with that plan -- seems reasonable. And sorry if I came off sounding like a dick at times -- fact is, sometimes I am kind of a dick. And sometimes I even get modded for it. ;-)
(Score: 2) by NCommander on Tuesday April 08 2014, @05:54AM
Must ... resist ... urge ... to ... implement ... -5 ... dick ... moderation ...
(nah, j/k. I'm pretty good at trying to listen to feedback, and really really trying not to become a dictator. My one prize is a low UID, and even the AC beats me in that department ;.;)
Still always moving
(Score: 2) by hemocyanin on Tuesday April 08 2014, @05:58AM
Actually, I would totally vote YES for a -1 dick mod. That would rock. Especially if you could go to -5!
(Score: 3, Interesting) by starcraftsicko on Tuesday April 08 2014, @11:18AM
One moderation feature of the green site was that late posters and replies to later threads often when unnoticed. A risk I'm willing to take.
Forking this convo -- blah. I wonder if anyone will notice.
.
I think that some form of supermoderation (whatever that means) is a great idea. I think that M2 is trouble for all the reason that so many have laid out. More importantly, it doesn't solve the problems you're actually having.
Problems:
1 - not enough posts get moderated AT ALL.
2 - not enough posts get to +4 and +5
3 - sometimes unpopular opinions get modded down
.
M2 won't fix any of those.
.
My Modest Proposal(other than to consume the Irish in the event of a food shortage) regarding M2 and supermoderation:
.
.
Why is this better than M2? No echo chamber effect, and it actually could solve the noted problems:
A great post could be instantly scored +5.
Points could be spent on good but not great posts to maybe bring them to +2/3.
"The community" of moderators could still lock a post at +5 and overwhelm a bad supermod.
My $.02
This post was created with recycled electrons.
(Score: 2) by NCommander on Tuesday April 08 2014, @02:16PM
Well, I did, so there's that :-).
I'll admit, its an interesting proposal, but its too easy to game with astroturfing; if comments locked at +5, it would be possible to spam a discussion heavily then use sockpuppets to moderate in mass without M2 acting like a throttle.
Still always moving
(Score: 2) by starcraftsicko on Thursday April 10 2014, @11:21AM
As I understand M2 as it worked on the green site, the idea was to use it to lock out bad moderators by reducing their frequency or removing them altogether. As I understand it, we are considering using M2 here to PROMOTE some moderators. .
.
The problem with either approach is that both suffer from confirmation/groupthink bias. I think that part of the reason that M2 on the green site had so little real effect is that this was recognized by the implementor(s).
.
On the green site, getting a less-popular opinion modded up was an art. The most reliable way was to start your post with the words: "I know this will be modded down but". That seems to work here too.
.
Supermods and +5 locking are potential methods to 'notice' these posts without resorting to the above formula.
.
I think that you are solving the problems you don't have here, or one that could naturally solve itself.
.
In order to create a useful sock-puppet (one that can moderate, meta-moderate, and possibly super-moderate, etc.), the puppet has to make useful contributions to the community. They have to make posts that are interesting or funny or whatever. For all practical purposes, they have to be, or appear to be, regular users.
.
To keep a puppet once created, you have to use it in a way that does not draw the wrong kind of attention - so use has to be in line with the standards of the community for a user. Anything blatantly other would eventually be noticed by management and could lead to the death of a puppet.
.
[Example: I build a stack of 5 puppets. yay. I want to promote my new VIAGRA blend. I post my advert in an unrelated story's comments. My puppets mod it up. yay for me.... but you or ANY SN staffer would probably see this and lock me and my 5 puppets out of any future moderation. That's a lot of effort I'll have wasted in the end.]
.
More likely, I might use puppets to bump my good, but otherwise unnoticed posts. [Remember that if I pump bad posts, someone could notice and nuke me ; same if i did it too often] While this is ethically dubious, it is not overtly harmful. M2 could help this... but if the puppets are used correctly, M2 would miss this too [remember, puppets are promoting good posts].
.
The problem (1) to be solved is that not enough posts get moderated at all, or not enough get moderated to +4/+5. Right now, to the extent that the points of any extant puppets are used to mod UP, they are contributing to the solution! Ultimately, supermods would be better as they would formalize the process and reduce the value of puppets.
.
The other problem (2) to be solved is that there are some down-mods that are unjust (-1 disagree). Supermods and +5 locking represent a solution here as one supermod can address an injustice quickly.
This post was created with recycled electrons.
(Score: 2) by The Mighty Buzzard on Tuesday April 08 2014, @05:32PM
Problem here is the idea of logging on and off. Web traffic being largely stateless, this would be a pain to track.
Also, I know at least I could not always be trusted with unlimited mod points that could be spent demoting comments. Most of the time but not always. And regular mods with their limited pool and once per comment could do squat about it.
I think leave it as is for now.
My rights don't end where your fear begins.
(Score: 2) by starcraftsicko on Thursday April 10 2014, @10:23AM
Hence the 30 minute cap. If a supermod has been appointed in the last 30 minutes, don't promote more. If one hasn't been appointed in the last 30 mins, appoint the next. Anything beyond that (in terms of precision) is gravy.
Also, I know at least I could not always be trusted with unlimited mod points
So hopefully, you're an honest sort that would opt out.
.
If not, the work of a deranged mad-modder would be easy to spot. Once spotted, they could be removed from the pool. At most, you'd create more work/opportunity for the regular mods or for the next super-mod.
This post was created with recycled electrons.