Stories
Slash Boxes
Comments

SoylentNews is people

posted by NCommander on Monday April 07 2014, @07:03PM   Printer-friendly
from the that-was-a-lot-of-comments-to-read dept.
The tl;dr Version:
  • Drastically reduce number of moderators
  • Return of meta-moderation
  • Allow more moderation with fewer moderators
  • Supermoderations to lock posts that need it

So you guys did an amazing job in letting your voices be heard in both the moderate^post and Why Did You Lurk? posts. I've read through every comment, left a couple dozen of my own across both discussions, and have sat here and digested it. The most valuable thing we have is an incredible signal/noise ratio. This is a byproduct of this site being relatively small, and with a highly engaged community. As time goes on, we're going grow; this is a natural part of any website; a web site that is not growing is entering a death spiral.

Any community needs new members to come in as older members either become less active, or leave. Kuro5hin is a great example of what happens if you just completely shut your doors, or create barriers that are too high prevent new members from joining in.

The problem then becomes, given more and more members, can we keep a high signal to noise ratio? I think it's possible.

One thing that always struck me about slashcode's moderation system is that its fairly unique in trying to keep a good signal to noise ratio. Most sites provide a simple discussion system, and doesn't do anything to try and keep the S/N ratio high. Some sites have a +/- system like Reddit or Ars Technica, but these systems seem to mostly encourage groupthink; Ars seems to do it best with its "Controversial" tag, but even then, these systems don't do much to manage S/N, it operates more on the basis of "popularity".

That being said, the system as it exists today doesn't really work. Plenty of comments never leave their starting moderation score, (at least on the other site), get buried under hundreds and hundreds of other comments. During the moderate^post discussion, I was linked to this journal by wjwlsn, talking about seeing far too many downmods. Moderation is supposed to be about raising good comments to visibility, and removing trash; its *not* a system for silencing those you disagree with. Given most comments never leave their starting scores, and Score: 0/-1 comments rarely get viewed, and then upmodded, this compounds an already bad problem. There were two comments on that journal that really stuck out to me:

Re:Downmods

"One good laugh for you is one mod point in the trash for some disagreeable asshole"

So anyone that disagrees w/ you is an asshole?

I don't see the problem, the down mods are there to be used AFAICT. What's wrong w/ modding something you disagree w/ down? There are times when I don't have a dog in the fight so I don't feel like commenting, but if I read something that seems wrong I don't have a problem w/ modding it down.

Human problem

This was often a problem on the other site as well: Express an unpopular or non-PC opinion, be modded into oblivion. Sometimes I put up posts like that just to spark discussion, but a post at -1 doesn't spark anything.

I hope people here will think about it, but it's a basic human problem: We all tend to be lemmings (or sheep, if you prefer), and want to go along with the flock. If we don't stop ourselves, downmodding unpopular opinions is just human nature...

Despite the fact the moderation guidelines explicitly say you don't downmod for being wrong, its obvious this is happening, and even starting to effect discussions here. Other comments (and actions here) show that the problem here is not just limited to one or two people. However, short of manually checking the moderation log and banning people from moderation, at the moment, our options at stopping this sort of behaviour are limited.

The problem is the system is too open to abuse; with the moderation rework, we've also got more modpoints flying per user. Furthermore, the discussion system itself doesn't help any; D1 is very much stuck in 1997, is in drastic need of some modernization, and we've had a ton of requests to bring something like D2 back. JavaScript experience on staff to create a D2 system is somewhat short at the moment, That being said, I think we can improve the situation in the short-term.

In the short term, I think we've got two goals we need to accomplish: making the discussion system more usable, and reworking the moderation system to focus on increasing signal to noise vs. being used as a "wrong/right" system. Let me cover these in turn.

The biggest thing right now is the threaded interface we use by default is kinda clunkily; even "Parent" doesn't work in the way you'd expect it. As a short-term solution, I'm going to introduce a "Hybrid" option, which when a post is short on comments, displays everything as "Nested" does now. This will allow people to easily see comments vs. having to go through the current clickfest required to see replies and reply. At a certain cut-off, the view will change to threaded, which will keep the page manageable. The cutoff will be user modifiable, as well as the option to always use threaded or nested by default if you happen to hate the new behavior.

Furthermore, we're going to add a new option under score, which is called "Average", which is exactly what it sounds like on the tin. As moderators make their way through a discussion, they will elevate the average scores in a discussion which will cause cruft to fall below the average, and drop out of view, which will help to keep the index in nested mode as long as possible until sheer quantity forced it otherwise.

While these two changes will help considerable in reading through various discussions, it doesn't actually help solve the underlying problem of misused moderations. So I'm going to rewrite the moderation algorithm and create a v2. Here's what I'm proposing to rework it.

Moderate & Post
So, after that entire discussion, I found most in favour of loosing the existing conditions. I'm treading carefully on changes here as it may upset the balance too much, but the current XOR methodology is too restrictive. The first change will be very simple.

You can moderate in any discussion you haven't posted in

You can post in a discussion after you've moderated, and your moderations won't be undone.

After posting, you can no longer moderate

My biggest concern with moderate^post was it created a chilling effect due to users reluctance to undo their moderations. This system should allow moderators to join in the discussion without influencing it too heavily. I'm willing to make more changes here, but this is something I want to take baby steps on.

Bringing Back Metamoderation
I've honestly been against M2 (at least as how its currently implemented), but its clear a system to rate moderations is desperately needed. I suspect most here have never seen the original M2 system; it disappeared on the other site some years ago, but it was extremely clunky; you got 10 comments, then a box to mark if you agree/disagree. Those votes then disappeared into the backend, never to be seen again. Not exactly a system designed to encourage repeat use. Furthermore, they never had any impact on posts themselves; bad moderations were never repealed.

Obviously, we can do better than this. My proposal is to tie metamod into the new karma system (which itself will be subject of a second post), and then attaching weights. Bad meta-moderations will get weighted by the M2 scores attached to them; if enough people say that a moderation was bad, that moderation is undone. Good meta-moderations will inform the moderator that they did a good job and influence my next bullet point.

M2 scores will be visible to a user so they can see why they're getting X points, or why they stopped getting any. This system only works with transparency, and that's whats needed.

Reducing Moderator's Workload
The fact is there are a lot of posts that deserve moderation even to +3, but just never get there. Its demoralizing to have posts just sit and never have anything happen. Furthermore, users who have a reputation of posting well should be assumed that they will continue to post well. As such, as part of the moderation reworks, I want users to be able to post higher than just +2, and get to higher and higher scores without needing intervention.

Sliding Scale of Points
With a system in place to weigh people as moderators, your M2 scores will affect how many mod points you get. People that vindictively downmod will (hopefully) get negative M2 modifiers which will undo the damage, AND cause bad moderators to get fewer and fewer points until they stop getting points all together. I need to work out how these weights will work, but in effect, bad moderators will be removed from the pool of potential moderators (as they will be awarded a grand total of zero points).

That being said, some users do change their ways, so M2 weights will (over time) age back to zero, so if someone has a change of heart, and manages to have positive karma, they will slowly begin to get modpoints back after being made ineligible. Everyone should have a chance to redeem themselves, accomplished by allowing moderation bans to expire naturally.

Ending Expiration Of Modpoints
Modpoints (theoretically) expire to prevent people from hoarding them or saving it for their favourite stories. In practice this causes huge amounts of irritation, and doesn't work very well. On the other site, its relatively easily to just have 10 or 20 accounts which can become eligible for modpoints, and allow complete and total hijacking of discussions.

Furthermore, both here, and on the other site, mod points are handed out like candy; there's a *very* good chance you'll have those mod points when your favourite story comes around, so the entire expiration system is pointless. If we can make M2 work properly, then abuses of the moderation system will fix themselves via peer-review, and we don't need to worry about group fanboys trying to moderate dissent out of existence. Under this revised system, modpoints will replenish themselves throughout the day, up to a users modpoint cap. A user who has a cap of 24 will get one modpoint every hour until they've hit their quota.

This should also reduce pressure on people to moderate constantly, and prevent moderator burnout. That being said, for those who just don't want to moderate, they can opt out as always.

Increased Costs to Downmod
The fact of the matter is that despite the abuse, downmoding exists for a legit reason; without a way to reduce "clutter", trolls and flamebait, the index would quickly become unnavigable at low scores. So the question becomes, how do we limit the abuse. M2 will help, but even then, there should be limits to the amount of damage that can be done by any one account. The easy solution is to make downmodding harder than upmodding. The backend already supports variable costs for moderations, so, the question is how much should it cost. The fact of the matter is most people don't like to downvote but it doesn't take a lot to upset the balance.

My thought here is it should be 2*score_of_comment (minimum of 2), which means those who have a good reputation for posting are harder to downmod. This is likely to need adjustment if we started get pelted with ungodly amounts of spam, but until then, I think this is enough to help curtail some of the worst abuses of the moderation system.

Underrated/Overrated
That being said, for every rule, this is our exception. Underrated/Overrated theoretically serve a good purpose, but frequently just aren't used being properly. I want to preserve this functionality, but prevent abuse. I've come up with a couple of ideas on this, but they remain relatively complex. Right now, the best idea I've had is allow Overrated/Underrated to be applied once per comment by any moderator, and the poster isn't dinged any karma for it. That being said, I'm open to suggestions in light of the rest of the reworks on how to keep Underrated/Overrated without allowing rampant abuse

Moderation Eligibility
Right now, basically, if your account is old enough, and you have positive karma, you're eligible to be a moderator. This is a bit too open for my taste; moderators should be vested in our community. What I want to do is change it that there's a cooldown of a few weeks before an account becomes eligible for modpoints, and limit it to high karma accounts (under the current karma system). This will drastically reduce the pool of moderators, but make the system much harder to game, and much less vulnerable to astroturfing.

My thought is if you have to have multiple sockpuppets regularly posting insightful and interesting comments to get enough modpoints to astroturf a discussion, they will be helping to drastically improve the S/N ratio! Metamod will cause those astroturfed moderations to go away once peer review has a chance to review them. It should require real effort to get those first ten moderator points, lest the system be further corrupted by trolls.

With fewer moderators, the system has to work to better and smarter; we need ways to connect moderators to posts that need moderation that may be buried in the index.

Suggest/Report Buttons
Suggest does what it sounds like; it places a comment on a to-be-implemented list of comments that should be moderated; this helps comments and moderators connect. To prevent abuse, only logged in users can suggest comments for moderation. If need be, we'll implement rate limiting as well to limit the number of comments suggested in a period.

Report on the other hand is when someone spots moderation abuse. Despite everything above, its still possible that moderation abuse will still happen. A comment may be controversial enough that it gets pounded to the bottom of the tree. Report calls in the last line of defence, and makes a comment eligible for super-moderation.

Wait, what? Supermoderation? Indeed, this is our last line of defense against abuse. A small subset of users who have very high metamod scores, and high karma will be handpicked by the staff to become supermoderators. Supermoderators will receive a special supermod point which, when applied to a post, allows the supermoderate to send it to Score +4/+5 (moderator choice), and lock it from further moderation.

This is designed to keep controversial posts from being moderated out of existence, and keep discussion, instead of making it vanish into the realm of -1. I realize this is easily abusable, so we'll be keeping a very close eye on supermoderators to make sure there's no abuse going on; if we're lucky, supermoderations will be an extremely rare event, but as time has shown, any system can be gamed. This is to help prevent it.

In Closing
I'm determined to try and keep this community one of extremely high quality. Now, the fact is, I may be trying to solve an unsolvable problem, and perhaps there is a great reason why most forums do not try to filter for signal/noise. That being said, I'm willing to attempt to try and solve or at least drastically improve it. As with all things, this system will constantly be under review, and if need be, be ripped out and replaced should it prove to be untenable with larger amounts of users. I also plan to implement moderation previews which should help make sure you don't misapply moderations.

A lot of this work will tie into the karma reworks I want planned, so keep your eyes peeled out for that post in the next few days.

NCommander adds: *sigh*, after this went up, someone went and linked me to the moderation rework ideas on the wiki. I didn't see that page before (as I don't frequent the wiki unless I'm checking the backend documentation) and wasn't linked in the previous discussion. I'm going through it now to see what I want to incorporate. If you were an author on those pages, please make yourself know, and provide feedback and I'll revise my plans.

 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 4, Funny) by isostatic on Monday April 07 2014, @07:15PM

    by isostatic (365) on Monday April 07 2014, @07:15PM (#27673) Journal

    Allow more moderation with less moderators

    Fewer.

    My brain throws an exception when people incorrectly use less and fewer.

    (Score: -1, grammar nazi)

    Starting Score:    1  point
    Moderation   +2  
       Funny=2, Total=2
    Extra 'Funny' Modifier   0  
    Karma-Bonus Modifier   +1  

    Total Score:   4  
  • (Score: 2) by NCommander on Monday April 07 2014, @07:41PM

    by NCommander (2) Subscriber Badge <michael@casadevall.pro> on Monday April 07 2014, @07:41PM (#27688) Homepage Journal

    Fixed, thanks.

    --
    Still always moving
    • (Score: 1, Informative) by Anonymous Coward on Monday April 07 2014, @07:49PM

      by Anonymous Coward on Monday April 07 2014, @07:49PM (#27699)

      If you're open to corrections...

      bi-product -> byproduct or by-product
      perverse -> preserve
      My though -> My thought

      I was going to just ignore them, but you responded fairly amicably to the other guy.

  • (Score: 1, Offtopic) by hemocyanin on Monday April 07 2014, @07:42PM

    by hemocyanin (186) on Monday April 07 2014, @07:42PM (#27690) Journal

    I have mod points! But I'm not going to mod you offtopic because I'm going to post offtopic and I share the same pet peeve.

    The easy way to get less/fewer right, is to think about whether the thing you're talking about a single pile of essentially non-unique stuff vs. individually identifiable units of that stuff, or in some way related to that stuff.

    For example:

    less sand | fewer grains of sand.
    less water | fewer molecules of water.
    less work | fewer jobs.
    less moderation | fewer moderators.

    And sure, by playing grammar nazi, I fully expect to have made some error.

  • (Score: 2) by BsAtHome on Monday April 07 2014, @07:47PM

    by BsAtHome (889) on Monday April 07 2014, @07:47PM (#27694)

    More is less. You haven't read the manual, apparently.

    Pointing out and suppressing every error in natural language, spoken or written, makes the language stagnate. It also prevents any humorous pun from being part of the language.

    • (Score: 2) by hemocyanin on Monday April 07 2014, @08:07PM

      by hemocyanin (186) on Monday April 07 2014, @08:07PM (#27724) Journal

      re: More is less

      Interestingly, no it doesn't. There really isn't a "more" issue like less/fewer. (ah yes, I do realize this pedantic to the extreme, and not tracking the posters meaning of "more is less").

      e.g.,

      more sand | more grains of sand.
      etc.

      you could try more/many, but "many" just means many, not "more than some other number" like fewer does in the opposite direction.

    • (Score: 1) by egcagrac0 on Monday April 07 2014, @08:29PM

      by egcagrac0 (2705) on Monday April 07 2014, @08:29PM (#27760)

      LESS(1)

      NAME

           less - opposite of more

      DESCRIPTION

           Less is a program similar to more (1), but which allows backward movement in the file as well as forward movement.

      The man page for more doesn't say it's less, and the man page for less clearly says it's the opposite of more. ;)

      • (Score: 2) by BsAtHome on Monday April 07 2014, @08:38PM

        by BsAtHome (889) on Monday April 07 2014, @08:38PM (#27765)

        Actually, the man-page states "less - opposite of more", but then continues to state "similar to more" and then "but which allows backward movement in the file as well as forward movement".

        So, the man-page is inconsistent in that it says a) less != more, b) less ~= more and c) less > more.

        Less is more, more or less. That is the conclusion about less, at least according to the man-page.

        Anyway, it exemplifies exactly what I mean. The confusion is complete when you can make errors in natural language and that is a good thing.

      • (Score: 2) by maxwell demon on Monday April 07 2014, @08:56PM

        by maxwell demon (1608) on Monday April 07 2014, @08:56PM (#27779) Journal

        But less is more, more or less. Well, actually less is more than more, but at its core it's more.

        --
        The Tao of math: The numbers you can count are not the real numbers.
  • (Score: 5, Interesting) by hemocyanin on Monday April 07 2014, @08:09PM

    by hemocyanin (186) on Monday April 07 2014, @08:09PM (#27730) Journal

    Probably more important than moderation, is collapsing threads like this one. You need collapser mods, people who can identify these offtopic hijacks. That's a real problem on Slashdot -- the first post is some funny/amusing/stupid riff on the article, and you're scrolling for half a mile to get to the meat. A new kind of moderator with one type of mod would be interesting -- that's where offtopic should go and it should automatically collapse.

    • (Score: 1) by NCommander on Monday April 07 2014, @08:17PM

      by NCommander (2) Subscriber Badge <michael@casadevall.pro> on Monday April 07 2014, @08:17PM (#27744) Homepage Journal

      I actually toyed with the idea of score inheritance, so if a thread went up, the comments under it went up, and if a thread went down, well, same thing. The biggest issue is it might bring too much crap up with it, requiring overrated mods to go flying to bring it back down. History has shown people on the whole tend not to downmod much.

      --
      Still always moving
      • (Score: 3, Interesting) by cosurgi on Monday April 07 2014, @08:56PM

        by cosurgi (272) on Monday April 07 2014, @08:56PM (#27778) Journal

        Or maybe indeed a new mod-type: "+0 Should stay collapsed".

        --
        #
        #\ @ ? [adom.de] Colonize Mars [kozicki.pl]
        #
      • (Score: 0) by Anonymous Coward on Tuesday April 08 2014, @01:08AM

        by Anonymous Coward on Tuesday April 08 2014, @01:08AM (#27883)

        Collapsing a thread shouldn't be the responsibility of a mod.

        Users should be able to collapse/hide threads for themselves.

        So if User A wants to collapse a thread he/she can while User B sees the thread uncollapsed.

        • (Score: 3, Interesting) by maxwell demon on Tuesday April 08 2014, @06:29AM

          by maxwell demon (1608) on Tuesday April 08 2014, @06:29AM (#27992) Journal

          Indeed, ideally there should be a way to have a finer control on how comments are shown by defailt from your user page. Possibly even with some sort of specification language (non-Turing complete of course, so it cannot be used to attack the site). Since this site is targeted at nerds, this might also double-serve to keep this site a nerd site even as it grows, because it's mostly nerds which are not afraid of using text configuration files.

          --
          The Tao of math: The numbers you can count are not the real numbers.
        • (Score: 1) by jcaplan on Tuesday April 08 2014, @01:07PM

          by jcaplan (1870) on Tuesday April 08 2014, @01:07PM (#28116)

          This has been my #1 wish for a long time. Half the comments appear under a very early comment, but it is hard to scroll down to the next top-level comment. I just want a little button to press to hide all replies to a comment.

          Benefits:

          1. I don't have to look at all the off-topic crap.

          2. As others collapse their own views of uninteresting threads, more attention and comments may appear under interesting threads.

          3. Commenters who take the time to read the linked article or write a thoughtful reply will have greater visibility.

    • (Score: 4, Insightful) by Sir Finkus on Monday April 07 2014, @09:48PM

      by Sir Finkus (192) on Monday April 07 2014, @09:48PM (#27812) Journal

      I think maybe it would be helpful to have a "meta" thread once a week or so that can discuss the stories and site without flooding the comments section of the actual stories. I suppose journals fill this role somewhat, but I think people really need a centralized place.

      This obviously wouldn't get rid of all the off-topic stuff, but hopefully it should put a major dent in it.

      • (Score: 1) by Nugoo on Tuesday April 08 2014, @12:17AM

        by Nugoo (2584) on Tuesday April 08 2014, @12:17AM (#27866)

        That sounds like a good idea. You could stick the meta discussion onto a weekly "state of the site" update.

        --
        I explicitly place the above in the public domain.
  • (Score: 0) by Anonymous Coward on Monday April 07 2014, @09:08PM

    by Anonymous Coward on Monday April 07 2014, @09:08PM (#27785)
    We need FEWER grammar nazis and LESS complaining.
    • (Score: 2) by elf on Tuesday April 08 2014, @12:48PM

      by elf (64) on Tuesday April 08 2014, @12:48PM (#28105)

      Agree, although gramma is important gramma conversations aren't :)

      I did not moderate at the other site so I don't really know what D2 was all about, the suggestions here look good and I look forward to some modernizations :)