SoylentNews is people
SoylentNews
An article posted by Cory Doctorow on Boing Boing http://boingboing.net/2014/09/15/downvoting-considered-harmful.html has interesting insight into moderation:
A study http://cs.stanford.edu/people/jure/pubs/disqus-icwsm14.pdf [PDF] published in a journal of the Association for the Advancement of Artificial Intelligence found that sites that have a "downvote" button to punish bad comments lock the downvoted users into spirals of ever-more-prolific, ever-lower-quality posting due to a perception of having been martyred by the downvoters.
Cory continues: What's more, positive attention for writing good posts acts as less of an incentive to write more good stuff than the incentive to write bad stuff that's produced by negative attention.
How Community Feedback Shapes User Behavior http://cs.stanford.edu/people/jure/pubs/disqus-icwsm14.pdf [Justin Cheng, Cristian Danescu-Niculescu-Mizil, Jure Leskovec]
Why Reddit sucks: some scientific evidence http://www.washingtonpost.com/blogs/monkey-cage/wp/2014/09/09/why-reddit-sucks-some-scientific-evidence/ [Henry Farrell/Washington Post]
So... do you downvote? if so, why? Does this article make you reconsider your down-modding?
[Editor's note: I offer for your consideration and commentary our very own SoylentNews Moderation FAQ.]
(Score: 2) by martyb on Wednesday September 17 2014, @03:36AM
What if, instead of modding up or modding down, one could Rate it At?
As pointed out in TFA, other sites have basically no limit to how low or high a comment could be moderated. Along with that, is the concept that one can mod any given comment up or down.
Here on SN, we offer an essentially identical-appearing control wherein one can mod a comment up or down (with an attendant reason be it "Funny", "Insightful", "Interesting", "Insightful", "Troll", "Off Topic", "Overrated", "Underrated", whatever) but here, there is a limit on the ultimate moderation: -1 to +5. (Cue duck walking, quacking, etc.)
When I moderate I aim for rating consistency. I would expect, that if there were some way for me to look ONLY at comments moderated +5, that I would see the cream-of-the-crop, the comments that make me really think, that give me a fresh insight, that just make it all worthwhile. Similarly, if I could select to look ONLY at those comments moderated at +4, I would expect to see arguably VERY good, but not quite THAT excellent comments. Approximately:
Along with these ratings, we could continue to have the Reason Codes we have now (with the exception of the now-redundant overrated/underrated). What this does is make it explicit that there are limits here on how low or high a comment can be moderated. Since the control would no longer look like an up/down vote, it is my contention that it would be less likely to be used as one.
"That was really good. Made me look at things in a different way. Was not well organized, though, so I had to struggle to get the gist of it." Using the proposed system, I'd rate it at "+4", select a reason of "Insightful", and click on the Moderate button.
Under the covers, the code can look at the pre-existing moderation for the comment (let's say it was a "+1" and "Interesting".) Since my rating of "+4" is greater than the current moderation of "+1", the code determines that it needs to be modded up. In the current implementation on SN, that would be treated just as if I had moderated it "+1 Insightful".
My thoughts are this would encourage a mind-set of "Okay, that comment is VERY Good — that's a +4 and Interesting", "Hmm, nice try at humor but keep your day job — that's a +2 Funny". There is a subtle, but I would contend, significant difference in one's thinking when presented with these controls instead of the "Like" or "Up/Down" UI controls that users may be familiar with on other sites.
In other words, if it walks like a canary, and sings like a canary, do not make it look like a duck! =)
Wit is intellect, dancing.
(Score: 2) by Common Joe on Thursday September 18 2014, @10:23AM
I like your idea, but I'm not sure it would work out in reality. In fact, it's already been implemented at Amazon and IMDB. It's basically the 5 star system they use. There are a lot of people who only seem to give a one star or a five star; they have no sense of gray area at all. You and I would would give someone 2 or 3 or 4 stars for a comment, but others (even on SN) might just simply always give a -1 or 5. How badly would that skew the results?
Again, I'm not sayin' your idea is bad. I kind of like it, but I feel like it needs something else to overcome the problem I pointed out. Anyone else care to throw in their two cents and poke at this idea?
(Score: 2) by martyb on Thursday September 18 2014, @02:03PM
I'd seen "star ratings" before, but never made the connection between that and what I proposed; thank you for the insightful observation!
I've struggled for years to escape my innate perceptions of a black/white, true/false, all/nothing world. Shades of gray cause the permutation space of possibilities to escape my ability to envision all of them, so it continues to be something with which I struggle. I find I am getting better at escaping that objective perception bias and embracing a more subjective perception. On occasion, I even get glimpses of color... wow!
Yet, when it comes to rating things, I seem to have no difficulty in iterating through the choices and finding a non-extreme value. I have absolutely no idea why that is comparatively simple. As you point out, though, many others just reduce the choices to like/dislike, agree/disagree et cetera and thus dramatically skew the weightings.
I fear this is an attempt at a technical solution to a social (for lack of a better word) problem. Do you recall that old checkbox chart from back in the old /. days? My immediate thought is to throw back a dialog box "You are proposing to rate a comment at +5 when it is currently rated +1; do you REALLY think this is one of the best quality posts you have ever seen on this site?" And down the rabbit hole we go!
Mayhaps this is where meta-moderation would come to the rescue? You raise an excellent point and, at this moment, I have no solution to offer. I suppose one could ask a different question: "Given the foregoing, is the proposed solution any better than the existing moderation mechanism?"
Separately, I've been toying with the idea that each comment presented on the site be enclosed by an HTML DIV whose class specifies the comment's current score. Using your reply as an example, where we currently have:
Would instead be presented as:
Where: "p" == "plus" and "m"=="minus", so we'd have: mod_m1, mod_p0, mod_p1, mod_p2, mod_p3, mod_p4, mod_p5. Them, I could apply a user style sheet (yay CSS!) and present each of those with a different background-color or a different font-size, etc. This would provide a subtle visual cue wherein I could readily discern comments whose moderations were inconsistent against other comments on the site. Yet another embryonic thought, but I toss it out for your consideration and feedback.
Wit is intellect, dancing.