SoylentNews

• Kickstarter Hacked (Accepted)
• What "News for Nerds" Sites Should I Use? (Accepted)
• Lucas Nussbaum Re-Elected as Debian Project Leader (Accepted)

So, given my last journal, a writeup on how they work today. For the most part, my original story on this topic is true, but I changed a fair bit since then and now, nor did I go much into the thought process in how it was divined.

In contrast to the original system, the current one wants to keep a specific number of moderation points always in circulation, with the concept that mod points are a constantly moving and fluid item. Moderation simply doesn't work if there isn't enough of the damn things, and having too many wasn't a problem at all (Overrated exists for a reason).

The original idea is we should dynamically generate our pool of modpoints based on our activity levels, so the original implementation of this script took the comment counts for the last 24 hours, with the basic notion that every comment should have the potential to be moderated at least once. This number was multiple by two, and provided our baseline moderation count. Since we were based our mod point count on a 24h window, mod points were set to expire every 24 hours instead of every 72. At this point, I hadn't realized the fundamental problem with the slashcode moderation system; my thoughts were "need lots of mod points", "this is incredibly complex, I can do better". That realization came as I was stripping the old one out of slash.

As part of this, I also changed the eligibility requirements for moderation. Instead of having a specific number of tokens, I wanted only users who were active to get mod points. The ability to retain drive by moderations by lurkers was something worth maintaining, and part of what I suspect makes up the bulk of Slashdot moderations.

I also wanted to avoid the problem of "moderator burnout", or users getting mod points too frequently, and just being turned off from moderation. I know that happened to me on slashdot, and others as well who ignored modpoints (or chose to become ineligible). As such, I wanted there to be a cooldown on how frequently someone can get modpoints.

That being said, I didn't want everyone and their mother being moderators all at once, so I decided that 30% of all active users (defined (at the time) as anyone active within the last 24 hours) who had neutral or better would be eligible for modpoints.

Version 1 was fairly simple. It basically took the comment count for the last 24 hours, multiple by 2, this is the minimum number of modpoints that exist at all times. Take all users who were active in the activity_period, take mod_points_to_issue/(elligable_moderators*.3), and hand out those points equally. As a failsafe, the system will hand out ten mod points minimum (the underlying thought here being that I don't just want to get one or two modpoints; more is better, so lets take Slashdot's 5 and multiple it by 2).

And for the most part it worked. When we were in closed alpha on Thursday, we opened the test site to 100 users to try and test it in something resembling real world logic. And, for the most part it worked, because everyone was very highly active. You might see the mistake with that logic when applied to a production site.

Come go-live. User counts surge through the roof, active users are flowing in (can't believe we hit 1k users in a single day), and the moderation script starts handing modpoints in the thousands. At one point, there was close to 2000 modpoints in circulation at any given time).

For that moment, moderation was working well. Then users started going offlining, and EODing, or worse, users were getting modpoints when they signed off, and not seeing them until they signed in. The script was happy, 30% of users were moderators, but there were a lot of +1s. When I looked at the database, most people who had modpoints hadn't been signed in for hours.

Suddenly in a flash of inspiration, I saw the mistake. Slashdot could get away with handing out users with no activity level because even with 80% of their system being moderators, most people would be inactive at any given time. With our 30%, there simply weren't enough modpoints in the hands of active users.

So, in an attempt to salvage the situation, I did a critical adjustment on how the damn thing works. Activity periods for users was seperated into a new variable, and dropped to 1 hour (then five minutes, so any logged in user has a chance), and process_moderation had its crontab shorted to five minutes (it used to run hourly).

To keep modpoints constantly in circulation, expiration time was dropped to four hours, so only people who are active RIGHT NOW were moderators, especially since our editor team had posted 20 articles that day already. Whenever a user looses his points (via expiration or using them all), their slot is freed up, and a new user immediately gets modpoints.

That change in logic underpins version 2 of this script. Now the minimum count is what we hand out, except in the very rare case that we need more modpoints in circulation, in which case, the active users start getting more and more (up to a cap of 50, then it spills past 30 of users). For the most part, it seems to be working, comment moderation scores are generally going up, but it may still require further tweaking to make it work well. I generally am not seeing as many +3-5s as I like, but its right now a whole hell a lot better than it used to be.

I'm open to any thoughts, criticisms, or whacky ideas relating to how mod points are being dished out. Let me hear them below.

So for the curious (or the morbid), I thought I'd do a bit of a writeup on how modpoints worked in the stock slashcode. To my knowledge, this is how they work on slashdot.org today, and all other Slash sites.That being said, caveat emperor. I'm not QUITE sure I understood the code correctly, and I'm writing this from memory, but if enough people want it, I'll fish the old code out of git, and paste it here.

In stock slashcode, every user has something called a "Token" count, which represents their chances at getting modpoints. The best way to think of tokens is like chances at winning a raffle. Keep this in mind, as it will become relevant in short order. Tokens are (theoretically) generated from various clicks in the site UI,and are granted off some serious voodoo involving magic numbers, and other such insanity.

My best understanding is tokens are only issued after a specific random number of clicks are hit, and are later pulled out of the access log by the process_modertion slashd script. But more on that later. The logic that does this is fairly uncommented perl spread across several perl modules, so its rather hard to keep track off.

Tokens convert to modpoints on a strict ratio (if I remember correctly, its 8 tokens becomes one mod point, so you need at least 40 tokens to be eligible to receive modpoints, stock slash only hands out modpoints in increments of five).

Having tokens is not however enough to make you eligible for mod points, it only represents your chances at getting modpoints. When process_moderate kicked, it would go through and essentially dump the entire user table for users that had tokens, were willing to moderate, was not banned from moderation and within the 80% percentile of oldest accounts. This is where metamoderation comes into play. (note: this was true when metamod existed, firehose replaced it, and I have no idea how the logic (if at all) has been changed to handle that)

For users that had been metamodded, those acted as a weight, either increasing or decreasing your chances at getting modpoints in the system. For moderations that were good, you got additional chances in the index, and the reverse decreased them. It also appears that your individual metamods were (somehow) taken into account, but I haven't quite pierced the logic of that. As the metamod module is broken, I never looked to see how it works in practice.

Now, none of this promises you'll actually GET modpoints. As I said, its a raffle. And its a raffle that includes accounts that been inactive but still have tokens. At this point, random users are choosen to get modpoints, which then converts your tokens to modpoints. If you get picked more than once, then you get another increment of 5.

So far, so good? Right now, you might be asking what's the problem with that. Aside from being perhaps a bit longwinded, there seems to be nothing wrong. The problem isn't that the algorithm is wrong, its fundamentally broken.

If you want a hint, I recommend checking out http://slashdot.jp or http://barrapunto.com/ (which are the only other slash sites still on the net that I know of), and look for +5 comments. Take your time, I'll wait.

The problem comes from what ISN'T in the algorithm; it takes no account into how many modpoints MUST be in circulation. I had the advantage of being a frequent poster on macslash.org while it was still around. In the years I was active on that site, I can count the number of +5 comments I saw on one hand. +4s were almost just as rare.

For a comment from an normal user to get to +5, it needs four seperate people with mod points to vote it up four times, and it needs that many people who 1. have modpoint 2. want to use them 3. want to use them on THAT comment.

That's a lot of freaking ifs. While this site was still in closed-testing, the stock modpoint algorithm ran from Monday to Friday until I ripped it out and replaced it with my version of it. In that entire time, it issued a grand whooping total of 10 modpoints (5 to my dummy account which I use for account testing, I don't remember where the other 5 went). At that point, we were getting about 20 comments per article.

In short, the stock modpoint method is not only broken, it is fundamentally broken, and it only works on Slashdot because their userbase is large enough that it works out of dumb luck. Even then, I question that as a lot of good comments never seem to get to +2 or +3, let alone the higher tears. This is was prompted the rewrite, which I'll document in my next journal.

Testing some UTF-8:

占星術を科学的でないと考える米国人の割合が減少しているという米国立科学財団(NSF)のリポート(/.J記事)に対し、実は「占星術

More text, I just want to see the error message.

Don't know how much filler is required.

https://soylentnews.org/~MichaelDavidCrawford/journal/

Current dev notes:

Admin privs work, sec levels are as such
100 = editor
500 = editor with security (can use security page)
1000 = admin
10000 = su_admin

Need: Privacy Policy, way to respond to DMCA, what stats should be made public