SoylentNews is people
SoylentNews
So, to say the last week has been a dumpster fire is drastically underselling what I've been through. This, combined with having to put things in place to migrate off Twitter, and otherwise deal with all the fallout of that hot mess has, to put it frankly, put free time at something of a premium, hence why this post took so long. For those who missed it, I did fairly long overhaul of our backend, upgrading boxes from Ubuntu 14.04, and rebuilding and replacing others.
At the moment, the site is mostly working, with two exceptions, site search is still down, and IRC is still down. Deucalion has taken up the task of rebuilding the IRCd on modern server software, so it's time to lay down the road going forward past this point.
Read past the fold for more information ...
State of the Backend
Right now, the backend is mostly built on an outdated version of mod_perl 2.2, and MySQL cluster, which is very much not a good place to be. Originally as envisioned, I planned this site to be able to be easily scalable, with a larger user base. That's why the infrastructure was designed to be as scalable as it was, with the downside of having a much higher overhead than a more traditional setup has. Furthermore, rehash (the code that powers this site) is, uh, to put it frankly, a beast to work on. It's a 90s era Perl code base and pretty much everything else that implies; if it wasn't for the fact that rehash is one of the main reasons to use SoylentNews, I'd argue it might be time to replace it.
Right now, I'm working on doing another round of server hardening. As it is at the moment, I've got rehash and Apache running in an AppArmor jail, and everything is pretty well sandboxed from everything else, but I still need to go through and adjust a lot of firewalls, and finish decommissioning out a bunch of the boxes. That said, the site is running faster than it has in a long while since a lot of small things got corrected as we went. Sometime this weekend, I'm going to finish adjusting the firewalls to lock it down further, and that should mostly get back to the point where I might have restful sleep again. That being said, there's still a fair bit more to do.
Moving ahead, we need to get off MySQL cluster, and either onto the current mod_perl, or, ideally, FastCGI, to end the Apache dependency entirely. Unfortunately, working on Rehash is quite difficult, and it requires a very specific setup to be viable. My current plan here is to basically get it working in Docker, so its easy to spin up and spin down instances, and return to a less cursed variant of MySQL. This is probably a few hours of work, but I'm hoping that overall it is going to be easy and straightforward to do since most of the backend is fairly well documented at this point. This also leaves me in a decent position to implement a couple of long overdue features, but modernization efforts come first. I'm hoping to livestream my efforts on this on the weeks to come, and I will make stream announcements as I go along.
Policy and Code Changes
My intent, based off the policy changes that were made to disallow ACs to post on stories is to sunlight the feature entirely, including in journals and more. The decision to have ACs on SoylentNews was made in 2014, when the Snowden leaks were only a few months old. Furthermore, we've seen from experience that the karma system doesn't go far enough at keeping bad actors from still getting a +2 status. By and large, the numbers underpinning the system need a rework. My general thought is to cap karma at either 10 or 15, and drastically decrease how far into the basement you can go, as well as uncapping posts in moderation to be able to go to -5.
As a rule, incredibly bad takes do get moderated out of existence, but because there's no real penalty for doing so, we get constant shitposts. Time to make this a bit harder to abuse. I've documented the antispam measures on the site before, but the site keeps track of IP addresses and subnets in the form of hashed /24, and /16s (/64 and /48 for IPv6), which has a karma number attached to them. If an IP range goes too far into the basement, it ends up posting at 0 or -1. By adjusting the caps, it should allow this threshold to be reached much more easily, and help bring the signal to noise ratio back to something more "positive".
Furthermore, I believe its generally in the site's interests to allow editors to delete comments. This functionality is actually built into rehash, but has been long disabled. At the time, I felt the community was best self-moderating, but I think on the whole, its better to treat this like a moderated subreddit, and have messages get a notice that they've in-fact been deleted ala reddit. This is a fairly large departure for the site as a whole, but I think one justified given the state of the Internet on 2022. I am open to discussions on all of this, but let me see what all your thoughts are like.
Final Notes
I do intend to keep livestreaming my progress with the site as we go along; and we raised another ~500 dollars towards Trevor Project during the last livestream. I've left that stream unlisted until I've had a chance to finish implementing all the hardening measures I've discussed, but I'm hoping at the end of it, I'll have a pretty good documentary on what it takes to modernize an aging website. As usual, if you want to support me directly: Ko-fi is available for one time donations, or Patreon for a recurring donation.
~ NCommander
[ If you are an AC and wish to make a constructive comment, please see my journal. janrinok ]
(Score: 5, Interesting) by looorg on Monday November 21 2022, @01:27PM (5 children)
If you lower the karma cushion wouldn't that just make it easier to nuke people or opinions that you do not like? If the floor (or basement) gets lower then those accounts might never make it back. Rightful in some cases but others might just get nuked from orbit due to a difference of opinions, after all there still by modders isn't any kind of consensus on what constitutes trolls and disagreements etc or for that matter what is the other side of spectrum -- everyone finds different things interesting and informative. It's really down to opinions and perhaps there is to much emphasis put on hope or that the common greater good or sense will kick in and take over.
Perhaps this can be resolved by having some kind of karma-decay. It could work both on negative and positive numbers. Over time dragging you down (or up) to zero. If you had one (or many) bad posts eventually they'll be forgotten and forgiven over time as it decays towards zero again. Just as having a gigantic cushion, currently peeking at +50, would over time decay down towards zero or some upper nice_guy_bottom. Incentive to keep up the good work to keep you up there and not just live on previous postings. If mod points are given out each and every 24h cycle you might as well run all the accounts and +- the cushion according to some rules to such as moving a point per day towards zero or some other arbitrary limits.
If the roof is changed to being plus 10 or 15 instead of plus 50 are you still going to hand out 10 mod points per day?
If this then combines with a crackdown on disposable- or throwaway email addresses, whatever they are -- I used google as a disposable email previously but I assume it's more of the once that you don't even really register and you just keep for hours and then they are gone. Still what constitutes a throwaway address that won't be accepted should then be defined and stated, as noted by others.
While I would like an edit function, I can't even recall the number of times you write something and then post and then moments later you notice that your spelling was wrong etc. Always less then amusing. But then perhaps there should be a window on how far into the past you are allowed to edit a post before it becomes locked. Similar then with delete if you just want to remove it all. But it might not play nice with the current structure and people responding to posts that then just go away, will they also then go away or will they be somehow left hanging or will a deleted post just in essence be edited to say something that "post removed", but then what if the person that have previously responded to it had quoted etc. Weirdness could ensue.
Beyond that I think what has been said or noted as problematic have already been said.
(Score: 3, Interesting) by canopic jug on Monday November 21 2022, @01:39PM
Or there could be a half-life for each karma point so that it drops back over time, thus providing a very strong incentive for positive interactions. However, the full implications of that would have to be worked out in advance before adding that to the already long list of things to code to be sure that it can't be gamed in a way worse than the current situation.
Money is not free speech. Elections should not be auctions.
(Score: 5, Interesting) by fab23 on Monday November 21 2022, @02:47PM
In the HN FAQ (linked from somebody else), I discovered this part:
In my opinion this would be safe solution to be able to edit / delete a comment after posting. It is kind like the send delay some mail clients (web) provide, where you can abort if needed.
(Score: 2) by NCommander on Monday November 21 2022, @11:48PM
Right now, at most, you can loose three points per post if your post starts at +2, and you get lowered to -1. If you starting at karma 50, you end up at 47. Meanwhile, if you forgo the +1 boost, you can get +4 per post, and its often easier to go up than down. Submitting stories also gives a user +3 karma. The problem here is the constant low grade noise, which adds up and makes the site unpleasant to use. By and large, troll accounts tend to still have 40-50 karma on average since people do mod them up from time to time, and this includes misinformation on the site.
Still always moving
(Score: 2) by NCommander on Tuesday November 22 2022, @12:07AM (1 child)
By and large, it doesn't happen. Most of the major troll accounts here have positive karma, despite being downvoted into oblivion because they generally don't post with +1 bonus, so at most its a 2 point lost. One insightful post negates everything, and submitting stories also gives you karma back, even if they're not accepted. As it stands, you have to be at -10 to have your starting score lowered to -1, and -20 (I think) to have your score lowered to -1 to start with. That means you have to have a sustained period of nothing but garbage getting downmodded before the karma system has any effect. I don't remember seeing anyone actually get that low beforehand.
Oh, that's a good question. TBD.
If there was an edit button, it would be time locked to maybe 30 minutes, and show a flag that a post has been edited. Solves the quick mistake stuff, prevents really rampant abuse.
Still always moving
(Score: 2) by looorg on Tuesday November 22 2022, @01:37AM
Right. So the cushion is to large then, or the punishment to small. If you at most lose four points for a post out of 50 point cushion that is hardly a dent and it's as noted quite easy to fix your occasional trolling, bad or misunderstood posting. As noted moderation isn't always equal.
Still there is a big difference in losing 4/50 (2/25 or about 8%) points vs losing 4/10 (2/5 or 40%) or 4/15 (or 27%). One is clearly to small and insignificant to matter. While the others might be to large. Still if submitting a single story is more or less all that it takes to cover it up then one could argue if there really is a punishment or moderation at all. There is, or might be one, for the story people are reading but as a consequence for the poster? Probably not.
I would assume most people with an account are more or less at or around 50 karma as it is these days. The only real way I find out that I have been down-modded somewhere is if the number shrinks below 50, and I might not even notice since someone else could just pull it back up again since the order of moderation matters.
So getting to -10 points if +50 is the max then yes that would take some serious effort. Still even if the max was just +10 then getting to -10 is five complete garbage posts without anyone liking it or you making a single submission of a story either. So while it was unlikely at +50 changing it to just +10 might not really matter or have that big an impact either. Most of the plus +50 then is just really extra padding.
I'm not even sure how you would really solve it. Just closing the gap doesn't even seem to matter in that regard. Sure it's less padding but it's still a lot of wiggle room.
Regarding the edit option. Yes it might solve the quick fat finger mistake but on the other hand I don't really think it matters to much. Not to mention even if there was a window of editing I'm sure I would miss that one to and I wouldn't notice it until it just closed. It's just that you miss it when you don't have it but if you had it then you might not even use it anyway.