SoylentNews is people
SoylentNews
So, to say the last week has been a dumpster fire is drastically underselling what I've been through. This, combined with having to put things in place to migrate off Twitter, and otherwise deal with all the fallout of that hot mess has, to put it frankly, put free time at something of a premium, hence why this post took so long. For those who missed it, I did fairly long overhaul of our backend, upgrading boxes from Ubuntu 14.04, and rebuilding and replacing others.
At the moment, the site is mostly working, with two exceptions, site search is still down, and IRC is still down. Deucalion has taken up the task of rebuilding the IRCd on modern server software, so it's time to lay down the road going forward past this point.
Read past the fold for more information ...
State of the Backend
Right now, the backend is mostly built on an outdated version of mod_perl 2.2, and MySQL cluster, which is very much not a good place to be. Originally as envisioned, I planned this site to be able to be easily scalable, with a larger user base. That's why the infrastructure was designed to be as scalable as it was, with the downside of having a much higher overhead than a more traditional setup has. Furthermore, rehash (the code that powers this site) is, uh, to put it frankly, a beast to work on. It's a 90s era Perl code base and pretty much everything else that implies; if it wasn't for the fact that rehash is one of the main reasons to use SoylentNews, I'd argue it might be time to replace it.
Right now, I'm working on doing another round of server hardening. As it is at the moment, I've got rehash and Apache running in an AppArmor jail, and everything is pretty well sandboxed from everything else, but I still need to go through and adjust a lot of firewalls, and finish decommissioning out a bunch of the boxes. That said, the site is running faster than it has in a long while since a lot of small things got corrected as we went. Sometime this weekend, I'm going to finish adjusting the firewalls to lock it down further, and that should mostly get back to the point where I might have restful sleep again. That being said, there's still a fair bit more to do.
Moving ahead, we need to get off MySQL cluster, and either onto the current mod_perl, or, ideally, FastCGI, to end the Apache dependency entirely. Unfortunately, working on Rehash is quite difficult, and it requires a very specific setup to be viable. My current plan here is to basically get it working in Docker, so its easy to spin up and spin down instances, and return to a less cursed variant of MySQL. This is probably a few hours of work, but I'm hoping that overall it is going to be easy and straightforward to do since most of the backend is fairly well documented at this point. This also leaves me in a decent position to implement a couple of long overdue features, but modernization efforts come first. I'm hoping to livestream my efforts on this on the weeks to come, and I will make stream announcements as I go along.
Policy and Code Changes
My intent, based off the policy changes that were made to disallow ACs to post on stories is to sunlight the feature entirely, including in journals and more. The decision to have ACs on SoylentNews was made in 2014, when the Snowden leaks were only a few months old. Furthermore, we've seen from experience that the karma system doesn't go far enough at keeping bad actors from still getting a +2 status. By and large, the numbers underpinning the system need a rework. My general thought is to cap karma at either 10 or 15, and drastically decrease how far into the basement you can go, as well as uncapping posts in moderation to be able to go to -5.
As a rule, incredibly bad takes do get moderated out of existence, but because there's no real penalty for doing so, we get constant shitposts. Time to make this a bit harder to abuse. I've documented the antispam measures on the site before, but the site keeps track of IP addresses and subnets in the form of hashed /24, and /16s (/64 and /48 for IPv6), which has a karma number attached to them. If an IP range goes too far into the basement, it ends up posting at 0 or -1. By adjusting the caps, it should allow this threshold to be reached much more easily, and help bring the signal to noise ratio back to something more "positive".
Furthermore, I believe its generally in the site's interests to allow editors to delete comments. This functionality is actually built into rehash, but has been long disabled. At the time, I felt the community was best self-moderating, but I think on the whole, its better to treat this like a moderated subreddit, and have messages get a notice that they've in-fact been deleted ala reddit. This is a fairly large departure for the site as a whole, but I think one justified given the state of the Internet on 2022. I am open to discussions on all of this, but let me see what all your thoughts are like.
Final Notes
I do intend to keep livestreaming my progress with the site as we go along; and we raised another ~500 dollars towards Trevor Project during the last livestream. I've left that stream unlisted until I've had a chance to finish implementing all the hardening measures I've discussed, but I'm hoping at the end of it, I'll have a pretty good documentary on what it takes to modernize an aging website. As usual, if you want to support me directly: Ko-fi is available for one time donations, or Patreon for a recurring donation.
~ NCommander
[ If you are an AC and wish to make a constructive comment, please see my journal. janrinok ]
(Score: 5, Interesting) by janrinok on Monday November 21 2022, @08:50AM (5 children)
Administrators are NEVER above community criticism - we have to take it on the chin and justify why we are taking any particular action.
AC posting for account holders is essential for many of our community. It should be permitted anywhere on the site.
Deleting comments and even complete journal entries is sometimes justified but it is a very rare occasion indeed and it is not an action to be taken lightly. Doxing, pornography, political soapboxing, personal verbal assaults, incitements to violence etc have no place in our community. Doxing is particularly difficult - we cannot verify every statement made and we must therefore assume that, if it appears to reveal personal information, it is accurate and thereby act accordingly. This is one area where having the email contact of the abused/victim is highly desirable.
Deleting comments smacks of censorship - which I am categorically against. I do not want an echo chamber of repeated acceptable views. It must not become a method of expressing 'I Disagree' even by an editor. Moderation should be the first choice and deletion the very last option. The process should be, as a minimum. a 2 stage process. Any editor should be able to remove a comment from view regardless of it's current moderations. Often there is only 1 editor available to take action. The removal from view must therefore be easy to action but must also be reversible. The commentator should be notified that it has happened by automatic admin-to-user message therefore an AC comment must be attributable to a genuine account. True deletion, IMO, should require at least 2(?) different administrators in agreement and should also be controlled procedurally with records of when and why the deletion was carried out and by whom. If we do not keep such records we leave ourselves open to accusations that we are 'losing' an individual's comments or that we are doing something underhand. We must remain accountable.
The hijacking of another person's published story or journal entry to express entirely off-topic and irrelevant views is unacceptable. Discussions will often change direction and this is entirely normal but abuse by others who simply refuse to have an account to create their own journal is not.
Dummy or fake accounts (not sock puppets) should be removed from the database if they are not activated within a specific time period (28 days?). This will effectively counter the numerous fake accounts that are being created by bots - for what purpose I have no idea.
The current rules for sock puppets (they are forbidden) are sufficient.
The acceptability of throwaway or single-use email addresses for accounts should be discussed. The use of them makes creating sock puppets significantly easier than it need be, and makes the prevention of the abuse much more difficult. The detection and countering of sock puppets places a significant burden on administrators.
I am not interested in knowing who people are or where they live. My interest starts and stops at our servers.
(Score: 5, Insightful) by NotSanguine on Monday November 21 2022, @01:12PM (4 children)
I get the idea and nobody (except those whose goals include wasting everyone's time) wants to see constant shitposts, spam and other sundry nastiness.
But deleting content is something to which I am (mostly, with specific exceptions) categorically opposed, mostly because it can be abused, and unpopular ideas could end up caught in a too broad application of such a policy. Not that I think (at least given the current make up of the staff) such abuse is likely, but one of the things that kept me here for so long was knowing that what I (or others) express will be memorialized in the context of the discussions around that expression.
That said, no content should be allowed that might threaten the existence of the site (CSAM, doxxing, credible death threats, etc.) through the ruinous cost of litigation and/or the involvement of law "enforcement". SN couldn't continue if under such threats, and I want SN to continue.
As for other stuff, as unpleasant or disruptive as it might be, the moderation system is here to handle that, IMHO. An additional feature that might be useful would be akin to the '-10 Spam' mod (and even combined with it) to not display content that's been modded below a certain point, along with a Hacker News style 'showdead' option [ycombinator.com] for logged-in users.
But even that seems like a bit over the top to me, as I've always read at -1 and while some content was patently offensive and worthless, if folks don't see it unless they choose to do so, the jerks who post such garbage will be discouraged from doing so.
It would seem that such a 'showdead'-like option might be a good compromise between controlling shitposts (stuff that could threaten the existence of the site should be terminated with extreme prejudice, but that's not what I'm referring to here) and maintaining a mostly free speech environment.
I expect that some folks will disagree (or agree) with this take, and I'd be happy to discuss it with anyone who'd like to do so -- which is an important part of SN, and the main reason I've (mostly) stuck around since near the beginning.
No, no, you're not thinking; you're just being logical. --Niels Bohr
(Score: 5, Interesting) by acid andy on Monday November 21 2022, @02:18PM (3 children)
I agree provided, as I've suggested in the past, "not display" means the comment isn't sent to the client at all, rather than just being hidden by CSS or similar. Some of the more offensive shitposts here may not have been considered strictly illegal by the admins (possibly legal in the US and possibly not in other jurisdictions) but that doesn't mean all readers would be comfortable with having that content on their computer or browsing history.
While I'm commenting here I want to add that I really, really don't like the idea of completely disabling AC comments and don't even like the idea of banning them on journals very much either.
Welcome to Edgeways. Words should apply in advance as spaces are highly limite—
(Score: 2) by NotSanguine on Monday November 21 2022, @09:50PM (2 children)
An interesting point. Although I'm not sure that's a reasonable ask (not sure if a 'showdead'-type function is either), as (IIUC -- someone please correct me if I'm wrong about that) page rendering is done client-side rather than server-side.
If you live in a jurisdiction that criminalizes viewing certain speech (or even if you just live with folks who like to look through your browsing history -- a little creepy, but I try not to judge), installing browser addons to delete your cache/browsing history is probably a good idea even if you don't visit SN, whether shitposts are displayed or not.
No, no, you're not thinking; you're just being logical. --Niels Bohr
(Score: 0) by Anonymous Coward on Tuesday November 22 2022, @12:10AM (1 child)
Aren't going to work if data is collected by the service provider and/or government. Local files can also be undeleted unless extra care is taken.
I'm not certain exactly how it builds the pages of comments but consider that the server does make user specific alterations to the page such as showing your own user name and the modding controls if you have modpoints. And of course you can add modifiers to friends and foes which would alter their displayed comments scores on the page. I'm sure that's happening server side. So skipping over comments based on user settings ought to be possible on the server. It's just a question of how big the code changes would be and what side effects that change might have on the other features and code.
(Score: 2) by NotSanguine on Tuesday November 22 2022, @12:45AM
Yup. Good opsec is important where necessary. But that wasn't what GP was talking about.
And it's not my job to teach good opsec. I'd also expect that GP (and IIRC, they are fairly technical) doesn't need me (or anyone else) to explain how to stay secure online.
No, no, you're not thinking; you're just being logical. --Niels Bohr