Stories
Slash Boxes
Comments

SoylentNews is people

Meta
posted by NCommander on Monday November 21 2022, @08:25AM   Printer-friendly
from the Let's-figure-this-out dept.

So, to say the last week has been a dumpster fire is drastically underselling what I've been through. This, combined with having to put things in place to migrate off Twitter, and otherwise deal with all the fallout of that hot mess has, to put it frankly, put free time at something of a premium, hence why this post took so long. For those who missed it, I did fairly long overhaul of our backend, upgrading boxes from Ubuntu 14.04, and rebuilding and replacing others.

At the moment, the site is mostly working, with two exceptions, site search is still down, and IRC is still down. Deucalion has taken up the task of rebuilding the IRCd on modern server software, so it's time to lay down the road going forward past this point.

Read past the fold for more information ...

State of the Backend

Right now, the backend is mostly built on an outdated version of mod_perl 2.2, and MySQL cluster, which is very much not a good place to be. Originally as envisioned, I planned this site to be able to be easily scalable, with a larger user base. That's why the infrastructure was designed to be as scalable as it was, with the downside of having a much higher overhead than a more traditional setup has. Furthermore, rehash (the code that powers this site) is, uh, to put it frankly, a beast to work on. It's a 90s era Perl code base and pretty much everything else that implies; if it wasn't for the fact that rehash is one of the main reasons to use SoylentNews, I'd argue it might be time to replace it.

Right now, I'm working on doing another round of server hardening. As it is at the moment, I've got rehash and Apache running in an AppArmor jail, and everything is pretty well sandboxed from everything else, but I still need to go through and adjust a lot of firewalls, and finish decommissioning out a bunch of the boxes. That said, the site is running faster than it has in a long while since a lot of small things got corrected as we went. Sometime this weekend, I'm going to finish adjusting the firewalls to lock it down further, and that should mostly get back to the point where I might have restful sleep again. That being said, there's still a fair bit more to do.

Moving ahead, we need to get off MySQL cluster, and either onto the current mod_perl, or, ideally, FastCGI, to end the Apache dependency entirely. Unfortunately, working on Rehash is quite difficult, and it requires a very specific setup to be viable. My current plan here is to basically get it working in Docker, so its easy to spin up and spin down instances, and return to a less cursed variant of MySQL. This is probably a few hours of work, but I'm hoping that overall it is going to be easy and straightforward to do since most of the backend is fairly well documented at this point. This also leaves me in a decent position to implement a couple of long overdue features, but modernization efforts come first. I'm hoping to livestream my efforts on this on the weeks to come, and I will make stream announcements as I go along.

Policy and Code Changes

My intent, based off the policy changes that were made to disallow ACs to post on stories is to sunlight the feature entirely, including in journals and more. The decision to have ACs on SoylentNews was made in 2014, when the Snowden leaks were only a few months old. Furthermore, we've seen from experience that the karma system doesn't go far enough at keeping bad actors from still getting a +2 status. By and large, the numbers underpinning the system need a rework. My general thought is to cap karma at either 10 or 15, and drastically decrease how far into the basement you can go, as well as uncapping posts in moderation to be able to go to -5.

As a rule, incredibly bad takes do get moderated out of existence, but because there's no real penalty for doing so, we get constant shitposts. Time to make this a bit harder to abuse. I've documented the antispam measures on the site before, but the site keeps track of IP addresses and subnets in the form of hashed /24, and /16s (/64 and /48 for IPv6), which has a karma number attached to them. If an IP range goes too far into the basement, it ends up posting at 0 or -1. By adjusting the caps, it should allow this threshold to be reached much more easily, and help bring the signal to noise ratio back to something more "positive".

Furthermore, I believe its generally in the site's interests to allow editors to delete comments. This functionality is actually built into rehash, but has been long disabled. At the time, I felt the community was best self-moderating, but I think on the whole, its better to treat this like a moderated subreddit, and have messages get a notice that they've in-fact been deleted ala reddit. This is a fairly large departure for the site as a whole, but I think one justified given the state of the Internet on 2022. I am open to discussions on all of this, but let me see what all your thoughts are like.

Final Notes

I do intend to keep livestreaming my progress with the site as we go along; and we raised another ~500 dollars towards Trevor Project during the last livestream. I've left that stream unlisted until I've had a chance to finish implementing all the hardening measures I've discussed, but I'm hoping at the end of it, I'll have a pretty good documentary on what it takes to modernize an aging website. As usual, if you want to support me directly: Ko-fi is available for one time donations, or Patreon for a recurring donation.

~ NCommander


[ If you are an AC and wish to make a constructive comment, please see my journal. janrinok ]

 
This discussion was created by NCommander (2) for logged-in users only, but now has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 5, Interesting) by tangomargarine on Monday November 21 2022, @08:47AM (7 children)

    by tangomargarine (667) on Monday November 21 2022, @08:47AM (#1280769)

    My intent, based off the policy changes that were made to disallow ACs to post on stories is to sunlight the feature entirely, including in journals and more. The decision to have ACs on SoylentNews was made in 2014

    So wait...you're saying, no more AC posts anywhere, ever? Can't say I'm a big fan of this idea. After the fiasco of...whatsisface, aristarchus or He Who Must Not Be Named or whoever it was, hasn't it been somewhat demonstrated that being able to post anonymously is still a valuable feature? Because that 1 in 1000 whack-a-doodle will decide to devote his life to making multis and trying to destroy the entire site and doxxing you and hounding you until you stop posting?

    My general thought is to cap karma at either 10 or 15

    Hmm...how confident are we that the "multiple account" problem has been solved? Because if not it becomes that much easier for said abusers to stop honest users that tick them off from commenting.

    and drastically decrease how far into the basement you can go

    Not sure, but sounds abusable as per above. What is the current floor?

    as well as uncapping posts in moderation to be able to go to -5.

    No objection here, I guess, but would that have any added effects?

    As a rule, incredibly bad takes do get moderated out of existence, but because there's no real penalty for doing so

    Count how many posts a user makes that gets to -5? Admittedly that's still abusable with enough monkeys (see above).

    Time to make this a bit harder to abuse. I've documented the antispam measures on the site before, but the site keeps track of IP addresses and subnets in the form of hashed /24, and /16s (/64 and /48 for IPv6), which has a karma number attached to them. If an IP range goes too far into the basement, it ends up posting at 0 or -1. By adjusting the caps, it should allow this threshold to be reached much more easily, and help bring the signal to noise ratio back to something more "positive".

    Ah, details! Excellent! Thank you.

    Furthermore, I believe its generally in the site's interests to allow editors to delete comments.

    Hmmm...will there be any oversight system involved? Of course any monarch can be noble, but we roll the dice with each.

    And of course such a task is rather thankless. Will editors be notified on some sort of automated trigger, on certain posts? Or are they expected to do their own scanning?

    --
    "Is that really true?" "I just spent the last hour telling you to think for yourself! Didn't you hear anything I said?"
    Starting Score:    1  point
    Moderation   +3  
       Interesting=3, Total=3
    Extra 'Interesting' Modifier   0  
    Karma-Bonus Modifier   +1  

    Total Score:   5  
  • (Score: 2) by janrinok on Monday November 21 2022, @09:00AM

    by janrinok (52) Subscriber Badge on Monday November 21 2022, @09:00AM (#1280771) Journal

    See my https://soylentnews.org/meta/comments.pl?noupdate=1&sid=52490&page=1&cid=1280770#commentwrap [soylentnews.org]

    I agree with and support some of your comments.

  • (Score: 2) by Kell on Monday November 21 2022, @09:20AM (1 child)

    by Kell (292) on Monday November 21 2022, @09:20AM (#1280773)

    I'm normally supportive of the efforts the admins make here, but there are good points raised here. I do not support these initiatives - the green site worked well for years as it was, and it's a proven formula. I'm not opposed to deleting dox info but I agree that oversight is needed... somehow.

    --
    Scientists ask questions. Engineers solve problems.
  • (Score: 5, Interesting) by janrinok on Monday November 21 2022, @09:29AM (1 child)

    by janrinok (52) Subscriber Badge on Monday November 21 2022, @09:29AM (#1280774) Journal

    As you are probably aware, I have spent a large proportion of my time on here looking at multiple accounts.

    There are some accounts that always appear at the same time. moderate the same stories the same way, and support each other. However, there is at least one credible explanation for this (which has certainly been believed by some administrators here over the last 8 years) - they all work in the same office. The accounts belong to different people. If I can prove abuse of the moderation system or suppression of specific accounts then I will take action. We must nevertheless err on the side of caution. The accounts are tagged and I am watching them closely, both physically and with software. There are a few others too.

    The sock-puppet problem is now reasonably under control. The major abuses by sock puppets have stopped and sock puppet accounts are much harder to create without being spotted. New accounts are often identified and disabled within 90 seconds on average. But there will still be some on the site somewhere. If they begin to abuse the system then, again, I can hopefully spot them and disable them. There are a couple of sock-puppets that we have identified but are sitting dormant for the time being. I am taking no action but this is intentional - for now they serve a purpose that we can exploit to our own advantage.

    how confident are we that the "multiple account" problem has been solved

    If by solved you mean there are no multiple account holders then I must answer "No, there probably are some". Are they abusing the system to the detriment of other users? I don't think so; there is no evidence to support the idea.

    • (Score: 2) by tangomargarine on Monday November 21 2022, @05:08PM

      by tangomargarine (667) on Monday November 21 2022, @05:08PM (#1280833)

      I of course didn't mean to imply that the only acceptable answer to me is "yes, the system works perfectly now."

      The sock-puppet problem is now reasonably under control. The major abuses by sock puppets have stopped and sock puppet accounts are much harder to create without being spotted. New accounts are often identified and disabled within 90 seconds on average. But there will still be some on the site somewhere. If they begin to abuse the system then, again, I can hopefully spot them and disable them. There are a couple of sock-puppets that we have identified but are sitting dormant for the time being. I am taking no action but this is intentional - for now they serve a purpose that we can exploit to our own advantage.

      how confident are we that the "multiple account" problem has been solved

      If by solved you mean there are no multiple account holders then I must answer "No, there probably are some". Are they abusing the system to the detriment of other users? I don't think so; there is no evidence to support the idea.

      Thank you for your hard work!

      --
      "Is that really true?" "I just spent the last hour telling you to think for yourself! Didn't you hear anything I said?"
  • (Score: 4, Interesting) by Thexalon on Monday November 21 2022, @12:58PM

    by Thexalon (636) on Monday November 21 2022, @12:58PM (#1280796)

    I will just add that as an occasional target of mod-bombing at least in the early days, having the karma cushion is definitely a nice-to-have.

    And I also keep it in perspective: If the forum is large enough to bother trolling, there will be trolls. If you try to filter the trolls, you will be imperfect and also catch some legitimate people engaging in important free speech. If you manage to filter only the trolls most of the time, the trolls will complain that you're interfering with their free speech. If you try to create systems that increase the number of accounts and IP addresses involved, the trolls will set up one account for home, another account for the office, a third account that relays through the Tor onion, etc. Your only real block is the point where it becomes more of a pain in the butt to troll than it is fun or profitable to do it, which can take a while. So don't beat yourself up over being less than perfect at this, because it's pretty much a guarantee, and far better-funded outfits than Soylent ever was have failed at least as thoroughly.

    --
    The only thing that stops a bad guy with a compiler is a good guy with a compiler.
  • (Score: 2) by NCommander on Tuesday November 22 2022, @12:01AM

    by NCommander (2) Subscriber Badge <michael@casadevall.pro> on Tuesday November 22 2022, @12:01AM (#1280922) Homepage Journal

    My intent here was that AC posts would be removed entirely. Based off the comments, I'm seeing I'm in the minority about this, so perhaps I will have to go back and rethink this. Frankly, I don't think the feature belongs on the modern Internet, because it essentially allows for consequence free posting, and requires either editors or moderators to do something. I've literally seen first hand the consequences of this.

    There's a karma based subnet system, but IP addresses (even at a subnet level) change so often that I honestly don't think its useful as an anti-abuse measure. Keeping the "Post Anonymously" button may be the way to go, but put a karma cost on using it. I'm undecided at this very moment.

    Hmm...how confident are we that the "multiple account" problem has been solved? Because if not it becomes that much easier for said abusers to stop honest users that tick them off from commenting.

    While I won't say its perfect, you'd have to go out of your way to get multiple accounts to actual positive karma by posting something positive. I'd have to recheck the criteria in which mod points are given, but I believe its still just requiring positive karma to be used. By and large, most of the mod points used expire. Maybe this is too simplistic though

    --
    Still always moving