Join our Folding@Home team:
Main F@H site
Our team page
2022-07-02 10:17:28 ..
2022-10-05 12:33:58 UTC
2022-10-05 14:04:11 UTC --fnord666
Support us: Subscribe Here
and buy SoylentNews Swag
We always have a place for talented people, visit the Get Involved section on the wiki to see how you can make SoylentNews better.
On the 12 February 2014, SoylentNews published its first 'official' discussion wittily entitled "Welcome to SoylentNews!"
There were 23 comments to that story but that belies the incredible effort made by a small team of enthusiasts who were determined to create and manage a site that lived up to their own expectations of how a site should behave. There are no advertisements and we are not beholden to any corporate group or business interest. All the support is still completely voluntary. We have also committed ourselves to reaching at least the 10 year birthday celebration too.
In the first story, a certain NCommander made the following remark:
Now that we're here, we hope to have made the wait worth it, but we depend on everyone in the community. To make this site a success, we depend on each and every single user even if its just from passing word of mouth. Remember, every single user can submit stories, moderate, and contribute to discussions all at the same time, and that's what makes us unique. May I be the first to welcome you to your new home.
That is just as true today as it was then. We are in a similar position to that in which we found ourselves 9 years ago. Software is having to be rewritten and repackaged. It is still based on software written 10 years before we even went live. It works but has a specific set of requirements that are becoming increasingly more difficult to meet. Hardware is being restructured necessitated by the changing economic circumstances that we as a site, individuals, and a community are having to learn to live with.
To say that it has not all been smooth sailing would be an understatement. The world and our community have changed far more than anyone could have envisaged in those first few weeks and months. The site has had to change also - sometimes to enhance what we offer each other as a community and at other times to prevent all of the good work being undone by a small number of people, one result of their actions being to drive away a significant proportion of our community. But here we are - a bit battered and bruised but preparing to move forward while meeting both legal requirements and community expectations as we do so.
Being in the same situation that we found ourselves in 2014 means that NCommander's words are just as relevant today as they were back then. We rely on the community to help us in exactly the same way. We would like to rebuild the community and we can do that with your help by word of mouth. We still rely on your submissions, comments and moderations so please continue to make them. And we are also in need of people who are willing to give a small amount of their time to a wide range of tasks that are necessary to keep the site operational. There is no specific commitment of time or effort - just to contribute whatever you are comfortable with. Get in touch and let us know what interests you, and we will look for a role that will meet those interests wherever possible. If we find a match then we will both benefit but if not there is no commitment on you to join the team.
As I write this we are trying to introduce 2 new volunteers to the sys-ops team, and possibly another editor in the next few weeks - more on that once they have found their seats!
However, if nothing else, please keep doing what you have been doing to support the site - your comments and views are what makes this site what it is. The site is nothing without the community.
So it's been awhile since I last wrote, and I'm a bit overdue for a status update. So, let me give you all the short version on what's been going on.
First, I've been doing a lot of backend work to drastically reduce the size of the SoylentNews bill month to month. We had a lot of infrastructure that was either unnecessary, or have gotten so many free tier upgrades that they were being vastly underutilized. Along the way, I've given a lot of fine tuning to bits, although I won't say its been problem free, since we went a few weeks without working sidebars. I'm truly sorry for the delays in getting up and running. My personal life chose to become very exciting in December, and I'm still dealing with the fallout of that entire mess. As such, what I had planned went a bit pear-shaped, and I went unexpectedly radio silent. ...
More past the break ...
The biggest problem is that most of the backend is undocumented. I wrote some documents in the early days of the site, but by and large, the site was mostly maintained by individuals who are no longer active on staff. The internal TechOps wiki was woefully out of date, and even I find myself struggling to know how the entire site is put together. Considering it's been online for over 9 years, and was a bit of a rush job out the gate, well, you know, it happens. I think at some point at the decade mark, I will want to chronicle more about SN's history, but let's first make sure we've got a site when we get there.
By and large, I'm not involved in the day to day operations. janrirok has been, and is, at this point the de facto project leader. My role with SoylentNews these days is kinda vague and undefined, since I stepped down privately in 2020, and then stepped back last November. I also find myself very uncertain if I want to even be involved at all, but, ultimately, I was here at the start, and while SoylentNews was always a collaborative project, I left a mark on both what this site is and will be that has persisted over the better part of a decade.
As such, I feel personally obligated to get SoylentNews to the best shape I can possibly get it, and give it the best chance of success I can give it. However, we're in the uncomfortable situation that we have a dated Perl codebase running on undocumented infrastructure that has been creaking along with no major reworks in almost all that time. You can imagine I've been having a fun time of this. Most of the relevant information mostly exists in my head, since I was the one who got Slashcode running all those years ago.
Right now, my biggest victory is I managed to get us off MySQL Cluster, and onto a more normal version of MySQL which drastically reduces memory and disk load in favor of slower load performance.
Moving forward, the solution is to have a reproducible deployment system, likely based around Docker, or possibly even Kubernetes, with all aspects of rehash (the site software) documented. We use GitHub to handle site development, and I think it would be in our best interests to integrate a full CI pipeline for both development and production environments. While implementing this, I also intend to entirely redo every aspect of the backend, complete with proper documentation, so something beside me can actually maintain it. After that, it will actually be practical for SoylentNews to survive past a single person, and we can have a more serious discussion on what the road forward looks like.
I do realize that the last few months have had a lot of ups and down, mixed with excitement and disappointment. I can't really say for sure where we're going, but you know? I want us to reach that decade mark together, and then we'll figure out where we're going beyond that.
Until next time,
Some of you may have seen the claims in certain journals that SoylentNews is blocking new accounts. I have been looking at the actual figures for accounts created during December 2022.
A total of 2198 accounts have been successfully created during December. Unfortunately the vast majority of these were created by a bot and they will never become active because the bot does not have access to the email addresses being quoted. The email address is required so that the initial password can be issued. The site is designed to handle a much larger number of accounts and at a far greater rate than the bot is using. This causes us no problems whatsoever. It must be keeping somebody amused though. I think it would be nice to have some form of 'captcha' (NOT Google's fire hydrant, bicycle, bus and pedestrian crossing counter!!!) to prevent bots such as these.
It is possible, indeed likely, that there will be a small number of genuine accounts buried in there somewhere but it is difficult to identify them until they become active.
In those accounts we have successfully identified 28 attempts at creating sock-puppets which have been disabled automatically. This gives 1.27% of all attempts to create an new account are blocked, which is significantly less that the the claim that 'all new accounts' are being blocked.
The latest UID to be created is 22820.
Many of you are frustrated, as am I, by the bugs that are currently present in the system and wondering why they are not being fixed. I am now in a position to perhaps shed a little bit of light onto the reasons we are where we are...
There are significant challenges happening behind the scenes that even I was unaware of until an hour or two ago. One is a problem with our domain name itself. You may recall that in the very initial stages of the site the domain was taken by Barrabas who then only relinquished it for the sum of several thousand dollars. I do not have the details but is appears that there is some form of new dispute regarding the name, which NCommander and another board member are currently trying to resolve. If I have understood it correctly, it seems that another entity is trying to grab the name but all that I have to go on are a couple of cryptic comments on IRC. If and when I have more information I will pass it on.
Addendum: After a discussion with another member of staff I believe that our own domain name is safe. NCommander is having an issue with a domain elsewhere. I will keep you informed.
NCommander has a professional life away from this site (as do all the staff with myself being the only exception). There has been significant upheaval in his work over recent weeks and, as a result, he has not had the time that he had expected to have to work on the site. Others are contributing where they can but any resolution of the current problems is unlikely to happen before Christmas. He has also made commitments that he has to honour and he is burning the candle at both ends for the moment. At the end of the day he has to pay the rent and put food on the table. The site is an entirely voluntary and unpaid activity for all of us.
The three major bugs (lack of mod points, the failure to display new journal entries, and the search facility not finding recent stories) are all slashd issues. Some community members are currently scouring the logs/code to try to pinpoint the precise error so that, when NCommander does have some time, the necessary software changes can be carried out quickly.
There has been a bit of a feeling of pessimism over the site recently - so I thought I would do some analysis of the figures to see exactly what the situation is.
Since September we have had a total of 465 unique accounts contributing with comments - and most contribute many times. To be honest that figure is higher than I thought it might be. There are more who have been moderating but not commenting. Figures for how many people actually read our site regularly are difficult to calculate accurately because we cannot count those who do not log in. But by comparing current active participants and page hits with historical data gives a rough estimate of a community of around 700-800 people but that potentially has a wide margin of error. There has also been a reappearance of quite a few accounts that have been dormant for the last couple of years who have returned to the site. I hope that this trend continues.
I was also concerned about the fall in the number of comments since non-logged-in AC posting was disabled on the main pages. Again, it turns out that it is significant but it is not as bad as the figures first suggest. When I account for the number of off-topic, troll, and down-modded posts the comment figures are lower than they were a few months ago, but only marginally so. The quality of discourse is much higher now and is, I think, more acceptable to community as a whole. I am not suggesting that ACs do not make a significant and valuable contribution to the discussions. In fact, it is clear that the majority of Anonymous Cowards do act sensibly and contribute just the same as most other members do. But to enable them to return to the main stories we have to have a way of differentiating between each AC rather than having to control them all in a single account. A recent proposal on how to achieve this was rejected so, for the time being, I am concentrating on other tasks.
It has been claimed that we are not accepting new accounts. This is simply untrue and it is a statement being repeatedly made by an Anonymous Coward. As at the time of writing we have a total of 21836 accounts, but over the last few weeks a 'fake account' bot has been running and has created well over a thousand account entries which simply do not exist. However, there have been 77 new accounts created recently which have been active on the site. These figures do not include the many sock-puppet accounts that have been created; over 600 have been identified and disabled. As far as I can tell, and contrary again to many AC claims, only 3 or 4 of them ever belonged to Runaway1956 and none have been active since last year.
Without a doubt one of our biggest problems currently is funding. NCommander has managed to reduce our server count significantly and has therefore reduced our running costs. People everywhere are struggling to make ends meet and many have to make a difficult decisions regarding how to spend their money. If you are able to make a subscription - no matter how big or how small - it would be appreciated by the entire community.
Over the holiday period we will be reducing the story output to 5 stories a day (the 'weekend' rate). I will be travelling to visit my own family many of whom I have not seen since the start of the Covid pandemic. I will only have periodic internet access, and we will try very hard to find current newsworthy stories to publish. Good submissions are always welcome and we try to select those which have a submitter's identity over those that are created by some of our alternative collection bots.
I read somewhere that there are actually 11 different holidays being celebrated around this time of year. So on behalf of all of the team here at SoylentNews may I wish you, your family, and those close to you, the very best wishes for the holiday period and into the New Year.
We are aware of several bugs in the system at the moment, including the lack of a journal index on the front page, and the failure to update modpoints daily. Thank you to all of those bringing them to our attention. Please continue to do so as you may be the first person to see something going wrong.
Unfortunately, NCommander is currently unavailable to fix these immediately but he is aware of them. He is occupied with real world issues and currently has a very limited internet connection. He will fix them as soon as is possible and we ask that you be patient. It is less than ideal but unfortunately we have no other option.
Well, it's been a bit of time since the last time I posted, and well, I had to think a fair bit on the comments I received. It's become very clear that while I'm still willing to at least help in technical matters, the effort to reforge SN is much higher than I expected. In addition, given the, shall we say, lukewarm response I got to my posts and journal entries, well, I'm clearly not the right person for the job.
I think at this point, it's time to figure out who is going to lead SN going forward. After my de facto stepping down in 2020, the site has, for want of a better word, been a bit listless. At the moment, no one on staff really has the cycles to take that position on. A few people have expressed interest in the position, and I've talked with Matt, who is co-owner of the site about this. By and large, whoever fills the seat will have to figure out what, if anything, needs to change in regards to moderation policy, content, and more.
If you're interested in potentially fulfilling the role, drop me an email at michael -at- casadevall.pro, with the subject of "SN Project Leader", and include the following:
I'll leave this call for candidates open until December 14th, at which point Matt and I will go through, and figure out our short list, I'll talk to editors, and solicit more comments from the community. I'm hoping to announce a successor in early January, and formalize the transition sometime in February, which will be the site's 9th anniversary.
So, to say the last week has been a dumpster fire is drastically underselling what I've been through. This, combined with having to put things in place to migrate off Twitter, and otherwise deal with all the fallout of that hot mess has, to put it frankly, put free time at something of a premium, hence why this post took so long. For those who missed it, I did fairly long overhaul of our backend, upgrading boxes from Ubuntu 14.04, and rebuilding and replacing others.
At the moment, the site is mostly working, with two exceptions, site search is still down, and IRC is still down. Deucalion has taken up the task of rebuilding the IRCd on modern server software, so it's time to lay down the road going forward past this point.
Read past the fold for more information ...
Right now, the backend is mostly built on an outdated version of mod_perl 2.2, and MySQL cluster, which is very much not a good place to be. Originally as envisioned, I planned this site to be able to be easily scalable, with a larger user base. That's why the infrastructure was designed to be as scalable as it was, with the downside of having a much higher overhead than a more traditional setup has. Furthermore, rehash (the code that powers this site) is, uh, to put it frankly, a beast to work on. It's a 90s era Perl code base and pretty much everything else that implies; if it wasn't for the fact that rehash is one of the main reasons to use SoylentNews, I'd argue it might be time to replace it.
Right now, I'm working on doing another round of server hardening. As it is at the moment, I've got rehash and Apache running in an AppArmor jail, and everything is pretty well sandboxed from everything else, but I still need to go through and adjust a lot of firewalls, and finish decommissioning out a bunch of the boxes. That said, the site is running faster than it has in a long while since a lot of small things got corrected as we went. Sometime this weekend, I'm going to finish adjusting the firewalls to lock it down further, and that should mostly get back to the point where I might have restful sleep again. That being said, there's still a fair bit more to do.
Moving ahead, we need to get off MySQL cluster, and either onto the current mod_perl, or, ideally, FastCGI, to end the Apache dependency entirely. Unfortunately, working on Rehash is quite difficult, and it requires a very specific setup to be viable. My current plan here is to basically get it working in Docker, so its easy to spin up and spin down instances, and return to a less cursed variant of MySQL. This is probably a few hours of work, but I'm hoping that overall it is going to be easy and straightforward to do since most of the backend is fairly well documented at this point. This also leaves me in a decent position to implement a couple of long overdue features, but modernization efforts come first. I'm hoping to livestream my efforts on this on the weeks to come, and I will make stream announcements as I go along.
My intent, based off the policy changes that were made to disallow ACs to post on stories is to sunlight the feature entirely, including in journals and more. The decision to have ACs on SoylentNews was made in 2014, when the Snowden leaks were only a few months old. Furthermore, we've seen from experience that the karma system doesn't go far enough at keeping bad actors from still getting a +2 status. By and large, the numbers underpinning the system need a rework. My general thought is to cap karma at either 10 or 15, and drastically decrease how far into the basement you can go, as well as uncapping posts in moderation to be able to go to -5.
As a rule, incredibly bad takes do get moderated out of existence, but because there's no real penalty for doing so, we get constant shitposts. Time to make this a bit harder to abuse. I've documented the antispam measures on the site before, but the site keeps track of IP addresses and subnets in the form of hashed /24, and /16s (/64 and /48 for IPv6), which has a karma number attached to them. If an IP range goes too far into the basement, it ends up posting at 0 or -1. By adjusting the caps, it should allow this threshold to be reached much more easily, and help bring the signal to noise ratio back to something more "positive".
Furthermore, I believe its generally in the site's interests to allow editors to delete comments. This functionality is actually built into rehash, but has been long disabled. At the time, I felt the community was best self-moderating, but I think on the whole, its better to treat this like a moderated subreddit, and have messages get a notice that they've in-fact been deleted ala reddit. This is a fairly large departure for the site as a whole, but I think one justified given the state of the Internet on 2022. I am open to discussions on all of this, but let me see what all your thoughts are like.
I do intend to keep livestreaming my progress with the site as we go along; and we raised another ~500 dollars towards Trevor Project during the last livestream. I've left that stream unlisted until I've had a chance to finish implementing all the hardening measures I've discussed, but I'm hoping at the end of it, I'll have a pretty good documentary on what it takes to modernize an aging website. As usual, if you want to support me directly: Ko-fi is available for one time donations, or Patreon for a recurring donation.
[ If you are an AC and wish to make a constructive comment, please see my journal. janrinok ]
Alright, I've literally been at it for seven hours. Here's what done, here's what needs to done, and if you find something broken, let me know.
Here's what's going to happen sometime tonight or tomorrow
There's likely a lot of things still broken. if you find something broken, leave it in the comments. I'll get to it in the next day. At least we're not on 8 year old software anymore. Current plan after taking care of the above, going to find a good host (probably Fastmail) for editor emails, I do have the backups of everyone inbox should they want it. This has been, very much an exercise in pain. If you want to support me directly: Patreon for recurring, or Ko-fi for one time.
I'll be back in a few hours to work on this more. We raised another $560 USD for charity, and I hope folks had a great time. The next parts I need to just do, so the next stream will likely involve dealing with getting rehash to run in Docker.
Update - 2022/11/12: DNS resolved, and at least site outbound email should work now
Update - 2022/11/13: Site outbound email is actually working now
So, quick update here. The site was down for most of the night because the database cluster shot itself in the head. I had restarted a machine to install updates, and this caused the backend cluster to entire to entirely loose its mind. Unfortunately, I didn't have a manual dump of the database made, just a VM snapshot, since, well, I wasn't tinkering with it directly. I've mostly been trying to patch things to the point that I can sleep, and leaving things down like IRC and email which need to be seriously overhauled before they can go back up.
As far as damages go, it looks like we lost 10 or so days of messages, which uh, sucks for multiple reasons. We're currently on ##soylentnews on Libera.Chat while I pull bits of the site out of the flames, but I'm at the point that if I don't sleep, I will make things worse. Corruption in the production database is very much not what I wanted, and we're very much in limp mode for the moment. I'm going to let staff handle IRC and comments while I sleep, and then I'll post another update when I'm awake.
See you in a few hours
It's been a long while since I wrote one of these, and well, to say things are depressing is very much an understatement. It's been over eight years since we first went online, and the world has literally changed several times over. Presidential elections, a global pandemic, war in Europe, and well, we've been here through it all. It's a testament to our staying power that SN has remained online through it all, as a volunteer and community driven project.
That's not to say it's all been good news though ...
About two years ago, I de-facto resigned from the project after internal conflicts, and SoylentNews has slowly been rotting to death. To say the state of the backend is bad is very much an understatement. I found the SN emails were on spam blocklists, and well, I won't even talk about the state of the software ATM. It's holding together mostly out of the sheer amount of overengineering, and good intentions. The last major overhaul was I did when the site was migrated to rehash 7 years ago ... yeah it's been awhile ...
At this point, I think we need to talk about where we're going, because its either going to be long slow painful death, an execution, or an attempted comeback tour. This is your host NCommander, and today, we're going to look towards the future ...
I guess I should start with what happened to me, since I was head of the project, and I essentially disappeared. For those who knew me, I went through some serious life problems from 2016-2019; I was still active on Soylent's IRC, but I was very much one degree removed from the project. I did write things up like talking about my time as a ICANN fellow, and screwing around with Windows 1.0 SDK, but it was very much a hands off thing. Even at that point, the backend was very much starting to show its age in the years since it first went online. However, there were a few things in the background that was threatening to brew over.
When we launched in 2014, SoylentNews was promised as a baston of free speech, and essentially a testament of the Internet of the early 90s. It was the Slashdot we all remembered, freed from corporate interests that had forced the beta interface. I actually spent a bunch of time documenting what I remember of the SoylentNews launch on my own wiki here, but if you want to remember what lead to the foundation of this site, gaze upon the eye of the beta interface and despair. The end result of this was the SoylentNews Manifesto, essentially our touchstone document of our core values and such. I was 26 when I wrote that document. The Snowden leaks had just happened, and well, the biggest threat on the horizon was the idea of mass censorship. I had, somewhat naively, had assumed that, given a choice, people won't willingly listen to misinformation. Then I saw the Trump presidency first hand, combined with active efforts to lie and distort the truth about a global pandemic, and leading up to the Capitol Riots on January 6th, 2021. From today, it's been eight years since I put pen to paper, and I've had a long time to think about those words.
By and large, while I don't disagree with the principles of what SN was founded on, I've been forced to admit that this has had a lot of undesirable consequences. As the Trump presidency continued, the signal to noise ratio began to drop on the site like a stone. I remember that, at times, this site sometimes felt like reading /b/ more than anything else, and was having serious concerns with the state of the community by 2017-2018. However, there were two major factors that stopped me for doing anything: first, I felt bound my own words, and second, there were members of the staff who preferred to keep things as is.
By and large, volunteer projects depend on the organization have a core set of ethics. While SoylentNews is, simply put, a success story in volunteer collaboration, I'm the one who set the direction in which staff followed. This was true when I took ownership of the project from John, and when we were all at mrcoolbp having a BBQ celebrating our road to incorporation. Even though I haven't been active on SN in years at this point, you can still see the impressions in the clay that have lasted over the better part of a decade. However, this has never been a one man show. Several members of the staff, primarily those who helped maintain the backend, felt that the status quo was better. I could have forced the issue, but I would have likely ended up alienated both staff and the community over it. As such, I just began to silently slide into the background from 2017 onward.
As I continued to move forward in my life well, I began to have the benefit of hindsight. Towards late 2015, and early 2016, I got picked up by Mixer (then called Beam), and created the Faster-Than-Light streaming protocol, redefining the standard of what was possible for video streaming. I then found my way working with whistleblowers, and doing a stint of policy work with ICANN. It's given me the benefit of hindsight, and a lot of wisdom that 2014 didn't me did have. I also struggled with constant health issues. In 2019, I made the decision to try and cross the United States by bike (which I documented on my second channel, Restless Yankee. I started that trip in March of 2020; no points guessing how that ended :)
It's hard to summarize my feelings about the site at that time. By and large, I felt like SN had been a well intended, although ultimately misguided effort to make the world better, and the state of the comments section reinforced that. However, that changed in 2020, with the pandemic. Medical misinformation was rampant on SN, and there were several large fights about this in staff-only IRC channel. I was essentially outnumbered; there were too many people who wanted the toxic waste dump. So I resigned. Or well, intended to. I wrote a fairly lengthily resignation letter, and staged it for release on SoylentNews, and looked at disentangling myself from the PBC that owns the site. For various reasons, I never put it public, although it was briefly leaked.
In the end, the result was the same. I left SoylentNews, and focused on my own YouTube channel, and started building my own community built around the lessons learned from the SN era of my life. By and large, I've succeeded beyond my wildest dreams.
But while I was off doing this, things were changing here.
Well, as it turns out, I left some very large shoes to fill, and my frustrations about the site didn't fall on deaf ears. I won't go through the full details; for one, I wasn't here, but problems relating to spam, misinformation, and more continued to grow. As before, the people who actually understood how the very legacy Slashdot.org derived codebase worked in were in favor of letting the site continue as is, while those, like janrinok, were beginning to reach a breaking point. I'll skip ahead to the punchline, the staff ended anonymous posting on the main page, which was a major departure from our initial promises
To say there was backlash is very much an understatement. When the dust settled, there were a small number of staff, mostly editors, left, who continued valiantly on for the next two years as the site slowly began to fall apart without maintenance. It's a testament to how much effort was put in both by the original Slashdot crew, and those of us who worked on rehash that it basically stayed up despite no one watching it, but it's starting to reach a breaking point. I took a look at the state of things earlier today. It's *bad*. To put it bluntly, SN's backend was always maintained on what could be described as "best effort", and there were some deferred maintenance issues piling up when I stepped away.
It's now two years later ...
Fortunately, simply because when we launched SN, we were forced on an obsolete stack, I did an obscene amount of machine hardening; the entire thing is running in an AppArmored bubble, and I left a stupid amount of notes on how it all worked. Add to that Slashdot was very the tech site of the 1990s and 2000s, and you have a pretty tough nut to crack. The pile of Perl running this site is old enough to drink. However, we've piled up a large amount of technical debt. By and large, this is not a sustainable situation, but at least for the time being its livable if someone put in the necessary TLC to make it keep going.
Since moving off from SN, I've been working to actively preserve pieces of history; for example, through a community effort taking place mostly on my Discord, we've seen AIX for Itanium get archived, and even restored to the point we have a working copy of GCC. This was one of the centerpieces of the SCO v. IBM lawsuites of the early 90s, and was a huge focus point for Groklaw. The legendary Project Monterey; preserved for future generations, and I was there when it happened. We've also worked to save versions of Banyan VINES, and I'm even hosting a "Slow Computing Speedrun" as we speak, which, among other things, has someone with a genuine PDP-10 being livestreamed at this moment! You could say it's been an interesting time ...
Recently, I was talking about SoylentNews, and what I could remember relating it being forked from Slashdot, which I was urged to write in a public channel. The discussion basically boiled down to "how often do you get to siphon a large amount of the Slashdot old guard". This lead to a bunch of volunteers to write an entire page relating to this sites history; and as many notable things as I could remember over the last eight years. The full log has been saved here, but I'll give you some samples:
On Slashdot Beta
On soylentnews.org being held hostage
On upgrading the code to mod_perl 2
After finishing that interview, I ended up taking a much closer look at the state of the site, and found myself immensely depressed. This is how it ends? With a wimper? Nah. I've made an entire career out of doing the fucking impossible. This site is proof of it. So, here I am again, preparing to step once again into the breach ...
Just because of how much time I spent on it, I refuse to let SoylentNews die a slow painful death, but I also don't want to be in the position where I'm going to have to maintain it as a one man show indefinitely. 8 years ago, we built this community ex nilihio in a week. I can find the time to at least get us back to serviceable. Right now, I'm probably looking at many hours of work ahead of me, but there's a silver lining. I do a lot of charity livestreaming on cursed and vintage technology.
So, why not livestream it? The honest truth is a lot of people like to watch me suffer excessively, and this is going to be a test of patience beyond anything else. It will also (hopefully) bring a lot of fresh eyes looking towards SoylentNews, and perhaps teach some people on what it takes to keep a 30 year old codebase going. It's literally an artifact of a bygone era, one that is, rather depressingly, disappearing before our eyes. However, there's a bigger thing I need to look at.
February will mark our ninth anniversary, and frankly, I think we can make it to a decade with just a bit of effort. At which point, I'm going to need to decide what we're going to do. I don't want to see a long slow painful death. If anything, I rather do an orderly shutdown, archive everything for posterity, and leave soylentnews.org as a marker that says we were here, and let me stress this point, if nothing changes. If we can find new staff, and folks who are willing to maintain the site, then SoylentNews will go on, for as long as there are people there are to tend for it. If not, then it will be left as a testament of what is actually possible when people get together.
Assuming the community is willing to go along with this plan, I'll likely host a stream this weekend doing a fairly through inspection of the site infrastructure, and fixing any critical issues I can find live. I'll put a stream announcement up on SoylentNews, on my personal Twitter, and on Mastodon (@firstname.lastname@example.org), as well as on my YouTube channel.
At least some of these streams will be fundraising for charity. Currently, I've fundraising for The Trevor Project, while in the past, I've fundraised for both National Network of Abortion Funds, and Planned Parenthood of America. I'm pretty sure the comments section might have thing to say, and frankly? I don't really care. If you don't like it, you can put in the effort yourself to save this place.
I think, at this point, this NCommander novel has reached its final form, so I will drop a link to my Ko-fi, and Patreon for one time and recurring support for me directly.
Until next time, this is NCommander, signing off, wishing you all a pleasant day ...
[This has been recovered from the WayBack machine from a link (https://web.archive.org/web/20221111011540/https://soylentnews.org/meta/article.pl?sid=22%2F11%2F10%2F1428220) very kindly provided by an Anonymous Coward. I cannot reproduce the original comments - there is no mechanism in the software to do such a thing - nor should it ever have been necessary. If you want to copy the comments that you made and resubmit them you can, of course, do so but they are unlikely to appear in the same order as they were made in the original posts. JR]
The proposal to enforce AC posting for logged in members only on the main page was promulgated to all staff and members of the board 7 days ago. Thank you to all those who contributed to the earlier discussions and clearly expressed their own views, suggestions and potential enhancements. All are being studied for implementation, if feasible, when staffing and resources permit.
There has been unanimous agreement from all responses received in favour of the proposed restriction. However, it was also apparent that there was a wish that this will be only until other alternative methods of restricting spamming, abuse and other disruptions to discussions can be identified and implemented. This is unlikely to be achievable in the short to medium term; other sites are struggling unsuccessfully with the same problem. The long-term aim remains to include AC posting in all discussions if at all possible
Therefore, beginning immediately, all AC posting on the main site will be limited to registered members who have logged in to their account. We regret that this leaves a number of AC community members unable to contribute as they once did, but anonymity remains a personal choice.
This will not affect discussions in journals which will have no limits and will be open to all.
If there is a demand for it, I will look at alternative methods of publishing a small number of stories each day into a journal.
On a more positive note, there is evidence that because of the recent restrictions on AC posting a significant number of existing accounts have returned and are commenting in the discussions. The quality of discussions (i.e. signal-to-noise ratio) is significantly better than it was several weeks ago. Although we have lost overall numbers of comments, the value of many of those lost comments appears to have been quite low. There has also been a noticeable improvement in moderations being awarded with more positive moderations being given when compared to negative ones. It is too early yet to draw any firm conclusions from other site statistics.
After experiencing extensive spamming, ad-hominem attacks, and trolling it became necessary to protect the site by preventing all Anonymous Coward (AC) comments by anyone who was not logged in. This was a reluctant measure but it proved to be 99.9% effective. It was however, far from ideal. It partially isolated many of the responsible ACs who contribute regularly to the site and provide a valuable input to many of our discussions. They are still able to use journals for posting as the editors of SoylentNews have no control over the content of journals or the comments made in them.
I sought an alternative solution and provided access to some of the stories on the front page and marked them as AC Friendly. I had hoped by demonstrating to those abusing the site that there was a simple solution that they would perhaps cease, or at least return to previously manageable levels. Unfortunately this was not to be. The abuse restarted almost immediately and has continued in every AC Friendly story that has been published. It has clearly demonstrated that this isn't a case of the abusers defending free speech or any other laudable and justifiable aim but simply an attempt to prevent the majority of the community from holding any form of discussion at all. I am not continuing the AC Friendly stories on the main page with the sole exception of this Meta story.
I next tried to switch the attempts to include our AC community around by providing stories from the front page initially to my own journal, but subsequently to the journal of a new account named 'AC Friendly'. This was rather labour intensive and was not something that I could continue to do in the long term. These efforts have been ignored and do not seem to be of any interest to the AC community. Likewise I will not continue this effort unless there is evidence that it is wanted.
There are many perfectly understandable reasons for wishing to post comments as an Anonymous Coward. This was recognised when the original Slashdot code was written and provision was made for such individuals in the software. It is a straightforward matter to log in to the site and then automatically post as AC from then on. This both protects the site itself and those using it. If your justification is that you do not trust the staff then I must question why you would want to remain on the site.
Free speech is an essential part of our ethos but it is necessary to realise that free speech and anonymity are not necessarily related. We want people to be able to express their views without fear of harassment, abuse, or unfair moderation. Only by doing so can we truly claim to have free speech. It means that even those with whom we strongly disagree have the right to express their opinions. Subsequent attempts to argue against those views should not involve any form of harassment of the individual making them. Any attempt to prevent someone from expressing their views is directly counter to the very concept of 'free speech'.
Likewise, anonymity is something to be valued. Attempts to unmask either named or anonymous accounts is unacceptable to this site's administration and will not be tolerated. Those who publish information that appears credible to us must be deterred from continuing by whatever means are necessary. We cannot verify every claim made regarding the personal information of a community member and we must therefore assume that it is has some basis in truth and is an attempt at doxing. It does not matter where the information stated in the claim originates or whether it has been stated on this site or elsewhere previously. If it has the potential to unmask a community member it will be treated as doxing. The site will do all it can to protect community members. We are also fortunate that in the 8 years we have been operating we have only had one account that felt it was an acceptable thing to do. That account has been closed.
There is absolutely no doubt whatsoever that the toxic environment that has developed on the site has cost us numerous valued community members - both staff and regular contributors of submissions and comments. It cannot be allowed to continue. There is also no doubt that there is a straightforward and simple solution, and that is to prevent AC participation without the creation of an account. The software was designed to do this and it is wasted effort trying to find alternatives when it is unnecessary to do so.
There has been quite a bit of discussion over the last week or so. We have had a former community member (who was also once a member of staff) return to the site with his own story. Initially he chose to remain anonymous but subsequently decided to continue his comments under his username. I encourage you all to read the link given and the subsequent comments given in reply. I am very grateful that has taken the effort to explain why he has did what he has done and I welcome him back to our community if he choses to stay. As part of my reply to him I made the following statement:
It is vitally important that everyone is able to express their own point of view without harassment or intimidation or even unfair moderation. We do not all agree with each other. That is the same in any community. But by full, frank and honest discussion we can at least understand each others point of view and possibly identify potential solutions. The freedom of expression is still essential on the site - but it can only exist if we can ensure that it can be conducted in a suitable environment.
I stand by that statement. Since that comment was published I have received other views and experiences of the toxicity of our site from a significant number of individuals, including regular community members and both current and former staff. Quite simply, if we do not change then in all likelihood we will not survive much longer. It is not too late to make the necessary changes but time is running out.
I promised you that no changes would be made to how the site operates without first giving you all the chance to express your own opinions. But you have to decide now which path you want the site to follow. This cannot be a simple vote - as an extreme example we have no way of verifying that AC comments are not the result of a single person, or if some sock-puppets are still active on the site. Everyone has the right to be heard. However, let me point out a few rules:
This is an important issue. It cannot be a simple vote but I encourage as many people as possible to express their opinions. It might be the last chance for you to do so. The Meta will stay active for several days to at least mid-week - but if it is abused excessively then it will be taken down and we will be forced to make a decision base on whatever views we already have or can get from elsewhere. I will endeavour to move the Meta in the story queue so that it remains on the front page. Many of our community log on at different times of the day or only on specific days. I would like to give everyone a chance to see the Meta story and to make their views known.
This is your opportunity - please do not waste it.
[Ed's Comment: See bold text - warning 2022-07-10 12:36 UTC]
I realise that this has been an unpleasant time for many of our anonymous community members, but I can assure you that it has been necessary. I am not yet prepared to go into details but I can at least update you with our findings so far. But first we have to look at some historical data.
Anonymous Cowards (ACs) have always been - and will hopefully continue to be - welcome members of our community. There are many perfectly understandable reasons for wishing to post as AC and how you chose to live your own personal life is of no concern of this site. Equally, you are welcome to use VPNs and other security measures to protect your privacy. We take similar measures to protect all of your data so that you will not be compromised by us. These measures are effective and to the SN administration ACs appear as a single user with the user identity of #1.
We cannot treat some ACs differently from others. While we can manage to sort out your comments etc with the aid of the hashes that we produce, they change so frequently as to be useless for any purpose outside of this site. But the Administration is only concerned with what happens within this site and so this point is moot. We have no interest in the rest of the internet so IP addresses are also of no interest to us. How your comments get from wherever you are to us is irrelevant. The bottom line is that ACs can only be treated as a single account. That account is granted certain permissions or not granted those permissions and they apply to every AC interaction.
Most of our community, both logged in and AC, participate in the discussions in an reasonable manner and discuss the topic that has been outlined and any threads that resulting from it. It is true that, particularly at weekends, there is a slight increase in the number of ACs appearing but on their own they are little more than a minor irritant. There is, however, a 3rd group, consisting of ACs who sole purpose seems to be to derail any sensible discussion. Over recent years they have become more aggressive and often use personal attacks rather than challenging what is being said. Some are more obvious than others and I am sure that you can all think of examples of such people for yourself. A very small number have stated that it is their aim to prevent SoylentNews from continuing.
On 22 Jun of this year we received an implied threat (https://soylentnews.org/comments.pl?noupdate=1&sid=49894&page=1&cid=1254201) suggesting the the person making it had a target date of 6 July for some event or other. It is possible that this is related to another 'prophecy' in which this individual foretold that the site would soon be dead. We believe that we can identify the person making that threat with a reasonable degree of certainty. However, since that time the number of ad-hominem attacks has increased and we have also been subjected to increasing amounts of spam. In small amounts either or both of these things can be shrugged of, but when they come increasingly aggressive and frequent, they can make the entire experience of being in this community very unpleasant. I know that we have lost both staff and numerous community members because of this toxic atmosphere - and not, as some would have you believe, because we administer the site!
Almost all of this behaviour is conducted by a very small number of ACs and occasionally via sock-puppet accounts. As the levels of harassment increased over the last few weeks it was obvious to us that we could remove it by simply preventing AC access. This was not an easy decision to make but we knew that we could protect the majority of the site by this simple action. The result is, as you know, that we reluctantly removed anonymous access by ACs to the front page.
We are now actively looking for more permanent solutions and hopefully to exactly what we had before. I have experimented with providing stories on the front page which are AC friendly, and also in my journal. We are still looking for a better solution but unless we can separate individual ACs then I cannot see what else can be done. I would welcome your feedback and suggestions. The outcome of our decision is also our loss as you can see if you look at the numbers of comments that we are now getting compared to before the ban.
I have spent a lot of time analysing the posts, both current and historical, to try to identify the person or persons responsible for this unwanted content. I am not going to name specific individuals because I believe that you can each reach your own conclusions. By looking at both the spam and comment content, and their meta data, I have established the following.
The person spamming our site is one of our own Anonymous Cowards who is currently blocked because we have removed access for the AC account - and that block affects all ACs. He is also one of the people regularly carrying out ad hominem attacks against other community members. He will be reading everything that we post about this issue.
Unless the abuses cease everywhere on the site including in journals, ACs will remain outside of the main site except for specially released stories until we can devise a better system. For us to currently do anything different would be foolish and irresponsible in the extreme. As soon as the abuse ceases we can readmit all ACs to the main site again.
I know that this will be as much of a disappointment to you as it is to me, and you may also be thinking of leaving. I ask you not to go. Rather I would encourage you all to let the abusers know that they are not fighting for your freedom of speech ("freeze peach") but they are by their actions actively preventing your participation in our site. There is one particular post (https://soylentnews.org/comments.pl?noupdate=1&sid=50204&page=1&cid=1257692) which suggests that this is being done on behalf of all ACs and that you all stand as one behind this action. I don't believe that anybody has the right to claim that if you haven't actually agreed to it.
Thank you SO much! I had no idea that my work on SoylentNews had such a wide-ranging impact on so many people! Reading the comments to my resignation — This is Difficult in So Many Ways; I Must Resign from SoylentNews — brought me alternately to tears of joy and roars of laughter.
That said, the reality is that others have arguably played a larger role in making SoylentNews into the success that it is today. A few names that readily come to mind: NCommander (UID 2!), paulej72, TheMightyBuzzard, and LaminatorX (our first Editor-in-Chief). There are so many more!
As much as I appreciate the kind words, it would mean so much more to me to see these well-wishes turned into subscriptions to SoylentNews! This idea came to me when I received a gift subscription to the site. (Thanks drussell!) It costs roughly $7,000 per year to run this site — primarily web hosting fees. (Nobody has ever been paid anything for their work on this site!)
We have raised about $1,100 so far this year. Our goal for June 30th is only about $2,400 away (and for the year is only about $5,900 away). That is net to us after processing fees. It's a BIG stretch, but I have faith... let's see what this community can do!
[Update: Revised paulj72 to paulej72. - Fnord]