Slash Boxes

SoylentNews is people


Log In

Log In

Create Account  |  Retrieve Password

Site News

Join our Folding@Home team:
Main F@H site
Our team page

Funding Goal
For 6-month period:
2020-01-01 to 2020-06-30
(All amounts are estimated)
Base Goal:


Covers transactions:
2020-01-01 00:00:00 ..
2020-01-26 13:24:49 UTC
(SPIDs: [1207..1215])
Last Update:
2020-01-26 13:25:20 UTC

Support us: Subscribe Here
and buy SoylentNews Swag

We always have a place for talented people, visit the Get Involved section on the wiki to see how you can make SoylentNews better.

posted by The Mighty Buzzard on Sunday January 19, @03:40PM   Printer-friendly
from the lingere dept.

Well, crap. I have no idea why or when it happened but the Threshold setting when using Threaded-TOS appears to be non-functional at the moment. It's supposed to set the value below which a comment and any of its subcomments will be collapsed, unless a subcomment is over the Breakthrough value which should cause that comment only to be expanded. Right now it's functioning as if Threshold were set to 6. I never noticed it because I have both settings set to -1.

I can't monkey with it right this second but I'll see if I can get it fixed some time this weekend. Just a hotfix patch to the live code not a full site update.

Beats doing construction work in the rain I suppose.

Update: Okay, I can't fix something that ain't broke and TOS is functioning as intended. I just forgot that Threshold applied only to top-level comments, all subcomment trees should be collapsed by default, and Breakthrough was the setting for subcomments to show up no matter what. This doesn't make sense to me but then I'm not the one who decided it should function like that and I don't use TOS. If you lot want it to function differently or want a new mode that's similar, drop your insipid inspired ideas here and if there's enough demand I'll put it on the todo list.

posted by martyb on Tuesday January 14, @08:00AM   Printer-friendly
from the editors-wanted dept.

The observant reader will notice that we have reduced the number of stories we post on weekdays from about 15 stories per day to about 13 stories per day. We would certainly like to continue with the higher rate, but we have been struggling to do so with current staff.

We try to post enough stories each day so that there is "something for everyone". Ultimately this site is for the community. It is also by the community; it does not run all by itself.

We Need Your Help

People's lives change. They move, get married, have health issues, change jobs, etc. All of these place additional demands on their spare time. SoylentNews is not immune to this; in fact we have experienced all of these. With less free time available, more work falls upon the other staff members — whose lives are already quite full.

What would help?

Volunteer! Have you ever thought about being an editor at SoylentNews?

You'll get to learn a super-sekret handshake, the passcode to enter our volcano-lair, and the admiration of your fellow Soylentils!

Right from the start, let me point out that we — SoylentNews — aim to be impartial. If you have an agenda that you would like to push or advocate (or denigrate) then skip to the next story.

On the other hand, not a great deal of time is needed. Even if you have only an hour or two per week that you could contribute, that would be greatly appreciated!

An earlier request for help summarized things nicely (slightly updated here):

Well we are all volunteers, so we contribute what we can, when we can, no one is expected to edit X stories/day (make your own hours). I would say a strong recommendation (maybe not requirement) is signing onto IRC once in a while (especially when editing) so one can communicate with the other editors. Editing a few stories a day, or even a few a week would be welcome. A typical submission takes me about 15-30 minutes to edit, usually on the longer side if I'm expanding or adding a balanced point of view without trying to put words into the mouth of the submitter. The rest is pretty much what you'd expect:

  • Opening up a story
  • Checking links
  • Checking basic grammar and spelling
  • Correcting wording
  • Expanding a weak submission (sometimes it's just a few links and a sentence or quote)
  • etc.

One also needs to:

How else could I help?

Nominate someone. Have you noticed someone who tends to compose well-written comments or journal entries? Think they might be a good fit? Please let us know!

How else could I help?

Submit a story! A well-written story that needs little editing takes a huge load off of the editorial staff. If just a few Soylentils submitted a story each day it would make a world of difference. Ideally, a story submission would be grammatically correct, have no spelling errors, be balanced and impartial, and be "publication ready". But even if your writing skills are not up to (or are even far from) that level, we are experienced at "cleaning things up", so do not let that keep you from making a story submission! If you see something tech-related that you think would be of interest to the community? Send it in!

How can you reach us?

Send an email to editors (at) soylentnews (dot) org. Or drop a message on our IRC (Internet Relay Chat) channel. Just issue the command "/join #editorial" and leave a message expressing your interest, along with your SoylentNews nickname, and someone will get back to you.

What's in it for me?

That is a good question. Speaking for myself, it started with my getting onto IRC and occasionally pointing out an error I'd seen in a story. Sometimes it would take a while before I could get the attention of an editor who could make the correction. I got frustrated. Finally, I suggested that if I were made an editor, I could fix things myself without having to track people down. Over the 5 ½ years since then, I have made mistakes... and learned how to own them, in writing, in front of all of you. I've developed a thicker "skin". I've made great friendships with people all around the world. The technical expertise on staff here is amazing; they've taught me so much. And, I hope, I've been able to teach them a thing or two, too.

Original Submission

posted by martyb on Tuesday January 07, @02:45AM   Printer-friendly

It brings me great pleasure to announce that — thanks to a few last-day-of-the-year subscriptions — SoylentNews has successfully reached its fund-raising goal for the second half of 2019! (Note: these amounts are unaudited and are, therefore, approximate.)

During the period 2019-07-01 through 2019-12-31, we received a net total of $2036.33 (from gross receipts of $2121.64) on a goal of $2000.00! There were 71 paid subscriptions which ranged from $4.00 up to $200.00

Here's a great big thank you to all who have subscribed to SoylentNews... every contribution is important!

Original Submission

posted by martyb on Wednesday December 25, @06:00PM   Printer-friendly


To those of you who may be celebrating a holiday at this time of year, on behalf of the staff at SoylentNews, please accept my best wishes to you and those you hold dear.

For Fun:

With glee I am pleased to announce that The Mighty Buzzard has given a gift to all registered users on the site. These are good until 2019-12-26 00:10:00.

Thank You!

As the end of the year approaches, I extend my humble thanks to all of you who have subscribed to SoylentNews!

As of this writing, we have received $1799.47 towards a goal of $2000.00 for the second half of the year. All funds received go directly to supporting the site: web hosting, domain name renewal, tax preparation, etc. Nobody has ever received any money for their work on SoylentNews. We are staffed entirely by volunteers who give of their free time to keep the site running and the story queue filled. By my estimate, we have probably passed the minimum funds received that are needed to support the site for this half of the year... but actually reaching the goal would give us a bit of a cushion against the unexpected. To those who have let their subscription lapse and to those who may have never subscribed before: please subscribe and help us meet our goal.


Rarely mentioned, but SoylentNews does have a folding@home team which recently surpassed one billion points earned and now holds 218th place in the entire world! Here are views of their progress individually and as a team. Their efforts contribute to helping medical research into Alzheimer's disease, Huntington's disease, and many forms of cancer, among other diseases.

This Year So Far:

The SoylentNews web site has been available 24 hours day, 7 days a week... except for a couple relatively brief unplanned outages. Further our IRC (Internet Relay Chat) has been available throughout this time, as well. (There may have been a brief downtime due to scheduled Linode maintenance on our servers.) Also, behind the scenes, all the supporting back-end stuff has been chugging along: web servers, databases, caches, E-Mail processing, and lots more. Those of you who have been with us since the start remember the frequent site crashes of those early days. Often several times per day! The rarity of site crashes, now, is a testament to the hard work of many people who gave of their time and expertise to make SoylentNews so stable, today.

These are the folk who quietly mind the underpinnings of and automate everything so well you don't even know they are there and would likely feel embarrassed if attention were brouht to them. Please join me in thanking them, anyway!

Some Numbers:

The editorial staff at SoylentNews has posted 4,884 stories to SoylentNews since 2019-01-01. With a very conservative estimate of 5 minutes per story, that amounts to 407 hours' time. Put another way, that is 10 full-time person-weeks.

Please join me in thanking the editors who have so generously given of their free time to make this happen!

Thanks Again!

Lastly, I am taking advantage of this opportunity to again thank everyone who offered their support during my recent health issue. I'm still a little wobbly typing with my left hand's pinky, but as things could have gone, I'll take this any day of the week, instead! It is my sincere hope that by sharing my experience, it may help someone else make a change in their life and avoid my experience or much worse.

Original Submission

posted by martyb on Saturday November 23, @04:10AM   Printer-friendly
from the trying-to-please-the-Momma-Bear,-the-Papa-Bear-AND-the-Baby-Bear dept.

We get a lot of story submissions here on SoylentNews. Well over 37,000 so far and still counting. (THANK YOU!)

We occasionally see a story submission from an Anonymous Coward or new user that is an obvious attempt to increase the number of views of the linked story. We regularly decline to publish those stories. The self-seeking or self-promoting aspect is rather obvious. Or they have a political viewpoint they want to spearhead. They stand out from the more mainstream stories that you have seen posted here over the past 5+ years.

The next-scheduled story gave me pause.

I was somewhat hesitant because of the foregoing reasons. But, I decided to accept it anyway and I'd like to explain my reasoning. One, the story submission comes right out and candidly admits it is from the author. Two, the story provides a level-headed description of a technical problem, its causes, and suggestions for mitigations. Three, it provides reasons why the suggested changes would be beneficial to all readers; not just people in the target audience. Further, the submitter has been an active member and contributor to the site — even before it went live. I've seen him at work and have never seen any self-seeking behavior or pushing of an agenda of any kind. Quite the contrary, all of the contributions I have seen have been strictly for the benefit of the site and the community. Lastly, the submission had been reviewed by another editor and the only question is the one that is now posed to you here.

What do you think?

Is this the first step down a slippery slope? Should we have a "bright line" policy that strictly rejects even the appearance of self-promotion? Should the editors just use their best judgement, based on their past experience and informed from the feedback to this story? Other?

Original Submission

posted by The Mighty Buzzard on Wednesday November 13, @11:10PM   Printer-friendly
from the lingere dept.

Just a quick note to let those of you who care know that our load balancer finally got bumped up to openssl 1.1.x and is now TLSv1.3 happy. For those of you who are especially paranoid, "ssl_early_data" is explicitly set to "off" in the nginx conf file, actively disabling 0-RTT, even though it's disabled by default.

That's all, carry on.

posted by Fnord666 on Saturday November 09, @05:24PM   Printer-friendly
from the hoping-for-a-speedy-recovery dept.

The observant among you may have noticed I'd been posting fewer stories recently.

On October 27th I went to the hospital. After a physical examination, CT scan, and MRI, it was determined that I had had a minor stroke. I had mild loss of use of the pinky and ring finger of my left hand as well as some loss of fine motor control in my left arm and leg. A couple days later saw me in the operating room with a stent being inserted in my right carotid artery. I was released on Halloween day for a few days' recuperation at a relative's home and am now cleared to go back to work. I'm starting easy with slightly shortened work days, as I lack the stamina I had before.

Still, as these things go, I can't help but think of how fortunate I am. Nothing internal seems to have been affected (heart, lungs, etc. all working fine.) I am right-handed, so no problems there. No problem with talking or swallowing, so that's a huge plus. Thanks to neural plasticity and prescribed exercises, I have already mostly recovered. The lack of stamina manifests as my just being too tired after a day at work to be able to do much in the way of posting stories out to the site. As for my mind, I can attest that I am still as tarp as a shack! =)

As much as I would like to think I'll just bounce back to normal in no time, I acknowledge that my activities here will be at a somewhat diminished capacity for a while; time will tell.

With my absence, the rest of the editorial team rose to the occasion and kept the main page fed with stories. This meant extra time and effort on their part. Please join me in thanking Fnord666, Janrinok, cmn32480, takyon, chromas, NotSanguine, and CoolHand for their efforts to help push out stories to the main page. There were probably others whom I failed to notice; please accept my apologies for their omission. Call them out in the comments, and join me in thanking them for their efforts.

On a related note, it is my pleasure to announce that Fnord666 has accepted my invitation to step up to fill the position of Alternate-Editor-in-Chief. When janrinok was Editor-in-Chief (EiC), I accepted becoming Alternate-EiC, and when he stepped down as EiC, it was a privilege to take on becoming the EiC. I foresee no imminent demise on my part, but recent events made it abundantly clear to me the value of having this position filled. Please join me in congratulating Fnord666 on his promotion!

(For completeness' sake, I wish to point out that all of the staff at SoylentNews are volunteers. Nobody here has ever been paid anything for their work on the site. Any monies received when you subscribe go towards paying hosting fees, domain name registration, tax preparation expenses, and other costs required to keep this site running. Speaking of which, we are at nearly 70% of our goal for this half of our fiscal year—many thanks to those who have already subscribed!)

Lastly, I now place a request to the community. With the holiday season coming, my free time will become even more limited by work demands. It would be such a help to us to see story submissions from members of the community. It's really not that hard to do. Take a look at what the general layout of each article looks like on the site. Nothing very fancy or elaborate is needed. Click the Submit Story link in the SlashBox on the left-hand side of the main page of this site. Provide a title for the story, select a topic, provide a link to what you are writing in about, and a few paragraphs from the linked story would prove extremely helpful to us! We aim for mostly tech-oriented stories, but that is not an absolute requirement. Please submit your story in English (either North American, British or other variant is fine). Do be aware that we aim for balanced reporting here; attempts to push an agenda with one-sided, biased, or slanted submissions will likely result in the story being declined. The story submission page provides a link to the Submission Guidelines that, if followed, greatly improves the chance that your story will be accepted. (NB: We tend to relax the standards a bit on the weekend so as to include slightly offbeat or humorous stories; we appreciate a good laugh, too!

Original Submission

posted by martyb on Saturday October 19, @09:00PM   Printer-friendly
from the so-much-for-uptimes dept.

[Update (2019/10/19 21:02:00 UTC): Both sodium and fluorine have rebooted. Next up will be beryllium rebooting 1d12h from now. That leaves rebooting of helium 18h later and 3h after that will have boron being rebooted. Again, any impact visible to the community should be minimal. See TMB's note, below. --martyb]

We have just learned that Linode, the provider of SoylentNews' server infrastructure, is planning a number of reboots.

[TMB Note]: This shouldn't mean any downtime for anything user-facing except IRC. There will be a few minutes where the comment counts won't update on the front page but those aren't realtime anyway and a few minutes where subscription updates will be delayed until the server that processes them comes back up.

Recently, we identified a commit to the upstream Linux kernel[1] as the cause of an increase in emergency maintenance on our platform. After implementing, testing, deploying, and gaining confidence in a fix, we are now ready to roll this update out to the remainder of our fleet. We're confident this will resolve the bug and ultimately lessen the amount of unplanned maintenance for your Linodes as a result of this specific issue.

To complete this, we will be performing maintenance on a subset of Linode's host machines. This maintenance will update the underlying infrastructure that Linodes reside on and will not affect the data stored within them.

If you are on an affected host, your maintenance window will be communicated to you via a Support ticket within the next few days. You can prepare your Linode for this maintenance by following our Reboot Survival Guide[2].

During the actual maintenance window, your Linode will be cleanly shut down and will be unavailable while we perform the updates. A two-hour window is allocated, however the actual downtime should be much less. After the maintenance has concluded, each Linode will be returned to its last state (running or powered off).

This status page will be updated once maintenance is complete.


The first server reboot is currently scheduled for Friday, 2019-10-18 at 05:00:00 UTC.

Read on after the fold for more details on the scheduled maintenance dates and times.

Note: All dates and times are in UTC:

Affected systems:

lithiumNo Maintenance RequiredLinode 4GB 
magnesiumNo Maintenance RequiredLinode 2GB(pending upgrade)
sodium2019-10-18 05:00 AMLinode 2GB 
fluorine2019-10-19 02:00 AMLinode 8GB(pending upgrade)
helium2019-10-22 03:00 AMLinode 8GB 
hydrogenNo Maintenance RequiredLinode 8GB 
neonNo Maintenance RequiredLinode 8GB 
beryllium2019-10-21 09:00 AMLinode 4GB(pending upgrade)
boron2019-10-22 05:00 AMLinode 4GB(pending upgrade)

Original Submission

posted by martyb on Monday October 14, @03:00AM   Printer-friendly

I am inviting the editorial team to take a much-deserved "break" this weekend by using "weekend story spacing"[*] on Monday. This is a long holiday weekend in the United States in celebration of Columbus Day (or Indigenous Peoples' Day). As a result, sites tend to post fewer stories. And, of the stories that are posted, a larger fraction are "filler" stories or fluff pieces, if you will. This, in turn, makes it harder for the editorial staff to find stories of interest to post to SoylentNews.

One of our editors is still on leave and the remaining staff has been stretched thin with his absence. Further, several of the editorial staff are facing real-life challenges that conspire to reduce the amount of time and energy that can be given to posting stories on SoylentNews. Do recall that all staff here are volunteers and what you see here is freely given of their own spare time.

We generally try to post 14-15 stories per day on weekdays, and about 10 stories per day on weekends.

Also, a reminder that Linode has informed us of some server maintenance they will need to perform. Except for a short while on IRC (Internet Relay Chat), any downtime should not be visible to the community. Linode reserves up to a two-hour window for their maintenance, but past experience has show that most prior maintenance is completed in less than 30 minutes and often as little as 10-15 minutes. See our earlier story Linode to Perform Maintenance; Several SoylentNews Servers Selected for Servicing for details. The first of our servers to be affected is sodium whose maintenance window starts: 2019-10-18 05:00 AM.

We will keep you informed as things progress.

Original Submission

posted by martyb on Monday September 30, @05:30PM   Printer-friendly
from the WallOfText++ dept.

What started it all:

On 2019-08-24 13:02:01 UTC an accusation ( was made that a Journal Entry "It would have been posted before 6 hours ago" (i.e. posted at approximately 2019-08-24 07:00:00 UTC) was deleted by a member of the staff at SoylentNews. The circumstances surrounding the making of the Journal Entry are elaborated upon in this comment. (

I have been with this site since before it went live. Its founding principal has been the making available of a forum whereby the community can submit stories — and post comments — to predominantly tech-related items. Further, each logged-in user has been made available the ability to post entries to their Journal.

As Editor-in-Chief I took this allegation seriously and performed an independent and in-depth investigation. My findings are presented below.

Note: It is not lost on me the futility of trying to prove a negative. It is for good reason that the criminal justice system in the US is founded on the principle of "innocent until proven guilty." It is not up the the accused to vindicate themselves, but for the accuser to bring sufficient evidence to bring about conviction.

NB: In the course of writing this, I discovered a bug in how the site displays wide elements contained in an ECODE element. It incorrectly wraps the text onto the next line (leading to a jumbled mess) when it should, instead, provide horizontal scroll bars. Please accept my apologies for its current appearance.

Executive Summary:

An in-depth investigation making use of: external resources, the UI presented by SoylentNews, and ad-hoc queries of the site database (DB) failed to locate a "smoking gun", i.e. found no clear proof that a Journal Entry was posted to the site and subsequently deleted by anyone other than an author.

It is my estimation that the user submitted an entry, but the site failed to receive and save it correctly. In other words, the user tripped over some kind of bug be it in the site's code, communications between the user and the site, or something else.

Recommendation: When a user completes making a Journal Entry and submits it to the site, the code should respond by using the newly-created journal parameters in conjunction with the normal journal-loading code to present the Journal Entry to the user as confirmation that the entry was properly received and saved. That is to say, affirmative feedback of receipt, storage, and accessibility of the Journal Entry.


For those who may not be aware, all staff on SoylentNews are volunteers; nobody has ever been paid anything for their work on this site. Everything has been done in people's spare time. Early morning before work. During work breaks. Nights, weekends, holidays, and vacations too.

I hereby express my gratitude to all who have supported my work on this investigation. I have invested at least two person-weeks of effort digging into this. This necessitated my letting up on processing story submissions and making them available on the site. Please join me in thanking janrinok, fnord666, takyon, and chromas who stepped up and carried much of the editing load while I have been busy on this investigation. Thanks are in order, too, for @SemperOSS; lending a most-welcome helping hand in finding locations of — and incantations for extracting pertinent data from — important information about how the site is configured and operates.


Early Site History:

For those who may not have been here at the beginning, SoylentNews arose as a result of frustration with how Slashdot failed to listen to what they called "their audience" in response to their rollout of a Beta version of their site. (And also lead to the infamous "Buck Feta!" war cry.) This, in turn, lead to the infamous "Slashcott" — a week-long period where users voluntarily abstained from visiting Slashdot. (Slashdot boycott). This Slashcott was then extended for an additional week.

While this was happening, some like-minded folk got together and found an open-sourced release of slashcode. This code was a few years out of date and depended on old versions of other applications. Among other things, this included the database server (MySQL), the web server (Apache), web cache (memcached), and Perl in which the site code had been written. In a frantic two-week-long period, these intrepid coders took this ancient software and hacked it into "running condition" and SoylentNews went public on February 16th, 2014.

The first few weeks of SoylentNews were filled with site crashes and reports of display artifacts and problems with the general operation of the site. In short, many bugs resulting from the feverish pace of development revealed themselves and, over time, were found and quashed. It is a testament to their hard work and dedication that problems with the site are, today, viewed as an aberration instead of a regular occurrence. I do not know whether or not the original Slashcode ever underwent a rigorous review. But, given the amount of hacking involved in just getting SoylentNews up-and-running, any stability of the current site is mostly a result of extremely dedicated Whack-A-Mole debugging.


  • Site Crash on 2019-08-15
  • Malicious URL?
  • Other Sources
  • Use the Source, Luke!
  • The Journals-Related Tables
  • The Discussion-Related Tables
  • Or is it?
  • The Comment-Related Tables
  • Trying Another Perspective
  • The Message-Related Tables
  • Conclusion
  • Recommendation

Site Crash on 2019-08-15:

We had a site crash on August 15, 2019. We do not know why. Given the site had already been down for several hours, it was deemed necessary to restore a backup to the drive on which the site database resides. No root cause analysis is practicable at this time. Over the next couple days, we found some artifacts from that unplanned crash and site restart. For example, we discovered that slashd — the daemon which launches periodic processes in support of the site (such as emailing notifications about comment moderations and replies) had not been restarted. Also, it is possible that some race condition arose that interfered with the site properly saving a Journal Entry.

Malicious URL?:

It was suggested that someone could construct a malicious URL to delete another user's journal. The author of a Journal Entry has a button they can click so as to delete one of their own Journal Entries. It was thought that, perhaps, a modification of the resultant URL might suffice to delete a Journal Entry for someone else. I tried this on our development server where I have "god mode" privileges; absolutely nobody on the production SoylentNews site has this privilege level. Even with these elevated privileges, my attempt failed: I saw no error message, but neither was the Journal Entry deleted.

Finding: Even the most-priviledged user cannot use a malicious URL to delete a Journal Entry.

Other Sources:

As part of this investigation, I looked for data that may have been captured externally. I posed queries to Google for the Journal Entry author's nickname and the date of the post, and found nothing. Another place I looked was the Wayback Machine on the Internet Archive. I found these archives for that date:

Notice how the link contains the date and time of when the site was scanned: "YYYY MM DD hh mm ss". The first two links bracket the date/time in question. I could find no indication of a Journal Entry there, either.

Finding: Failed to find an external source which showed that the Journal Entry had been posted to the site.

Use the Source, Luke!

Well, the source of data for the entire site: the database, that is. Having found no external corroboration, there was nothing left but to dive into the database.

It is worth mentioning at this point that the source code for SoylentNews has been made available on GitHub under Note that the original slashcode on which soylentnews was built was saved under and had a last commit in September of 2009. Observant readers will note that leaves a nearly 4.5 year gap from last commit and the use of it as a base for SoylentNews which launched on 2014-02-16 . Many dependent packages had undergone major updates and bug fixes since that time. This required significant modification in the received code just to be able to run at all.

The code which runs the current SoylentNews site can be found under

The Journals-Related Tables:

This was the obvious starting point. There are 4 potential tables of interest:

  • journal_themes
  • journal_transfer
  • journals
  • journals_text

Nothing much of interest in journal_themes:

mysql> DESCRIBE journal_themes ;
| Field | Type                | Null | Key | Default | Extra          |
| id    | tinyint(3) unsigned | NO   | PRI | NULL    | auto_increment |
| name  | varchar(30)         | NO   | UNI | NULL    |                |
2 rows in set (0.00 sec)

mysql> SELECT * FROM journal_themes ORDER BY id ;
| id | name     |
|  1 | generic  |
|  2 | greypage |
|  3 | bluebox  |
3 rows in set (0.00 sec)

There is nothing at all in the journal_transfer table:

mysql> SELECT count(*) FROM journal_transfer ;
| count(*) |
|        0 |
1 row in set (0.00 sec)


The journals table provides an "abstraction" of a Journal Entry. The actual text of a Journal Entry is stored in the journals_text table:

mysql> DESCRIBE journals_text;
| Field     | Type                  | Null | Key | Default | Extra |
| id        | mediumint(8) unsigned | NO   | PRI | NULL    |       |
| article   | mediumtext            | NO   |     | NULL    |       |
| introtext | mediumtext            | NO   |     | NULL    |       |
3 rows in set (0.00 sec)


We will come back to this table shortly. For now, suffice it to say that the id field provides an index into both the journals and journals_text tables, the introtext field contains the HTML for the Journal Entry, and the article field contains a plain-text rendition of the introtext field.

Now, let's look closer at the journals table:

mysql> DESCRIBE journals ;
| Field       | Type                               | Null | Key | Default           | Extra                       |
| id          | mediumint(8) unsigned              | NO   | PRI | NULL              | auto_increment              |
| uid         | mediumint(8) unsigned              | NO   | MUL | NULL              |                             |
| date        | datetime                           | NO   |     | NULL              |                             |
| description | varchar(80)                        | NO   |     | NULL              |                             |
| posttype    | tinyint(4)                         | NO   |     | 2                 |                             |
| discussion  | mediumint(8) unsigned              | YES  |     | NULL              |                             |
| tid         | smallint(5) unsigned               | NO   | MUL | NULL              |                             |
| promotetype | enum('publicize','publish','post') | NO   |     | publish           |                             |
| last_update | timestamp                          | NO   |     | CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |
| srcid_32    | bigint(20) unsigned                | NO   | MUL | 0                 |                             |
| srcid_24    | bigint(20) unsigned                | NO   | MUL | 0                 |                             |
11 rows in set (0.01 sec)


Let's first look for entries made within a day on either side of 2019-08-24:

mysql> SELECT id, date, uid, description, discussion FROM journals WHERE "2019-08-23" < date AND date < "2019-08-26" ORDER BY id ;
| id   | date                | uid  | description                                                 | discussion |
| 4512 | 2019-08-23 15:02:42 | 4388 | You Are Banned.                                             |      33255 |
| 4513 | 2019-08-23 16:25:06 | 6150 | "Our great American companies are hereby ordered..."        |      33256 |
| 4514 | 2019-08-24 04:38:24 | 4543 | Perdurabo.                                                  |      33265 |
| 4515 | 2019-08-24 08:17:01 | 4543 | By request, The HU.                                         |      33266 |
| 4517 | 2019-08-24 20:09:11 | 5086 | No, Virginia, "conservatism" is not the new counter-culture |      33274 |
| 4518 | 2019-08-25 13:35:30 | 2926 | The 'Soft' Racism of Gun Control                            |      33285 |
| 4520 | 2019-08-25 16:38:48 | 5291 | Valediction: On corruption and moderation                   |      33287 |
| 4521 | 2019-08-25 22:04:51 | 3682 | Plasma Tsunamis Responsible for Sunspots                    |      33289 |
8 rows in set (0.02 sec)


Notice that (id==4520) was posted by this user (uid==5291), but... it is not the droid Journal Entry we were looking for. Looking closer, notice how we are missing (id==4516) and (id=4519). Apparently, deleting a Journal Entry actually deletes it from the database, instead of leaving it in the DB but flagging it as being deleted (i.e. unavailable).

Historical Perspective: Slashdot was registered on 1996-11-17. Disk space usage was a much greater concern back then. For historical perspective consider that Windows 95 had just been released and that the Pentium MMX and Pentium Pro were introduced in about the same time period. As for disk storage, this timeline of hard disk drives notes that in 1996 Seagate shipped its first 10,000 RPM "Cheetah" disk drive and 1997 saw the release of the "IBM Deskstar 16GP 'Titan' - 16,800 megabytes, five 3.5-inch disks". Yes, 16,800 MB or 16.8 GB. Flash drives of that capacity can be purchased for under $10 today.

So, getting back to the missing entries, we can ignore (id==4519) as that was clearly well past the time of interest. That leaves (id==4516)... where did it go?

Let's see what we can learn from looking at the journals_text table.

mysql> SELECT id, substr(introtext, 1, 80) FROM journals_text WHERE 4512 <= id AND id <= 4521 ORDER BY id ;
| id   | substr(introtext, 1, 80)                                                         |
| 4512 | <p>This morning, I took a quick look at the website of the Church of the Flying  |
| 4513 | <p>You probably saw who said that, so I don't have to mention his name.</p><p>Bu |
| 4514 | <tt>I've mentioned before that I perceive hip-hop as more vibrant, more alive, t |
| 4515 | <tt><br><br>Please, anyone, contribut |
| 4517 | Recently one of our frothier, crazier members, who is in my opinion a fascinatin |
| 4518 | <p>The historically racist origins of gun control are hardly a topic for debate. |
| 4520 | <p>This will be my final interaction with this site, which I have supported for  |
| 4521 | <p><b>Rejected submission by RandomFactor at 2019-07-25 21:17:28 from the tSUNam |
8 rows in set (0.01 sec)

Hmmm, same missing entries as before; no joy here.

Attentive readers will have noticed the discussion column in the journals table. Further, for (id==4515) we have (discussion==33266) and for (id==4517) we have (discussion==33274). We will come back to those presently.

Finding: We have failed to find the sought-for Journal Entry, but have found two tuples of id and discussion that warrant further investigation.

The Discussion-Related Tables:

Stepping back for a moment, since comments can be posted to both Journal Entries and stories, it makes sense they may share some commonality. There are two tables related to discussions in our database: discussion_kinds and discussions.

mysql> DESCRIBE discussion_kinds ;
| Field | Type                | Null | Key | Default | Extra          |
| dkid  | tinyint(3) unsigned | NO   | PRI | NULL    | auto_increment |
| name  | varchar(30)         | NO   | UNI |         |                |
2 rows in set (0.00 sec)

mysql> SELECT * FROM discussion_kinds ORDER BY dkid ;
| dkid | name          |
|    1 | story         |
|    2 | user_created  |
|    3 | journal       |
|    4 | journal-story |
|    5 | poll          |
|    6 | submission    |
|    7 | feed          |
|    8 | project       |
8 rows in set (0.00 sec)

Now we are getting somewhere! There are entries for story and for journal, just as we had hoped!

The other table of interest was discussions. Let's see what we can find there.

mysql> DESCRIBE discussions ;
| Field         | Type                                                                                           | Null | Key | Default             | Extra                       |
| id            | mediumint(8) unsigned                                                                          | NO   | PRI | NULL                | auto_increment              |
| dkid          | tinyint(3) unsigned                                                                            | NO   |     | 1                   |                             |
| stoid         | mediumint(8) unsigned                                                                          | NO   | MUL | 0                   |                             |
| sid           | char(16)                                                                                       | NO   | MUL |                     |                             |
| title         | varchar(128)                                                                                   | NO   |     | NULL                |                             |
| url           | varchar(255)                                                                                   | NO   |     | NULL                |                             |
| topic         | int(10) unsigned                                                                               | NO   | MUL | NULL                |                             |
| ts            | datetime                                                                                       | NO   |     | 1970-01-01 00:00:00 |                             |
| type          | enum('open','recycle','archived')                                                              | NO   | MUL | open                |                             |
| uid           | mediumint(8) unsigned                                                                          | NO   |     | NULL                |                             |
| commentcount  | smallint(5) unsigned                                                                           | NO   |     | 0                   |                             |
| flags         | enum('ok','delete','dirty')                                                                    | NO   |     | ok                  |                             |
| primaryskid   | smallint(5) unsigned                                                                           | YES  | MUL | NULL                |                             |
| last_update   | timestamp                                                                                      | NO   |     | CURRENT_TIMESTAMP   | on update CURRENT_TIMESTAMP |
| approved      | tinyint(3) unsigned                                                                            | NO   |     | 0                   |                             |
| commentstatus | enum('disabled','enabled','friends_only','friends_fof_only','no_foe','no_foe_eof','logged_in') | NO   |     | enabled             |                             |
| archivable    | enum('no','yes')                                                                               | NO   |     | yes                 |                             |
| legacy        | enum('no','yes')                                                                               | NO   |     | yes                 |                             |
18 rows in set (0.01 sec)

So far, so good. Let's take a look inside.

mysql> SELECT id, dkid, stoid, sid, substr(title,1,20), url, ts, uid, last_update FROM discussions WHERE "2019-08-23 00:00:01" < ts AND ts < "2019-08-24 23:59:59" ORDER BY id ;
| id    | dkid | stoid  | sid              | substr(title,1,20)   | url                                               | ts                  | uid  | last_update         |
| 33258 |    1 | 370766 | 19/08/23/2359248 | Billionaire David Ko | // | 2019-08-24 00:28:00 |   76 | 2019-08-29 01:41:03 |
| 33259 |    1 | 370767 | 19/08/24/0012203 | Study Corroborates t | // | 2019-08-24 02:49:00 |   76 | 2019-08-27 04:52:14 |
| 33260 |    1 | 370768 | 19/08/24/0021202 | Alleged "Snake Oil"  | // | 2019-08-24 05:12:00 |   76 | 2019-08-25 06:48:09 |
| 33261 |    1 | 370769 | 19/08/24/0027221 | Federal Agency Turf  | // | 2019-08-24 07:32:00 |   76 | 2019-08-25 23:40:34 |
| 33262 |    1 | 370770 | 19/08/24/0038241 | Phone Companies and  | // | 2019-08-24 09:54:00 |   76 | 2019-08-27 02:50:36 |
| 33263 |    1 | 370771 | 19/08/24/0133226 | EU Officials Draft P | // | 2019-08-24 12:19:00 |   76 | 2019-08-25 11:48:20 |
| 33264 |    1 | 370772 | 19/08/24/0135254 | YouTube Banning Comb | // | 2019-08-24 14:37:00 |   76 | 2019-08-25 04:10:23 |
| 33265 |    3 |      0 |                  | Perdurabo.           | //              | 2019-08-24 04:38:24 | 4543 | 2019-08-27 08:28:48 |
| 33266 |    3 |      0 |                  | By request, The HU.  | //              | 2019-08-24 08:17:01 | 4543 | 2019-08-27 15:02:04 |
| 33267 |    1 | 370773 | 19/08/24/128219  | NASA Astronaut's Div | //  | 2019-08-24 16:55:00 |   76 | 2019-08-29 18:13:08 |
| 33268 |    1 | 370774 | 19/08/24/1216253 | Neil Young on Sound  | // | 2019-08-24 19:17:00 |   76 | 2019-08-26 18:42:21 |
| 33269 |    3 |      0 |                  | Test                 | // | 2019-08-24 12:55:02 | 5291 | 2019-08-24 17:17:12 |
| 33270 |    1 | 370775 | 19/08/24/1341250 | Deadly Superbug Outb | // | 2019-08-24 21:34:00 |   52 | 2019-08-26 04:19:22 |
| 33271 |    1 | 370776 | 19/08/24/1347258 | Capsule Containing R | // | 2019-08-24 23:53:00 |   52 | 2019-08-26 18:27:55 |
| 33274 |    3 |      0 |                  | No, Virginia, "conse | //      | 2019-08-24 20:09:11 | 5086 | 2019-09-09 06:29:43 |
15 rows in set (0.62 sec)

And how about that! There — at (id==33269) — is our missing Journal Entry number 4516.

Or is it?

Let's make no assumptions. Let's try to load it in a browser

Sorry, the requested journal entries were not found.

Now that is some progress. Really! We found something that was posted to a journal, but was deleted. The very one that the user stated they had created after noticing the apparent disappearance of his journal article.

So, umm, where is the other Journal Entry? You know, the missing one? Yeah, we're still working on that one, but at least our theory of what happens in the site code for an intentional deletion of a Journal Entry is validated.

Looking back, remember the discussion column from the journals table and those we were going to come back to. Now is the time; here is what we had:

for (id==4515) we have (discussion==33266) and for (id==4517) we have (discussion==33274)

Here, in the discussions table, we see that discussion 33266 corresponds to and discussion 33274 corresponds to so we have figured out where they went. Unfortunately, this does not provide any closure on the situation, but it does close off one potential avenue of explanation.

Finding: We have found evidence that when a user deletes a Journal Entry, any comments that had been posted to it persist in the database. None have yet been found.

The Comment-Related Tables:

As was the case for the journals table, SoylentNews' comments are split up as well. Here are all of the comment-related tables:

  • comment_log
  • comment_promote_log
  • comment_text
  • commentmodes
  • comments

I do not know why they still exist, but both comment_log and comment_promote_log are empty. As for commentmodes, that is the source for the pull-down list of how one chooses to see comments displayed:

mysql> SELECT * FROM commentmodes ORDER BY mode ;
| mode      | name         | description |
| flat      | Flat         |             |
| nocomment | No Comments  |             |
| threadtng | Threaded-TNG | NULL        |
| threadtos | Threaded-TOS | NULL        |
4 rows in set (0.00 sec)

So now we are down to the comments and comment_text tables. Last things first, the comment_text table is exactly what it says on the tin, where the text of each comment is stored:

mysql> DESCRIBE comment_text ;
| Field   | Type             | Null | Key | Default | Extra |
| cid     | int(10) unsigned | NO   | PRI | NULL    |       |
| comment | mediumtext       | NO   |     | NULL    |       |
2 rows in set (0.01 sec)

The key thing here is the cid (Comment ID) field which is a pointer (key) into the comments table. Having gotten all that out of the way, what can the actual comments table tell us?

mysql> DESCRIBE comments ;
| Field            | Type                  | Null | Key | Default             | Extra                       |
| sid              | mediumint(8) unsigned | NO   | MUL | NULL                |                             |
| cid              | int(10) unsigned      | NO   | PRI | NULL                | auto_increment              |
| pid              | int(10) unsigned      | NO   | MUL | 0                   |                             |
| opid             | int(10) unsigned      | NO   | MUL | 0                   |                             |
| children         | int(10) unsigned      | NO   |     | 0                   |                             |
| date             | datetime              | NO   | MUL | 1970-01-01 00:00:00 |                             |
| last_update      | timestamp             | NO   |     | CURRENT_TIMESTAMP   | on update CURRENT_TIMESTAMP |
| ipid             | char(32)              | NO   | MUL |                     |                             |
| subnetid         | char(32)              | NO   | MUL |                     |                             |
| subject          | varchar(50)           | NO   |     | NULL                |                             |
| subject_orig     | enum('no','yes')      | NO   |     | yes                 |                             |
| uid              | mediumint(8) unsigned | NO   | MUL | NULL                |                             |
| points           | tinyint(4)            | NO   |     | 0                   |                             |
| pointsorig       | tinyint(4)            | NO   |     | 0                   |                             |
| pointsmax        | tinyint(4)            | NO   |     | 0                   |                             |
| lastmod          | mediumint(8) unsigned | NO   |     | 0                   |                             |
| reason           | tinyint(3) unsigned   | NO   |     | 0                   |                             |
| signature        | char(32)              | NO   |     |                     |                             |
| karma_bonus      | enum('yes','no')      | NO   |     | no                  |                             |
| subscriber_bonus | enum('no','yes')      | NO   |     | no                  |                             |
| len              | smallint(5) unsigned  | NO   |     | 0                   |                             |
| karma            | smallint(6)           | NO   |     | 0                   |                             |
| karma_abs        | smallint(5) unsigned  | NO   |     | 0                   |                             |
| tweak_orig       | tinyint(4)            | NO   |     | 0                   |                             |
| tweak            | tinyint(4)            | NO   |     | 0                   |                             |
| badge_id         | tinyint(4)            | NO   |     | 0                   |                             |
26 rows in set (0.00 sec)

Zoinks! That's a lot, but we can ignore much of it for our purposes. We just want to know if any comments were made to the missing Journal Entry. Let's look at a subset:

  mysql> SELECT cid, sid, pid, opid, uid, children, date, last_update, subject FROM comments WHERE pid = 0 AND "2019-08-24 06:00:00" <= date AND date <= "2019-08-24 08:00:00" ORDER BY cid;
| cid    | sid   | pid    | opid   | uid  | children | date                | last_update         | subject                                          |
| 884595 | 33250 | 884485 | 884479 |    1 |        0 | 2019-08-24 06:00:50 | 2019-09-23 07:02:01 | Re:so?                                           |
| 884596 | 33244 | 884418 | 883941 |    1 |       19 | 2019-08-24 06:01:46 | 2019-09-23 07:02:01 | Re:Sign of the times                             |
| 884597 | 33265 |      0 |      0 |   18 |       15 | 2019-08-24 06:03:10 | 2019-09-23 07:02:01 | Wait, wut?                                       |
| 884598 | 33259 | 884585 | 884531 |    1 |        1 | 2019-08-24 06:08:34 | 2019-09-23 07:02:01 | Re:Wow! Then radiation hits fetus...             |
| 884599 | 33244 | 884520 | 883955 |    1 |        2 | 2019-08-24 06:10:36 | 2019-09-23 07:02:01 | Re:It's exaeta again, isn't it?                  |
| 884600 | 33244 | 884596 | 883941 |   18 |       18 | 2019-08-24 06:11:13 | 2019-09-23 07:02:01 | Re:Sign of the times                             |
| 884601 | 33252 | 884400 | 884336 | 2828 |        0 | 2019-08-24 06:11:29 | 2019-09-23 07:02:01 | Re:What's the appeal of mercedes?                |
| 884602 | 33265 | 884581 | 884581 | 2489 |        2 | 2019-08-24 06:18:09 | 2019-09-23 07:02:01 | Re:Is a rapper a singer?                         |
| 884603 | 33259 | 884598 | 884531 |    1 |        0 | 2019-08-24 06:27:33 | 2019-09-23 07:02:01 | Re:Wow! Then radiation hits fetus...             |
| 884604 | 33259 | 884530 | 884530 | 6836 |        6 | 2019-08-24 06:30:48 | 2019-09-23 07:02:01 | Re:If so, then...                                |
| 884605 | 33265 | 884597 | 884597 | 4543 |        9 | 2019-08-24 06:31:30 | 2019-09-23 07:02:01 | Re:Wait, wut?                                    |
| 884606 | 33265 | 884597 | 884597 | 2489 |        4 | 2019-08-24 06:41:07 | 2019-09-23 07:02:01 | Re:Wait, wut?                                    |
| 884607 | 33265 | 884605 | 884597 | 2489 |        8 | 2019-08-24 06:44:10 | 2019-09-23 07:02:01 | Re:Wait, wut?                                    |
| 884608 | 33244 | 884457 | 883942 |   52 |        0 | 2019-08-24 06:47:11 | 2019-09-23 07:02:01 | Re:WTF?                                          |
| 884609 | 33259 | 884604 | 884530 |  881 |        5 | 2019-08-24 06:48:32 | 2019-09-23 07:02:01 | Re:If so, then...                                |
| 884610 | 33265 | 884607 | 884597 | 4543 |        7 | 2019-08-24 06:55:40 | 2019-09-23 07:02:01 | Re:Wait, wut?                                    |
| 884611 | 33244 | 884461 | 883966 |   52 |        1 | 2019-08-24 06:58:54 | 2019-09-23 07:02:01 | Re:Why do we moderate? - Because it is important |
| 884612 | 33259 | 884585 | 884531 |    1 |        0 | 2019-08-24 07:00:14 | 2019-09-23 07:02:01 | Re:Wow! Then radiation hits fetus...             |
| 884613 | 33255 |      0 |      0 | 3272 |        1 | 2019-08-24 07:07:03 | 2019-09-24 07:02:02 | Works on reload.                                 |
| 884614 | 33244 | 884435 | 883966 |   52 |        3 | 2019-08-24 07:08:08 | 2019-09-24 07:02:02 | Re:Why do we moderate? - Because it is important |
| 884615 | 33222 | 883944 | 883944 |    1 |        0 | 2019-08-24 07:39:07 | 2019-09-24 07:02:02 | Re:Come out, exaeta, come out!                   |
| 884616 | 33244 | 884337 | 883966 |   52 |        2 | 2019-08-24 07:39:07 | 2019-09-24 07:02:02 | Re:Why do we moderate? - Because it is important |
| 884617 | 33261 |      0 |      0 | 2452 |       16 | 2019-08-24 07:40:06 | 2019-09-24 07:02:02 | 5G is stupid                                     |
| 884618 | 33265 | 884602 | 884581 | 4543 |        1 | 2019-08-24 07:42:09 | 2019-09-24 07:02:02 | Re:Is a rapper a singer?                         |
| 884619 | 33222 | 884211 | 883718 | 2645 |        0 | 2019-08-24 07:49:42 | 2019-09-24 07:02:02 | Re:Need specifics, actually.                     |
| 884620 | 33244 | 884534 | 883966 |   52 |       13 | 2019-08-24 07:54:11 | 2019-09-24 07:02:02 | Re:Why do we moderate? - Because it is important |
26 rows in set (0.00 sec)


The pid field is the comment's "Parent ID", i.e. the comment ID of the comment that is the parent of this comment. Let's look at only those comments which have NO parent id:

mysql> SELECT cid, sid, pid, opid, uid, children, date, last_update, subject FROM comments WHERE pid = 0 AND "2019-08-24 06:00:00" <= date AND date <= "2019-08-24 08:00:00" ORDER BY cid;
| cid    | sid   | pid | opid | uid  | children | date                | last_update         | subject          |
| 884597 | 33265 |   0 |    0 |   18 |       15 | 2019-08-24 06:03:10 | 2019-09-23 07:02:01 | Wait, wut?       |
| 884613 | 33255 |   0 |    0 | 3272 |        1 | 2019-08-24 07:07:03 | 2019-09-24 07:02:02 | Works on reload. |
| 884617 | 33261 |   0 |    0 | 2452 |       16 | 2019-08-24 07:40:06 | 2019-09-24 07:02:02 | 5G is stupid     |
3 rows in set (0.78 sec)


All of these comments pertain to the story they are attached to:

Finding: we found no evidence of a comment having been made to a Journal Entry, during the time in question.

Trying Another Perspective:

In this comment, it was suggested:

All your fans would have received a message about your journal entry instantly. I for one didn't. You can't invisibly delete a journal entry without:
a) resetting the journal auto-incrementing counter in the database (and there's no programattic way to do that, you can only move it forwards)
but also:
b) invisibly deleting *all* the messages that were sent out because of it
and therefore:
c) resetting the message auto-incrementing counter in the database (ditto)
and here's the killer:
d) on a live system
and the cherry on top:
e) with no logs

If we had someone with that level of technical competence on the administrative team, that would probably double the total amount of competence have - the above goes well beyond mere guru level.

Ockham's razor points to either human error or a bug.

We've previously shown that articles can be deleted; there is a button for it on the journal writer's edit page.

If the deleted Journal Entry was the one with the largest ID, the system just re-uses that slot.

But, the idea that a message might have been sent out announcing to a user that a friend of theirs posted a Journal Entry? that's interesting. Let's see where that leads us.

Finding: We have a new situation to investigate, but no results, yet.

The Message-Related Tables:

Here are all the message-related tables on the site:

  • message_codes
  • message_deliverymodes
  • message_drop
  • message_log
  • message_web
  • message_web_text

At this point, selections made by particular users will be discovered. In the interests of excluding 3rd-parties, please forgive me if some of this discussion is less detailed than previously.

Let's take these in order:

mysql> DESCRIBE message_codes   ;
| Field           | Type                             | Null | Key | Default | Extra |
| code            | int(11)                          | NO   | PRI | NULL    |       |
| type            | varchar(32)                      | NO   |     | NULL    |       |
| seclev          | int(11)                          | NO   |     | 1       |       |
| modes           | varchar(32)                      | NO   |     |         |       |
| send            | enum('now','defer','collective') | NO   |     | now     |       |
| subscribe       | tinyint(1)                       | NO   |     | 0       |       |
| acl             | varchar(32)                      | NO   |     |         |       |
| delivery_bvalue | int(10) unsigned                 | NO   |     | 0       |       |
8 rows in set (0.00 sec)

mysql> SELECT * FROM message_codes WHERE seclev = 1 ORDER BY code ;
| code | type                       | seclev  | modes | send       | subscribe | acl   | delivery_bvalue |
|   -2 | Registration Mail          |       1 | 0     | now        |         0 |       |               0 |
|   -1 | Unknown Message            |       1 | 0     | now        |         0 |       |               0 |
|    0 | Daily Newsletter           |       1 | 0     | now        |         0 |       |               1 |
|    1 | Daily Headlines            |       1 | 0     | now        |         0 |       |               1 |
|    2 | Metamoderation Results     |       1 |       | collective |         0 |       |               3 |
|    3 | Comment Moderation         |       1 |       | collective |         0 |       |               3 |
|    4 | Comment Reply              |       1 |       | now        |         0 |       |               3 |
|    5 | Journal Entry by Friend    |       1 |       | now        |         0 |       |               3 |
|    6 | New Submission             |     100 | 0     | now        |         0 |       |               1 |
|    7 | Journal Reply              |       1 |       | now        |         0 |       |               3 |
|    8 | New Comment                | 1000000 |       | now        |         0 |       |               0 |
|   10 | Daily Site Stats           |     100 | 0     | now        |         0 | stats |               1 |
|   11 | Email Story                |       1 | 0     | now        |         0 |       |               1 |
|   12 | Relationship Change        |       1 |       | collective |         0 |       |               3 |
|   13 | Bad login attempt warnings |       1 | 1     | now        |         0 |       |               2 |
|   14 | Daily Moderation Stats     |     100 | 0     | now        |         0 | stats |               1 |
|   15 | Subscription Running Low   |       1 |       | now        |         1 |       |               1 |
|   16 | Subscription Expired       |       1 |       | now        |         1 |       |               1 |
|   18 | Invalid HTML Input         |     100 |       | now        |         0 |       |               3 |
|   19 | Declined Submission Reason |       1 |       | now        |         0 |       |               3 |
|   20 | Admin to user message      |       1 |       | now        |         0 |       |               3 |
|   22 | Achievement                |       1 |       | now        |         0 |       |               3 |
|   25 | Remarks                    |     100 |       | now        |         0 |       |               2 |
23 rows in set (0.00 sec)


Now, this looks promising; (code==5) is "Journal Entry by Friend". Let's hold on to that and see what else we can find.

mysql> DESCRIBE message_deliverymodes ;
| Field    | Type                  | Null | Key | Default | Extra |
| code     | smallint(6)           | NO   | PRI | 0       |       |
| name     | varchar(32)           | NO   |     |         |       |
| bitvalue | mediumint(8) unsigned | NO   |     | 0       |       |
3 rows in set (0.00 sec)

mysql> SELECT * FROM message_deliverymodes ORDER BY code ;
| code | name        | bitvalue |
|   -1 | No Messages |        0 |
|    0 | E-mail      |        1 |
|    1 | Web         |        2 |
3 rows in set (0.00 sec)


A registered user (someone who has a nickname and UID on the system) can specify several things for which they wish to receive notifications, and how they want to receive them. This table is where those are defined. On to the next table.

mysql> DESCRIBE message_drop ;
| Field   | Type                             | Null | Key | Default           | Extra                       |
| id      | int(11)                          | NO   | PRI | NULL              | auto_increment              |
| user    | mediumint(8) unsigned            | NO   |     | NULL              |                             |
| fuser   | mediumint(8) unsigned            | NO   |     | NULL              |                             |
| code    | int(11)                          | NO   |     | -1                |                             |
| date    | timestamp                        | NO   |     | CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |
| altto   | varchar(50)                      | NO   |     |                   |                             |
| send    | enum('now','defer','collective') | NO   |     | now               |                             |
| message | mediumblob                       | NO   |     | NULL              |                             |
8 rows in set (0.00 sec)

mysql> SELECT count(*) FROM message_drop ;
| count(*) |
|      115 |
1 row in set (0.00 sec)


This table contains a listing of the messages that have been deleted today. Logged-in users can go to their messages inbox, and if they have any pending, web-based messages, will see this message at the top of the screen:

These messages will be kept in the system for only 14 days, whether they have been read or not. After 14 days, they will be deleted.

Messages marked with "*" are unread.

Unfortunately, I did not learn of this table until several days too late.

There's nothing to do at this point but to continue on. Besides, the next table is much more interesting.

mysql> DESCRIBE message_log ;
| Field | Type                  | Null | Key | Default           | Extra                       |
| id    | int(11)               | NO   |     | NULL              |                             |
| user  | mediumint(8) unsigned | NO   |     | NULL              |                             |
| fuser | mediumint(8) unsigned | NO   |     | NULL              |                             |
| code  | int(11)               | NO   |     | -1                |                             |
| mode  | int(11)               | NO   |     | NULL              |                             |
| date  | timestamp             | NO   |     | CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |
6 rows in set (0.00 sec)


One thing we can find out here is how often messages about friend's Journal Entries are posted to the site.

mysql> SELECT count(1) AS count, substr(date,15,2) AS mins FROM message_log WHERE code = 5 GROUP BY mins ORDER BY mins;
| count | mins |
|    39 | 05   |
|   148 | 10   |
|    77 | 15   |
|    12 | 20   |
|    22 | 25   |
|    60 | 30   |
|    46 | 35   |
|    13 | 40   |
|    25 | 45   |
|    21 | 50   |
|    62 | 55   |
11 rows in set (0.01 sec)


So, every five minutes, if someone has posted a Journal Entry, the site sends out notifications to all those who had tagged the writer as a friend.

As mentioned earlier, so as to avoid unnecessarily involving innocent third parties, a quick look through the tables containing relationships (friends) and notification requests reveals:

  • there are 14 people who flagged the journal-entry-writer as a friend,
  • of these, 12 have requested notification of when they have posted a Journal Entry
  • all 12 of these have requested notification by 'the web' instead of by 'e-mail'.

A search through the message_log table for any messages sent to any of these people for journal posts by the accuser came up empty.

Finding: No indication has been found that a message was sent to a user to notify them of a friend having posted an entry to their journal.


Multiple sources were investigated to try and find a Journal Entry that was posted to the site and subsequently deleted. External sources were examined and no copy of the alleged Journal Entry were found. A user who has submited a Journal Entry can subsequently delete it, but any comments made to that Journal Entry would persist after such a deletion. No such comments were found. Further, twelve users flagged this user as a friend and requested notification of their having posted a Journal Entry. No such notification was found to have been generated.

A staff member with sufficient privileges and knowledge could, potentially, have deleted a Journal Entry directly through the database. The window of opportunity is at most five minutes, and possibly as short as a few seconds. Such action would have to be performed on a live system and leave no trace of it having been done. This would, also, require the staff memeber to have noticed the posting very shortly after it was saved. Though not impossible, the likelihood of this being the case is vanishingly small.

The user could have written the Journal Entry and, somehow, forgotten to save it. Further discussion with the user revealed their having taken extreme care in updating and saving their Journal Entry.

Given these, and the fact that yet another bug in the code was found while writing this report, combined with the frantic pace of the site's initial development and the many bugs found subsequent to its initial deployment, it appears that the user encountered a bug in the site's handling of posting a new Journal Entry.

Recommendation: It is suggested that the site be updated so that upon successful receipt of a Journal Entry, said entry be immediately displayed to the user by employing the same means that the site uses to display any Journal Entry.

posted by martyb on Tuesday September 03, @06:00PM   Printer-friendly
from the still-looking dept.

An accusation has been made that a Journal Entry made by a member of the community was deleted.

tl;dr No "smoking gun" has been found so far and no conclusion has been made one way or another. The investigation will continue. I have the next two days (Wed. and Thurs.) off from work and intend to dig further into this, then.

When I first learned of this, I immediately started looking around as I found the accusation to be credible and, if true, unacceptable behavior.

Nothing I found that morning could corroborate the claim. I posted my findings and promised that when I got home from work that I would look into it. And I did exactly that spending several hours looking for data through both the UI tools presented within the site and with ad hoc queries to our DB. I dumped sections of tables of interest and brought them local to my home machine for later investigation.

Of course, as life has a way of doing, I was also in the midst of a crunch period at work and my spare time has been scarce of late (e.g. I had a training class to attend that was over 100 miles away and worked the entire Labor Day weekend). Further, we have two editors who are on a leave of absence and another who is on vacation, leaving a greater burden on the remaining editors to keep stories coming to the community. This has been exacerbated by the Labor Day holiday weekend in the US meaning fewer substantial stories get posted to the web at this time. In a nutshell that left fewer editors having to work even harder to find stories that could be brought to the community.

Do remember, also, that we had a rare site outage a couple weeks ago which necessitated restoring the site DB from backups. This was a manual process. There have been a few hiccups since then that were attributed to an unclean shutdown/restart as well as needing to restart various services besides just the database: the web server, load balancer caches (nginx), slashd (a cron-like daemon which periodically starts still other background processes and services), and ... you get the idea.

Still, in what little free time I had, I continued to look for information that could corroborate or refute the accusation.

It bears mentioning that there may well be no "smoking gun". But whether there is or is not, I firmly believe the community nonetheless deserves an investigation. It is a very different thing to toss one's hands up in the air and not even look as opposed to look deeply and diligently and find nothing.

I have decades' experience testing computer software on a wide range of platforms. I have worked at startups and fortune-100 blue chip companies. I once was brought in to a company to test a compiler when my only prior experience was a single college course. Another place I worked at had an automated test harness written in a language in which my only prior experience was it being one of 4 languages covered in a language-survey course in college. Within a week I had started refactoring the code base and by the time I left 9 months later, had removed and/or isolated hardware-based dependencies and rewritten the test harness to run in parallel. A full test run across several different hardware platforms and OS versions, which used to take 8-9 hours, now completed in just 1 hour. These are meant only to illustrate that I have no difficulties digging into things in which I had limited or no prior experience. When SoylentNews implemented Unicode character support using UTF-8 (Unicode Transfer Format - 8bit), I searched for the relevant RFCs (Request For Comment, aka Internet "standards"). One led to another and that to others and when I was done had found about a half-dozen of those. Further, there were different versions of the Unicode standard published by the Unicode Consortium, so I dug into those, as well. Needing test data, I wrote tools to extract/generate all the NCEs (Named Character Entities) as well as both hex- and decimal- numeric character entities. Oh, and then there was the matter of IDNs (Internationalized Domain Names; like "" but "foo" could be in any Unicode supported language!) These then were used in all user-facing, text-entry fields that I could discover and the results were reviewed. I am pleased to say that in the subsequent 5 years' time, I am aware of only two bugs that got through.

(Anecdote time. During the Slashcott, I got wind of some "alternate Slashdot" being developed, but could find no announcement of where it was. I did a bunch of searches. They came up empty. Tried a bunch more. No joy. Then I found what looked to be a bug report for this new site with a screen capture of the main page. And, it had this really strange URL in the address bar. Some domain name like "". Strange! But, I tried it and found it was just what I was looking for. Not knowing who was behind this or how to contact them, I was a bit nervous. Still, after enduring several formkey errors that day and into the next morning, I was finally able to create new account. I then kept a low profile until SoylentNews went live. And that, my friends, is how I got a coveted two-digit UID.)

I present the forgoing not to brag, but to give examples of the diligence and thoroughness that I am bringing to this investigation.

Further, I have somehow become the spokesperson for site-related "stuff" be it an annual summary of where we are at, announcements of server outages due to our hosting provider, Linode, providing updated hardware, different virtual hosting software, bug patches for Meltdown and Spectre, etc. When I have made mistakes I have owned them openly and on the front page no less. I cannot persuade you to believe me, and feel free to believe as you wish, but I can only trust that my integrity in all my actions on this site speak more forcefully and firmly than any words I may bring you in support of it.

My investigation is not complete, but given the amount of time that has passed, I felt it important to give an interim summary of what was planned and any results found so far.

Original Submission

posted by martyb on Friday August 23, @06:17AM   Printer-friendly
from the tragedy-of-the-commons dept.

I was going to post this to a particular story, but thought this might generate more attention and discussion as a general submission.

Seriously, what is going on with all these troll mods? Just because you disagree with someone, thus earning a "disagree" mod, does not mean that person is a "troll." To steal a definition from Urban Dictionary:

An Internet troll, or simply troll in Internet slang, is someone who posts controversial, inflammatory, irrelevant or off-topic messages in an online community, such as an online discussion forum or chat room, with the primary intent of provoking other users into an emotional response or to generally disrupt normal on-topic discussion.

Just because you disagree with someone, does not mean they are trying to do the above. Be faster on the "disagree" and slower on the "troll." Under such abuse, it is hard to have a good discussion and, in itself, is trollish behavior by "generally disrupt[ing] normal on-topic discussion." Other than people disciplining themselves, a concerted effort to police such abuses, or making moderation logs public on the bottom of a comment where the score is shown now, I'm unsure of what to do about. As it stands, it is getting increasingly ridiculous to read what discussion is here on any topic remotely controversial, and is expanding outside of even those. It is starting to drive me away from the site, and I'm somewhat confident it is doing the same for others. I'd be interested to see what others think about the depth of the problem, if they even believe it even exists at all, and what solutions you all have for it.

[Ed note. This story is published exactly as received. First off, it bears repeating that complaining about moderations in the comments often leads (rightly) to an off-topic moderation. That is a contributing factor to my decision to run this story. Secondly, moderation is something that I on occasion have found I've fat-fingered and given a different moderation than expected. Thirdly, in the grand scheme of things, a comment's moderation is — relatively speaking — small potatoes. It is NOT a measure of your IQ or value as a human being. or standing in the community. Just accept that stuff happens and that as likely as not, someone will be along to moderate it the other way. Which is a good opportunity to say: PLEASE USE YOUR MOD POINTS! Lastly, if you think a comment was moderated in error, then send the CID (Comment ID) link e.g. "(#876543)" in an email to admin (at) soylentnews (dot) org. Keep in mind however that we are all volunteers here and there most likely will be a delay between when you send out an email and when we can get around to it. --martyb]

[Updated: 20190823_111312 UTC See comment from JR who far more precisely and eloquently expressed the idea I was attempting to. I concur with his assessment. If I want people to upmod a comment of mine that I believe was unfairly downmodded, then I need to be willing to upmod other's mis-modded comments. For perspective, so far this month, anywhere from ~150-~350 mod points were used in any given day. It bears repeating: use your mod points!]

Original Submission

posted by martyb on Thursday August 22, @12:00PM   Printer-friendly
from the say-that-five-times-fast dept.

I just finished updating the certs for SoylentNews.

We get our certs through Let's Encrypt. Yes, we could automate the whole process, but it has been discussed and decided that given our... unique configuration, it is best to have a human in the loop than to let a script somehow run amok and then try to restore things when who-all-knows-what got deployed and things have gone sideways.

I have checked our web sites for production, dev, and staff as well as sending and retrieving e-mail; all seemed to be okay.

More than anything else, this is a check on us to see if we (well, me, actually) overlooked anything. If you do detect any issues, please post a comment to this story.

(Hat tip to The Mighty Buzzard for standing by in case I bollixed up something.)

[Update: Unless, of course, you cannot post a comment to this story! Then pop onto the #Soylent channel on our Internet Relay Chat (IRC) server and let us know over there. --martyb]

Original Submission

posted by janrinok on Thursday August 15, @02:00PM   Printer-friendly
from the stuff-happens dept.

Earlier this morning (around 0520 UTC) we experienced a sequence of faults which we do not, yet, fully understand. Unfortunately this coincided with several key admins being asleep as they should be.

Several of the team started to try to recover the site but all the obvious efforts failed to have the desired results (e.g. restarting the failed software and switching it all off and back on again). Chromas took the lead in this but he was unable to resolve the problems. SemperOSS showed up next and made a mighty effort but things failed to cooperate. Martyb was next in and checked on status of our servers and a few other things but ultimately could not provide much assistance.

TheMightyBuzzard turned up at his usual time and he and SemperOSS set to work trying to recover the site. After a long slog it became apparent that the existing software had suffered some form of corruption and that a simple recovery was not going to be possible. The decision was finally made to restore to yesterday's snapshot, but this comes with a downside. All the stories, comments, and journal entries since that snapshot was taken (approximately 2019-08-14 22:02:36 UTC) have been lost. This is regrettable but under the circumstances unavoidable.

Some of you joined us on IRC and provided real or moral support to those carrying out the recovery task. Others sent us emails informing us of the outage and we also received offers of help. For this we thank you. We also thank our community at large for your patience while we carried out the work that needed to be done. The investigation into what happened has still to be done and will take some time to do. The site is now up again, but as we are not 100% sure what caused the problem we cannot guarantee that the site will function properly and we will have to wait a day or two to be confident that we have resolved the issue.

We tried to keep everyone updated on our progress on our IRC channel but, if you could not get there, then there was little else we could do to update you on our progress. Displaying a short 'Site Down' message actually exacerbated the problems so we decided that it was better to leave the database alone until the problems had been resolved.

Finally, I would like to say thank you to the guys who did all the work under difficult circumstances. Their efforts are appreciated. We used the usual piece of software on IRC to find out who should bear the blame for this debacle (~blame) — it responded with the only name it has in its choice of staff. Bytram!

posted by martyb on Friday July 05 2019, @03:30PM   Printer-friendly
from the Buck-Feta! dept.

On July 5th of 2014, SoylentNews was officially given approvals for its letters of incorporation as a Public Benefit Corporation.

This is a good opportunity to take stock of where we are. Read on past the fold for coverage of:

  • Fundraising
  • Site activity
  • Folding@Home
  • Journals of Spam Accounts
  • Submit Stories
  • Thank YOU

Fundraising: For the first half of 2019, the community came through with room to spare! Not only did we attain our goal of $2000, but we also received approximately $549.38 towards our stretch goal of $1000 more. I'll leave the current totals in the Site News slashbox for a few more days before resetting for fundraising for the second half of the year. Please feel free to Subscribe as any new amounts received will go toward our next funding period. In case you were wondering, we had about 100 subscriptions during this period with the largest amounts being $200(x2), $120, $100(x3), and $80; all the rest were for $50 or less. Though this is an all-volunteer organization, we do have to pay for servers, domain registrations, and a CPA to file our taxes, so these subscriptions are key to our being able to "keep the lights on". THANK-YOU!

Site activity: A quick look around the site reveals some interesting statistics. As of my writing this, our story count stands at 27,861 stories. To that, add the 4,178 journal entries that have been posted by the community. We currently have 858,117 comments posted. We have had, however, comparatively few comment moderations: 586,164. We have 8,225 registered users, too!

In case you did not see the follow-up story, we have not performed moderation bans for excess up/down moderations in well over a year. Any moderations by user1 of comments by user2 that exceed 4 per day are reviewed and are simply reversed if it is determined than a mod bomb or sockpuppet moderation is happening. This is in lieu of our implementing code that will automatically handle this in the future when our dev team can scrounge up some spare time to implement it and merge it in. I had my information incorrect in a story posted a while back and want to set the record straight. Remember, there are a few thousand other users on this site, it is not necessary for you to single-handedly moderate all the comments that need it; if it really is that bad or good, then surely someone else will share your perspective and be along before too long to do the same. And, if not, it is not the end of the world; this is a place to discuss stories; not to set the entire world on the right path. In other words: Login to the site and Use your mod points!

Folding@Home: This has not been mentioned in a while, but we do have a SoylentNews Folding@Home (F@H) team... currently ranked #231 in the world! (Based on current trends, we should be #230 within a couple days.) This is purely voluntary by members of the community who saw an opportunity, banded together, and just got things started. Hat tip to SirFinkus for the original impetus and early organizing that has helped us get to where we are today. The F@H client automatically takes advantage of idle time on your CPU/GPU. Once installed and set up, it requests a work unit, processes it, and return the results. These calculations help support research into fighting maladies such as Parkinson's disease.

Journals of Spam Accounts: We have observed accounts being created which have posted no comments, made no story submissions, performed no moderations, but have made postings of a commercial nature to their journal. These are currently blocked from appearing in the "Most Recent Journal Entries" slashbox that appears on the main page until the user has accumulated at least 10 Karma. This reduces their visibility to the casual reader of the site, but could be leveraged to help boost the search engine rankings of the affiliated enterprise.

Some numbers: Out of 4178 journal entries posted to SoylentNews since the beginning, 528 were posted by users who have less than 10 Karma. Of those 528 journal entries, only 26 have a Karma greater than zero. Further, the most recent journal posting from those 26 was in March of 2018 — well over a year ago.

My take on this is that the good name of SoylentNews is being used to advance the standing of commercial enterprises. We strive to be impartial in our coverage and have never taken any remuneration from any outside enterprise lest it give even the appearance of favoritism to what we publish on this site.

Rather than make an edict from on high, I prefer to hear what the community thinks. What, if anything, have I forgotten to look at? Is this even a problem? If it is a problem, what, if anything should be done about it? Require a minimum Karma of 10 to even post in a Journal?

Submit Stories: It is summer time in the northern hemisphere and most research facilities are running with reduced staffing so folks can take advantage of the weather. Also, most schools are on summer vacation, as well. The result is that much less research is performed now than would be during the normal school year. This period is sometimes referred to as the "Silly Season" where on-line publications (many of whom are also running with fewer staff) tend to run more fluff pieces. Please bear with us as we try to find and provide the community with stories of interest with an emphasis on STEM.

This is also a good opportunity to try your hand at submittting a story to SoylentNews. Please see the Submission Guidelines. If you have any questions, feel free to pop into the "#editorial" channel on IRC (Internet Relay Chat) and ask for help. As some of our staff may also be taking a vacation, there may not be someone there to immediately respond, so please bear with us. I well remember the thrill I felt when I first saw a story submission of mine accepted and posted to the site!

If the best you can do is submit a link, then do that. If you quote any text directly from the story, please enclose it in <blockquote> ... </blockquote> tags. Of course the closer the submission is to publication-ready, the less work is required of an editor, and the better chance that one of us will run with it. Most importantly, have fun and don't be afraid to try. We all had the experience of making our first story submission and are willing to help you get comfortable with the process, as others did with us.

Thank YOU: We are the little site that could. Splintered off from the non-listening corporate overlords at Slashdot, a bunch of PO'd folk got together and bludgeoned an out-of-date code base into a solid and responsive system. The community that has formed here... I regularly see people looking to find what they can give to the community rather than what they can take from it. We could not have done it without YOU. Providing story submissions. Writing comments and journal articles. Moderating comments. It is a group effort, and this is a group I am grateful to be a part of.

In addition, there is the unheralded work of a small staff who volunteer their time to (try) and keep everything up and running smoothly. For example, Linode (our webhosting provider) recently rebooted 5 of our servers to handle an exploit that was making the rounds. Thanks to the excellent design and implementation, the site kept right on running uninterrupted. There was a minor glitch this morning that a restart of varnishd took care of. And that was it. No muss, no fuss. I count myself most fortunate to be working with such capable and talented people who give so selflessly of themselves to keep this all running so smoothly.

It has been a privilege to serve this community over the past 5 years and it is my fervent hope that we have many more years to come!

Original Submission