SoylentNews

As many of you noticed, we had a site crash today. From around 1300 until 2200 UTC (2021-05-20).

A HUGE thank you goes to mechanicjay who spent the whole time trying to get our ndb (cluster) working again. It's an uncommon configuration, which made recovery especially challenging... there's just not a lot of documentation about it on the web.

I reached out and got hold of The Mighty Buzzard on the phone. Then put him in touch with mechanicjay who got us back up and running using backups.

Unfortunately, we had to go way back until April 14 to get a working backup. (I don't know all the details, but it appears something went sideways on neon).

We're all wiped out right now. When we have rested and had a chance to discuss things, we'll post an update.

In the meantime, please join me in thanking mechanicjay and TMB for all they did to get us up and running again!

We had a few hours this evening where there were issues we didn't notice with new submissions. We had been mothballing (setting the number of matches necessary beyond what would ever match anything) some spam filters exclusive to submissions and managed to somehow confuse one of the Apache web frontends. The offending filters are now just deleted entirely and incapable of confusing even MS Paint.

Please excuse the embuggerance. We now return you to your regularly scheduled arguments.

TMB