SoylentNews is people
SoylentNews
Earlier tonight, I modified our varnish rules to redirect all traffic to https://soylentnews.org if they came in as plain HTTP. Unfortunately, due to dropping SSLv3 support to prevent POODLE attacks, IE6 clients will no longer be able to reach SoylentNews. If this seriously inconveniences a large number of users, we may go through the trouble of whitelisting IE6 to drop down to HTTP only.
In addition, I applied an experimental update to production to try and clear as many errors as possible from the Apache error logs, in an attempt to continue isolating any remaining bugs and slowdowns. I also ripped out more dead code related to FireHose, Achievements, and Tags. As such, site performance appears to roughly be back to where it should be, and I have yet to see any 500 errors post-upgrade (though I concede that said update has only been up for about 2 hours at this point).
Tor traffic is set to bypass HTTPS due to the fact there is no way to prevent a self-signed certificate warning, and by design, tor both encrypts and authenticates hosts when connecting to them. A few lingering issues with the tor proxy were fixed with most recent code push, and the onion site should be back to functioning normally
P.S. I'm aware that the site is generating warnings due to the fact we use a SHA-1 based certificate. We will be changing out the certificate as soon as reasonably possible.
(Score: 2) by Techwolf on Sunday June 07 2015, @02:11AM
What about all the
OK
The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator, admin@soylentnews.org and inform them of the time the error occurred, and anything you might have done that may have caused the error.
More information about this error may be available in the server error log.
That I have MANY other been getting and bitching about in other stories?
(Score: 2) by frojack on Sunday June 07 2015, @02:15AM
Yeah that is kind of annoying, and I doubt any reports by us would ever be able to be tied back to the logged event.
These things are always useless..
If I simply hit back button and resubmit, it invariably goes through.
No, you are mistaken. I've always had this sig.
(Score: 2) by NCommander on Sunday June 07 2015, @02:16AM
Have you gotten any within the last hour? The issue is intermediate and isn't leaving much of an error log to work from. I'm trying to remove noise from the logs to better isolate the error.
Still always moving
(Score: 2) by gman003 on Sunday June 07 2015, @04:40AM
I got two trying to load comments on another article, and one trying to reply to your comment. It's not as many as I was seeing but there's still errors there.
(Score: 2) by NCommander on Sunday June 07 2015, @05:05AM
I was able to catch at least one 500 in the error_log file (posted elsewhere in this thread). I'm slightly at a loss whats causing it, but will continue to debug.
Still always moving
(Score: 2) by goodie on Sunday June 07 2015, @08:05AM
Just got one trying to load this thread's comments. hit back and click again, worked.
(Score: 1) by PReDiToR on Sunday June 07 2015, @12:34PM
It's indistinguishable from magic [catb.org].
Do not meddle in the affairs of geeks for they are subtle and quick to anger.
(Score: 2) by Techwolf on Sunday June 07 2015, @02:25AM
This error cause the screw on a couple posts earlier today. I did a quick post about f-droid. Then had time later to look up the website name, did a reply to that with the name of the site, hit post and got the error, hit the back button and tired again. It went through. HOWEVER, I found myself logged out and that post was done as AC.
(Score: 2) by NCommander on Sunday June 07 2015, @02:40AM
Yeah, I've been having the same sort of issues. This patch fixes everything I could find, but the error log was spamming a lot of warnings and it was hard to isolate what was noise and what wasn't so I took a blowtorch and removed as much as I could. I'll get another changeset together to remove more warnings to find the actual error
Still always moving
(Score: 0) by Anonymous Coward on Sunday June 07 2015, @03:19AM
It didn't happen before. So something changed to introduce it, and this happened recently.
(Score: 2) by NCommander on Sunday June 07 2015, @03:24AM
Site upgrade to rehash is what broke it. These are issues that didn't show up in the development environment before deployment.
Still always moving
(Score: 0) by Anonymous Coward on Sunday June 07 2015, @03:41AM
What the fuck is rehash? If the site worked without rehash, but doesn't work with rehash, and whatever rehash is it doesn't offer any obvious benefit, then rehash should be removed.
(Score: 1, Interesting) by Anonymous Coward on Sunday June 07 2015, @04:30AM
Maybe you missed this Upgrade Plans: Slashcode 15.03/Rehash 15.04 [soylentnews.org] which was published to the site on March 13th. In part, it stated:
The code that runs the site is based on *old* versions of Apache and perl; they are so old that they are no longer supported (i.e. no longer receive security or bug fixes.) As I understand it, this leaves the site open to security vulnerabilities, among other things. Of course, there were incompatible changes made to those so things that used to work fine, no longer do so. Besides those changes, the DB engine, as I understand it, needed an upgrade, too.
Car analogy: the car looked okay, but the engine and the frame were all rusted out. We've done a complete replacement of all that underneath stuff, and things need to get ironed out and tuned so that it all works together again. But, now, we have low-profile radial tires for better grip, a turbo on the engine for more power, and a completely new suspension system for better handling in the turns and a better ride. Everything didn't quite line up properly, so we're doing some adjustments to get it all working together properly.
That is from my limited perspective and understanding; I'm sure Ncommander or one of the other devs will correct me if I'm too wide of the mark on that.
Remember how the site used to regularly *crash* for the first few months are go-live? The site had become so stable that these current annoyances are causing such a stir suggests to me just how fine a job our devs have done. I, for one, commend them for all their hard work which they generously donated in their free time, and without pay.
(Score: 0) by Anonymous Coward on Sunday June 07 2015, @07:28AM
You can't say that! You just can't!
ACs only post AC because they don't want the blowback! Named accounts are the only way you can tell who someone really is! Nobody would ever use a pseudonym, that would just be dishonest...
(Score: 2, Interesting) by Anonymous Coward on Sunday June 07 2015, @04:29AM
Could the errors be related to the mod_perl 2 conversion?
There are know issues where it over aggressively caches old code and modules that have changed and will not de-cache then without a hard restart, even when tools used to hint or prompt it to refresh it's cache are used. This causes it to incorrectly try to call into the no longer current code resulting in server errors. The caching can be inconsistent across different apache child processes, which further ads to the mayhem.