Rehash 15.05.1 - Release Notes
- Fixed internal cache breakage due to rows from the database no longer being returned by id - paulej72
- Located and fixed issues related to getSkin() not returning the mainpage nexus - paulej72
- Debugged SQL queries to locate deadlocks in MySQL cluster - NCommander
The primary cause of the slowdown was due to the fact that rehash did large JOIN operations on text columns in MySQL. This is bad practice in general due to performance reasons, but it causes a drastic slowdown with MySQL cluster, which prevents the query optimizer from doing what's known as a "pushdown", and allowing the query to execute on the NDB nodes. This caused article load to be O(n*m), where n was the number of articles in the database and m was the number of articles with the neverdisplay attribute set. The revised queries now load at O(1). Instead it had to do multiple pulls from the database and assemble the query data on the frontend, a process that took 4-5 seconds per problematic query. The problem was compounded that there are limited number of httpd daemons at any given moment, and any database pull that hit a problematic query (which were in index.pl and article.pl) would cause resource exhaustion.
Fortunately, our load balancer and varnish cache have a fairly high timeout waiting for httpd to come available, preventing the site from soyling itself under high load, or when we do an apache restart, which prevented SN from going down. Thank you for everyone's patience with this matter :).
~ NCommander
(Score: 2) by Tork on Tuesday June 02 2015, @10:32PM
🏳️🌈 Proud Ally 🏳️🌈
(Score: 2) by iWantToKeepAnon on Tuesday June 02 2015, @10:35PM
I like:))
We were running slashcode and now we are on rehash ... rehash was a part of slashcode (right?) that y'all split out of slashcode. I'm confused; can someone give a high level explanation of slash vs. rehash? Thx.
"Happy families are all alike; every unhappy family is unhappy in its own way." -- Anna Karenina by Leo Tolstoy
(Score: 2) by iWantToKeepAnon on Tuesday June 02 2015, @10:38PM
"Happy families are all alike; every unhappy family is unhappy in its own way." -- Anna Karenina by Leo Tolstoy
(Score: 5, Informative) by mrcoolbp on Tuesday June 02 2015, @10:51PM
Slashcode vs Rehash quick answer:
Basically with this major overhaul of some under-the-hood stuff (namely porting to mod_perl version 2), we've decided to differentiate our fork of slashcode with a new name -- rehash.
There's some more info in this post [soylentnews.org] (scroll down to "Introducing Rehash").
(Score:1^½, Radical)
(Score: 3, Interesting) by kadal on Wednesday June 03 2015, @01:42AM
I think we should use "soyled" instead of "slashdotted"
(Score: 3, Funny) by Ryuugami on Wednesday June 03 2015, @01:53AM
We already do.
Well, some people use "soyled", some people use "soylenced". Pick your favorite :)
If a shit storm's on the horizon, it's good to know far enough ahead you can at least bring along an umbrella. - D.Weber
(Score: 2) by captain normal on Wednesday June 03 2015, @05:57AM
Slashcode is dead. Long live Rehash!
Everyone is entitled to his own opinion, but not to his own facts"- --Daniel Patrick Moynihan--
(Score: 2) by kurenai.tsubasa on Tuesday June 02 2015, @11:07PM
This caused article load to be O(n*m), where n was the number of articles in the database and m was the number of articles with the neverdisplay attribute set.
<keanu-reeves>woah</keanu-reeves>
The revised queries now load at O(1).
Very nice. Turning essentially O(n²) into O(1) is always most excellent. Party on, dudes!
(Score: 2, Informative) by Anonymous Coward on Wednesday June 03 2015, @01:57AM
Unlike MySQL, PostgreSQL has an excellent query planner. Join whatever you need to without surprises.
(Score: 2) by NCommander on Wednesday June 03 2015, @10:14PM
Replacing the backend with postgreSQL is on the TODO.
Still always moving