Stories
Slash Boxes
Comments

SoylentNews is people

posted by NCommander on Friday March 14 2014, @06:44AM   Printer-friendly
from the timebombs-are-exciting dept.
We had an hour or so or downtime today. After debugging, the root cause came from the SSL certificates we use to establish a database connection from the webserver to the actual DB. As a prelude GoLive, we migrated from unencrypted connections to encrypted connections as we have to cross the Linode internal LAN. In an attempt to improve data security, we generated a set of SSL certificates and used those to encrypt the MySQL connections. In the flurry of golive, no one thought to check the expiry date on said certificates. Out of the box, OpenSSL generates certificates with a one month expiry unless manually changed.

As you might expect, one month later, the certificates expired, and the database stopped accepting remote connections. New certificates were generated with a ten year expiration, and we continue to work towards better documenting our internal processes on the wiki to prevent this sort of thing from happening again. Apache, and slashd are running again, and we appear to be back to status-quo in terms of site operation.

A full incident report will be written up and posted to the wiki in the next few days.
 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 0) by Anonymous Coward on Friday March 14 2014, @01:32PM

    by Anonymous Coward on Friday March 14 2014, @01:32PM (#16349)

    The only conclusion i came to was that he may have been blocked and if he was it must have been for something bad. If it was due to evidence of ddos on his part then fair enough. I'm also not concluding that he's guilty of ddos, particularly since bot development has been encouraged and it's easy to cause a flood whilst testing a bot. There is a test channel for it but if he was banned for flooding a channel other than test whilst developing a bot, it would seem an undue punishment on the face of it. You are of course welcome to not jump to conclusions, but the rest of us will jump to whatever conclusions we see fit tyvm.

  • (Score: 0) by Anonymous Coward on Friday March 14 2014, @01:36PM

    by Anonymous Coward on Friday March 14 2014, @01:36PM (#16354)

    Dude calls out mattie_p in his comment on the other site, so maybe we can get mattie_p to comment on this?

    • (Score: 0) by Anonymous Coward on Friday March 14 2014, @01:52PM

      by Anonymous Coward on Friday March 14 2014, @01:52PM (#16366)

      I read the irc log for the day in question (searching for khyber from the moment he joined the channel). I honestly don't see what he did wrong. If he made threats in a private message, there's no public evidence of it that I know of (yet). He may have been acting like a dick, but he's in good company here. My conclusion is that he was censored for being a dick on irc. Maybe it was justified, maybe it wasn't but I'm not buying the ddos accusation without any kind of evidence. The bigger issue here is that if censoring (for whatever reason) becomes acceptable, there will be a reputation that comes with that.

      • (Score: 0) by Anonymous Coward on Friday March 14 2014, @01:59PM

        by Anonymous Coward on Friday March 14 2014, @01:59PM (#16374)

        You're absolutely right: we shouldn't ban people for language they might want to use, even if they're insulting other people. This is a bastion of free speech, no? And if it's not, who has the say in what can be said or not said? Does the First Amendment apply here?

      • (Score: 2, Informative) by sglane on Friday March 14 2014, @02:30PM

        by sglane (3133) on Friday March 14 2014, @02:30PM (#16393)

        I honestly don't see what he did wrong. [...] but I'm not buying the ddos accusation without any kind of evidence

        Some excerpts from http://logs.sylnt.us/%23soylent/2014-03-11.html [sylnt.us]

        [03:29:27] I'm willing to smack both of these ignorant nagging niggers upside the head to teach both of these ignorant fucks a lesson

        [03:36:49] I'm sick of being held hostage
        [03:37:05] So, I'm about to turn all my video chat servers into a bandwidth buster. Not a DDoS. Just a legitimate bandwidth bill raiser.
        [03:37:38] I'll start at 40TB aggregate bw and slowly bring it up to 400TB with legit page refreshes, link trawling, all multi-ip cloud-based
        [03:37:41] and you'd admit this in chat?
        [03:37:47] Let's see how they like a hostage fighting back
        [03:37:51] Why not?
        [03:38:00] to what end?
        [03:38:18] wait... how are we hostages? And does being a hostage mean no more free cheese?
        [03:38:19] Teach them an expensive lesson in holding their comunity hostage
        [03:38:27] teach them this squabbling is about to bite them in the ass and HARD

        [03:51:34] Well, in about 5 minutes I'll have this script finished and running. Stil not cutting either of these two fuckers some slack until they learn their lesson.
        [03:51:49] PLAY NICE OR DON'T PLAY AT AL

        [03:56:32] Legitimate page refreshes and link trawling have already been held as legal. I'm not doing a DDoS. I'm simply loading a page as requested by HTTP link trawling. Nothing different from a bot crawler except this one doesn't respond to robots.txt
        [03:56:45] and it constantly refreshes every link to check if there's been a change in the page
        [03:56:57] Khyber, to what end?
        [03:57:06] right, but you've now stated that you're doing it with intention to harm the site owners and users of the stie
        [03:57:07] What is the best case scenario resulting from this?
        [03:57:07] My own personal satisfaction, damn the lot of you.

  • (Score: 5, Informative) by isostatic on Friday March 14 2014, @02:26PM

    by isostatic (365) on Friday March 14 2014, @02:26PM (#16392) Journal

    The only conclusion i came to was that he may have been blocked and if he was it must have been for something bad.
    If it was due to evidence of ddos on his part then fair enough

    [03:27:35] <Khyber> So what the fuck is this Im reading about a buyer? We're already sold out? Well, fuck it, no more contributions from me. This is the last time for me guys. I'm out of this bulshit.
    .......
    [03:29:27] <Khyber> I'm willing to smack both of these ignorant nagging niggers upside the head to teach both of these ignorant fucks a lesson
    [03:29:35] <Khyber> And I'm starting to hunt their asses down right now.
    [03:29:44] <ibogi> you kiss your mom with that mouth?
    .......
    [03:36:49] <Khyber> I'm sick of being held hostage
    [03:37:00] <prospectacle> Sounds reasonable
    [03:37:05] <Khyber> So, I'm about to turn all my video chat servers into a bandwidth buster. Not a DDoS. Just a legitimate bandwidth bill raiser.
    [03:37:38] <Khyber> I'll start at 40TB aggregate bw and slowly bring it up to 400TB with legit page refreshes, link trawling, all multi-ip cloud-based
    [03:37:41] <Blackmoore> and you'd admit this in chat?
    [03:37:47] <Khyber> Let's see how they like a hostage fighting back
    [03:37:51] <Khyber> Why not?
    [03:38:00] <iammasci> to what end?
    [03:38:18] <SpallsHurgenson> wait... how are we hostages? And does being a hostage mean no more free cheese?
    [03:38:19] <Khyber> Teach them an expensive lesson in holding their comunity hostage
    [03:38:25] * SpallsHurgenson has a cheese fixation tonight
    [03:38:27] <Khyber> teach them this squabbling is about to bite them in the ass and HARD
    .......
     
    [03:58:10] <Khyber> Get NCommander and Barrabas in here if you want ANY chance of a peaceful settlement
    [03:58:16] <swiss> You're going to raise the price of the site, possibly to the point where all the investors stop putting in money?
    [03:58:16] <MrBluze> There is no fighting n ow
    [03:58:22] <BadCoderFinger> The end result is taking the site down.
    [03:58:28] <Khyber> cuz I'm down to testing the script on my own site right now to see if everythign works. Two minutes tops.

    He's a troll, but that's a common reason to ban someone from IRC.

    • (Score: 2, Informative) by Anonymous Coward on Friday March 14 2014, @03:04PM

      by Anonymous Coward on Friday March 14 2014, @03:04PM (#16417)

      Who kept feeding the troll? Trolls don't begin by threatening a ddos attack. Surely anyone who has been around irc for more than 5 minutes would know that the worst way to deal with a troll is to respond. If khyber's rants were ignored as all rants should be, it likely would never have escalated to such bitterness. Irc is full of trolls. If ops go around threatening anyone that they think might be trolling, users won't know what they can talk about and they're likely going to kill the channel (not literally, but it won't be interesting enough to keep users engaged).