Stories
Slash Boxes
Comments

SoylentNews is people

Meta

SoylentNews.org had a Site Crash and We're Back! (mostly)

posted by martyb on Friday May 21 2021, @12:25AM   Printer-friendly

As many of you noticed, we had a site crash today. From around 1300 until 2200 UTC (2021-05-20).

A HUGE thank you goes to mechanicjay who spent the whole time trying to get our ndb (cluster) working again. It's an uncommon configuration, which made recovery especially challenging... there's just not a lot of documentation about it on the web.

I reached out and got hold of The Mighty Buzzard on the phone. Then put him in touch with mechanicjay who got us back up and running using backups.

Unfortunately, we had to go way back until April 14 to get a working backup. (I don't know all the details, but it appears something went sideways on neon).

We're all wiped out right now. When we have rested and had a chance to discuss things, we'll post an update.

In the meantime, please join me in thanking mechanicjay and TMB for all they did to get us up and running again!

 
This discussion has been archived. No new comments can be posted.
SoylentNews.org had a Site Crash and We're Back! (mostly) | Log In/Create an Account | Top | 184 comments | Search Discussion
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.

  • (Score: 3, Insightful) by janrinok on Friday May 21 2021, @04:39PM (1 child)

    by janrinok (52) Subscriber Badge on Friday May 21 2021, @04:39PM (#1137584) Journal

    We are, at this very moment, discussing options on a private channel. And currently ALL of our resources are currently working on recovering from yesterday, or keeping the site going today.

    The only thing that is causing a problem (repeatedly) is one element of the system configuration that is not providing us with any benefit whatsoever - so that is what we are currently working on removing. The rest of the site is working just as we want it to. Let me explain it in an auto analogy - which is the traditional way of doing things around here. What you are suggesting is that we currently have a flat tire but you are recommending that we also paint the car, change the upholstery and fit a new engine too.

    If it can be done in a day I will await your contribution by, shall we say, Sunday evening? Show me something working to convince me - not just make ridiculous suggestions that we haven't got the resources to complete anyway.

    Starting Score:    1  point
    Moderation   +1  
       Insightful=1, Total=1
    Extra 'Insightful' Modifier   0  
    Karma-Bonus Modifier   +1  
    Total Score:   3  

  • (Score: 0) by Anonymous Coward on Friday May 21 2021, @10:46PM

    by Anonymous Coward on Friday May 21 2021, @10:46PM (#1137643)

    The only thing that is causing a problem (repeatedly) is one element of the system configuration that is not providing us with any benefit whatsoever - so that is what we are currently working on removing.

    Perhaps that is for the best since you all apparently don't know how to use it properly. Quite a number of people use it under higher loads with better uptimes, after all.