Stories
Slash Boxes
Comments

SoylentNews is people

Meta
posted by martyb on Friday May 21 2021, @12:25AM   Printer-friendly

As many of you noticed, we had a site crash today. From around 1300 until 2200 UTC (2021-05-20).

A HUGE thank you goes to mechanicjay who spent the whole time trying to get our ndb (cluster) working again. It's an uncommon configuration, which made recovery especially challenging... there's just not a lot of documentation about it on the web.

I reached out and got hold of The Mighty Buzzard on the phone. Then put him in touch with mechanicjay who got us back up and running using backups.

Unfortunately, we had to go way back until April 14 to get a working backup. (I don't know all the details, but it appears something went sideways on neon).

We're all wiped out right now. When we have rested and had a chance to discuss things, we'll post an update.

In the meantime, please join me in thanking mechanicjay and TMB for all they did to get us up and running again!

 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 2) by bzipitidoo on Sunday May 23 2021, @02:25AM (1 child)

    by bzipitidoo (4388) on Sunday May 23 2021, @02:25AM (#1137873) Journal

    Subversion?? WTF? Do you know what you're talking about? That, and those other 2 tools you mention are software development tools, not system administration tools. A software repository is NOT a backup! Subversion isn't even the best, git is better. And git isn't even that new any more, been around for 15 years. git is distributed, cloning the repository in such a way that individual copies can act as backups. Not Subversion. Subversion is much more centralized. Lose that master repository, and you're going to have a rough time reconstructing it unless there is a backup.

    Backups are done with tools such as rsync. For monitoring, we used Nagios. Scheduling can be done with cron. Many other things were done with little scripts.

    Starting Score:    1  point
    Karma-Bonus Modifier   +1  

    Total Score:   2  
  • (Score: 0) by Anonymous Coward on Sunday May 23 2021, @06:15AM

    by Anonymous Coward on Sunday May 23 2021, @06:15AM (#1137906)

    That just a troll trolling, better off not to feed it.

    A completely unrelated point SVN == cathedral && git == bazaar