Stories
Slash Boxes
Comments

SoylentNews is people

Meta

SoylentNews.org had a Site Crash and We're Back! (mostly)

posted by martyb on Friday May 21 2021, @12:25AM   Printer-friendly

As many of you noticed, we had a site crash today. From around 1300 until 2200 UTC (2021-05-20).

A HUGE thank you goes to mechanicjay who spent the whole time trying to get our ndb (cluster) working again. It's an uncommon configuration, which made recovery especially challenging... there's just not a lot of documentation about it on the web.

I reached out and got hold of The Mighty Buzzard on the phone. Then put him in touch with mechanicjay who got us back up and running using backups.

Unfortunately, we had to go way back until April 14 to get a working backup. (I don't know all the details, but it appears something went sideways on neon).

We're all wiped out right now. When we have rested and had a chance to discuss things, we'll post an update.

In the meantime, please join me in thanking mechanicjay and TMB for all they did to get us up and running again!

 
This discussion has been archived. No new comments can be posted.
SoylentNews.org had a Site Crash and We're Back! (mostly) | Log In/Create an Account | Top | 184 comments | Search Discussion
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.

  • (Score: 0) by Anonymous Coward on Friday May 21 2021, @02:09PM (3 children)

    by Anonymous Coward on Friday May 21 2021, @02:09PM (#1137537)
    Go for it - it’s only a matter of time before it crashes again. No proper backups, no proper docs, does anyone want to bet TMB doesn’t know how to use utilities that have been around for ages like SVN or Mantis or Doxygen? I’m not talking new cutting edge stuff here, but stuff that even “old skool” coders have used.

    Nobody’s going to want to work with someone who is going to bitch about having to use any sort of structure in their workplace processes like version control. Or a bug tracker.

  • (Score: 4, Insightful) by Rosco P. Coltrane on Friday May 21 2021, @03:20PM

    by Rosco P. Coltrane (4757) on Friday May 21 2021, @03:20PM (#1137559)

    How much do you pay for the quality of service you seem to expect, that you sarcastically point to the lack of?

    Yeah, that's what I thought.

  • (Score: 2) by bzipitidoo on Sunday May 23 2021, @02:25AM (1 child)

    by bzipitidoo (4388) on Sunday May 23 2021, @02:25AM (#1137873) Journal

    Subversion?? WTF? Do you know what you're talking about? That, and those other 2 tools you mention are software development tools, not system administration tools. A software repository is NOT a backup! Subversion isn't even the best, git is better. And git isn't even that new any more, been around for 15 years. git is distributed, cloning the repository in such a way that individual copies can act as backups. Not Subversion. Subversion is much more centralized. Lose that master repository, and you're going to have a rough time reconstructing it unless there is a backup.

    Backups are done with tools such as rsync. For monitoring, we used Nagios. Scheduling can be done with cron. Many other things were done with little scripts.

    • (Score: 0) by Anonymous Coward on Sunday May 23 2021, @06:15AM

      by Anonymous Coward on Sunday May 23 2021, @06:15AM (#1137906)

      That just a troll trolling, better off not to feed it.

      A completely unrelated point SVN == cathedral && git == bazaar