The Mighty Buzzard writes:
Yeah, so, failure to babysit the db node that was scheduled for a reboot on the 5th resulted in a bit of database FUBAR that left us temporarily losing everything from then to now. Fortunately we had a backup less than six hours old, restored from it, and appear to be copacetic now. Except for the missing five hours and change.
I'd usually make some sort of dumb joke here but it was already four hours past my bedtime when I found out about the problem. My brain is no work good anymore. Fill in whatever dad joke or snark about getting a do-over for a change strikes your fancy.
(Score: 2) by coolgopher on Monday August 10 2020, @04:34AM (2 children)
Thanks TMB, and good job on the recovery.
(Score: 2) by The Mighty Buzzard on Monday August 10 2020, @05:05AM (1 child)
Eh, passable but I still ain't happy with it. That was a recoverable if sneaky crash. If I'd spent more time with clustered databases I might have caught it without data loss this time. I damned well will next time.
My rights don't end where your fear begins.
(Score: 2) by Reziac on Monday August 10 2020, @03:49PM
Look on the bright side... thanks to your efforts, we never get to see the same fuckup twice. I realize this makes it hard on the slow learners, but such is life.
And there is no Alkibiades to come back and save us from ourselves.