SoylentNews is people
SoylentNews
I've had some occasions of late to peer through the looking glass into a world that I hadn't seen much of previously. Specifically, I'm talking about the world of so-called "cloud" stuff, where you basically pay someone else to build and run stuff for you, instead of doing it yourself.
I'll skip the analysis of build vs. buy and just jump straight to the point where you've chosen "buy". Then you've had a whole bunch of fun outages caused by something going wrong with their services. Finally, you reach the point of a sit-down talk with the vendor to figure things out. Maybe they send some sales people too, or perhaps it's just engineers. You talk for a while, and before long, you realize what happened.
[...] This becomes obvious when talking about some problem you experienced at the hands of their system. The whole time, their dashboard stayed green because from their point of view, they had tremendous availability. We're talking 99.999% here! Totally legit!
Meanwhile, you were having a really bad day. Nothing was working. Your business was in shambles. Your customers were at your throat yelling for action, and all you could do is point at the vendor. What happened?
Well, this is the point where you find out that their "99.999%" availability is for their entire system. They see that, and they're good. It's not a problem! Everything is fine.
This also completely misses the fact that for you, everything was failing. It doesn't matter though, since your worst day still won't move the needle on their fail-o-meter. They won't see you. They won't have any idea anything even happened until you complain weeks later. You are the bug on the windscreen of the locomotive. The train has no idea you were ever there.
The problem is that they weren't monitoring from the customer's perspective. Had they done that, it would have been clear that oodles of requests from some subset of customers were failing. They would have also realized that certain customers had all of their requests failing. For those customers, there were no nines to be had that day.
Seriously, if you have a multi-tenant system, you owe it to your customers to monitor it from their point of view. Otherwise, how can you possibly know when you've done something that'll leave them in the cold?
(Score: 2) by bradley13 on Sunday July 21 2019, @05:39PM
Man, who pissed in your cornflakes?
Look, there are situations where the cloud is a stupid solution. And there are also situations where it is an incredible solution. The trick is to know the difference.
As for system administrators being mistreated: there are companies, and then there are companies. Let me digress...
When I'm consulting (which I do a fair amount of, mostly for SMEs that are too small to have their own IT staff), one of the things I look for is the visibility of whoever is taking care of their computing infrastructure. If whoever they've hired is really visible, always out there fixing stuff, that's actually a horrible sign. It means that stuff is always breaking, users are poorly trained, etc... If everything "just works", and most people aren't sure who to call if they have a problem, that's generally a great sign.
So: If you have a company where the sys-admin is good enough to keep everything running, almost invisibly - and if this sys-admin is being mistreated - then the sys-admin needs to move on to a decent company. Alternatively, there are sys-admins who feel terribly put-upon, because they are simply incompetent. Their infrastructure is always failing. People are always disturbing their peaceful game of minesweeper, because of yet-another-outage. From the tone of AC's rant, well, I think the latter scenario is the more fitting...
Everyone is somebody else's weirdo.