from the internet-was-designed-to-route-around-breakage,-right? dept.
Akamai DNS global outage takes down major websites, online services:
Akamai is investigating an ongoing outage affecting many major websites and online services, including Steam, the PlayStation Network, Newegg, AWS, Amazon, Google, and Salesforce.
While the company has already acknowledged the issue, pinning it on an Edge DNS service problem, Akamai is still trying to find the underlying cause behind the incident.
"We are aware of an emerging issue with the Edge DNS service," the company said in an Edge DNS Service Incident notice.
"We are actively investigating the issue. If you have questions or are experiencing impact due to this issue, please contact Akamai Technical Support.
"In the interest of time, we are providing you the most current information available, which is subject to changes, corrections, and updates."
[...] "At 15:46 UTC [Thursday], a software configuration update triggered a bug in the DNS system, the system that directs browsers to websites. This caused a disruption impacting availability of some customer websites," the company said.
"The disruption lasted up to an hour. Upon rolling back the software configuration update, the services resumed normal operations. Akamai can confirm this was not a cyberattack against Akamai’s platform.
(Score: 1, Insightful) by Anonymous Coward on Friday July 23 2021, @12:19PM (4 children)
>> Akamai can confirm this was not a cyberattack against Akamai’s platform.
Akamai can confirm this was not an *external* cyberattack against Akamai’s platform.
(Score: 5, Insightful) by Anonymous Coward on Friday July 23 2021, @01:15PM (3 children)
Modern web-developers are effectively cyber-attackers from within or without any company they are working in.
If you want tested, usable, reliable technology and platforms to be corroded and rendered unusable, web-developers and UX designers can make it happen for you wholesale.
(Score: 2) by bzipitidoo on Friday July 23 2021, @04:04PM (2 children)
Companies should stop the cheap hiring practices! But no, rather than hire a computer science grad, they'll hire at half the pay the liberal arts college dropout with a single cert, in MS Exchange because they confused Exchange with IIS and confused web server administration with web development.
(Score: 0) by Anonymous Coward on Saturday July 24 2021, @10:27AM
(Score: 0) by Anonymous Coward on Sunday July 25 2021, @05:58PM
huh, where I worked, they outsourced it all to India. Even the emo kids were laid off.
reported problems went way down after a short while because people put up with problems before giving up and rebooting and putting up with the problems again until they could no longer get away with it. people called the helpdesk only to get an official excuse about productivity loss that wasn't more easily blamed on the difficult win 10 transition period or how o365 was slower for everyone, all the time, as opposed to exchange having the occasional issue even when it had been outsourced away (it later got migrated for cost purposes and there's no going back without probably replacing the people at the top of the org chart)
there are only a few people at work now that even care about computers anymore and don't give a shit about whatever next windows comes out because they assume it will just suck anyway because of how business practices make things worse as it is.
(Score: 1, Insightful) by Anonymous Coward on Friday July 23 2021, @05:34PM (3 children)
The whole server/client scheme has to go. It doesn't work on public networks. I mean, it does, for the bad people
(Score: 4, Touché) by darkfeline on Saturday July 24 2021, @03:44AM
You're right, let's kill DNS and just use IP addresses for everything.
But let's be honest, having to remember IP addresses for everything is a chore. We're going to end up giving them labels anyway, whether that be on a sticky note or text file somewhere. Instead of having to look up the IP addresses from the names and type them in manually, it'd be helpful to have some software resolve the names for us. Since we have software to resolve the names, it'd be nice if we can share name mappings with other people, since otherwise we're back to the problem of manually typing in IP addresses. Of course, we can distribute hostname to IP maps using some kind of peer to peer scheme, but within an organization it's more efficient to set up a hierarchy of resolver servers so the local network isn't loaded with redundant map exchanges. It'll improve performance too. Instead of every organization inventing their own protocol, we may as well make a standard protocol, so we could share a single FOSS implementation of the resolver servers. This also provides the interesting feature to talk to different hosts across organizations (let's call them domains) using these names. We could call it Domain Name System or something.
Join the SDF Public Access UNIX System today!
(Score: 2) by mcgrew on Saturday July 24 2021, @11:36AM (1 child)
Yeah? Neither of my sites were down. The sites that died all had third party ads, most of which go through Akamai servers. There's nothing wrong with client/server if it's used the way it was designed to be used.
mcgrewbooks.com mcgrew.info nooze.org
(Score: 0) by Anonymous Coward on Sunday July 25 2021, @06:15AM
Client/Server is fine for private LANs, but it's too centralized for the public network. That needs to be ad hoc, peer to peer. It's the only way around political boundaries and limits.
(Score: 3, Informative) by Anonymous Coward on Friday July 23 2021, @08:30PM
This is not accurate. Akamai fixed the problem in less than one hour, but the disruption in downstream DNS lasted much longer in some cases. For example, the Akamai disruption in Japan lasted at least 6 hours according to tests via various local ISPs. DNS caching period related.
(Score: 5, Interesting) by fakefuck39 on Friday July 23 2021, @10:05PM
"Gain peace of mind by extending your DNS to the edge, with high availability and performance, and resilience against DDoS attacks backed by a 100% uptime SLA"
so they got an SLA, not an SLO, meaning they are open to financial penalties for any downtime.
And now this:
"This is why relying solely on your own DNS servers, or on a single provider, makes your DNS — and by extension, your business — vulnerable."
It's bad to rely on a single provider for your DNS. Which is why you should go with us as your single provider.
I've actually hit this a lot with "cloud" services, and it's a real issue companies are choosing to ignore. They move a bunch of shit to Azure, and replicate it in there to several geo regions. They take some snapshots in Azure, they can spin up a Rubrik in Azure to back up the VMs running in the cloud. Nasuni is a great example - in fact, it's hard to figure out how much you have to pay Nasuni when you buy a filer from them. It's hidden in the small print of a quote that has 60 pages of irrelevant spam, and fake numbers.
See, they'll estimate the cost of your backups and subtract it out of what you owe them, because you got snapshots in the cloud now. Forget the fact that you also got snapshots in your current solution, which are replicated to DR, but you still want a backup - nah, for those you need a backup, for ours you don't, because "cloud." What you owe them, in cash, is tucked on page 46 among some marketing slides, and there is not table of contents for it. You'll figure it out when you get the bill.
And then some people screw something up, and you lose your connection to Azure, and all your shit is down. Or maybe the gov told microsoft to freeze all your storage while they investigate you for something, and destroy your business while they do it. Or, maybe someone paid their bill late, and now your storage, your snapshots, and your backups are zeroed out, and the storage assigned to other customers.
Like with DNS here, cloud providers are the single point of failure. But you can't really get your shit out of the cloud once it's in there - like to take a backup. The egress charges will run you millions.
Now, there are some companies trying to address this. Instead of moving prod to cloud, you move only DR to cloud, using something like Zerto or VCF. So when you got issues with Azure, you resync from prod to AWS, and ingress is almost free.
Here's the real reason shit moves to cloud. There are a bunch of toxic MBAs out there, and they went to college to learn skills like lying, muddying the waters, shifting blame for their bad decisions, and stealing. This is all they learned, they're too dumb to learn anything else. By moving shit to cloud, and costing the company extra millions, this DNS outage is now not their fault. Oh, DNS is still down, the company still lost revenue, but it's someone else's fault now. And while this cloud DNS, with more issues than your old DNS, and longer resolution time, costs five times as much, it actually looks cheaper on paper. So they saved the company money. How do you save money by spending more? Well, you hide the real cash total on page 46, and you put the adjusted ROI total on page 1.
Tesla did this for a while. You'd go on their site to see prices for a car - they were the same as a toyota camry. It wasn't till you got to the last page of the checkout you realized you owe them another $40k. You see, a Tesla accelerates faster than your current car, so you get there faster. For each stop light you save 35 seconds, which we round up to a minute. For a 2 hour commute per day, you save 30 minutes per day. At $500/hr - and your time is worth that - you save $40k on the car during the 10 years you'll own it. So it actually costs less than a Camry you see - now give us $80k for the car.