Stories
Slash Boxes
Comments

SoylentNews is people

posted by takyon on Tuesday February 28 2017, @10:58PM   Printer-friendly
from the mission-critical-infrastructure dept.

From The Verge:

Amazon's web hosting services are among the most widely used out there, which means that when Amazon's servers goes down, a lot of things go down with them. That appears to be happening today, with Amazon reporting "high error rates" in one region of its S3 web services, and a number of services going offline because of it.

Trello, Quora, IFTTT, and Splitwise all appear to be offline, as are websites built with the site-creation service Wix; GroupMe seems to be unable to load assets (The Verge's own image system, which relies on Amazon, is also down); and Alexa is struggling to stay online, too. Nest's app was unable to connect to thermostats and other devices for a period of time as well.

Isitdownrightnow.com also appears to be down as a result of the outage.

Amazon has suffered brief outages before that have knocked offline services including Instagram, Vine, and IMDb. There don't appear to be any truly huge names impacted by this outage so far, but as always, its effects are widespread due to just how many services — especially smaller ones — rely on Amazon.

There's no estimate on when service will be restored, but Amazon says it is "actively working on remediating the issue."

PS - BTW - thumbs up to our great behind the scenes guys! Good luck N.


Original Submission

 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
(1)
  • (Score: 3, Funny) by Anonymous Coward on Tuesday February 28 2017, @11:14PM (3 children)

    by Anonymous Coward on Tuesday February 28 2017, @11:14PM (#473105)

    cumslutzdoanal.com
    fistmytightsphincter.co.jp
    massivecockinmyhairybutt.ca
    gardening.com

    All seem to be down at the moment.

    • (Score: 0) by Anonymous Coward on Wednesday March 01 2017, @12:07AM (2 children)

      by Anonymous Coward on Wednesday March 01 2017, @12:07AM (#473131)

      Yet another reason to get a .au. It won't go down because it'll have nowhere to go but up.

      • (Score: 2, Funny) by Anonymous Coward on Wednesday March 01 2017, @12:46AM

        by Anonymous Coward on Wednesday March 01 2017, @12:46AM (#473147)

        .Au domains are like gold. Can't beat them.

      • (Score: 2) by bob_super on Wednesday March 01 2017, @12:59AM

        by bob_super (1357) on Wednesday March 01 2017, @12:59AM (#473154)

        You mean .aq?
        Of course, .fk also beats .au, whatever women claim.

  • (Score: 2) by kaganar on Tuesday February 28 2017, @11:14PM (3 children)

    by kaganar (605) on Tuesday February 28 2017, @11:14PM (#473106)
  • (Score: 4, Interesting) by mendax on Tuesday February 28 2017, @11:49PM (14 children)

    by mendax (2840) on Tuesday February 28 2017, @11:49PM (#473121)

    This is a classic example of the evils of cloud computing. It's all well and good that cloud computing relieves a developer the need to own, manage, and house his equipment, but when cloud computing fails, it's a spectacular fail!

    --
    It's really quite a simple choice: Life, Death, or Los Angeles.
    • (Score: 5, Insightful) by bob_super on Tuesday February 28 2017, @11:58PM (5 children)

      by bob_super (1357) on Tuesday February 28 2017, @11:58PM (#473122)

      Well, they do have an battalion of very experienced people working on fixing the problem, while you either get the day off or check your backups...
      You're outsourcing most of the aggravation.

      • (Score: 3, Insightful) by Anonymous Coward on Wednesday March 01 2017, @12:12AM

        by Anonymous Coward on Wednesday March 01 2017, @12:12AM (#473135)

        Having been in a 'devops' group at one point. Trust me that aggravation is real. When I lost my job last year. I fucking danced out of that building. Not worrying about some server 3 states away eating itself is awesome.

        This sort of thing has positive and negatives. Positive is if you get a good provider the headache is nearly eliminated. They do a good job with 'never goes down'. The down side is when they do (and they will) you are stuck with 'ETA blah blah blah' and usually poor communication about what is going on. If they fold and go out of business. Well you are SOL. As you do not really own it.

      • (Score: 2) by mendax on Wednesday March 01 2017, @12:29AM (3 children)

        by mendax (2840) on Wednesday March 01 2017, @12:29AM (#473143)

        Well, you do have a point.

        --
        It's really quite a simple choice: Life, Death, or Los Angeles.
        • (Score: 0) by Anonymous Coward on Wednesday March 01 2017, @12:47AM (2 children)

          by Anonymous Coward on Wednesday March 01 2017, @12:47AM (#473148)

          Well, you all seem to begin comments with "well."

          • (Score: 0) by Anonymous Coward on Wednesday March 01 2017, @03:27PM (1 child)

            by Anonymous Coward on Wednesday March 01 2017, @03:27PM (#473329)

            Well, those comments are obviously going well right from the start.

    • (Score: 3, Interesting) by aclarke on Wednesday March 01 2017, @01:52AM (7 children)

      by aclarke (2049) on Wednesday March 01 2017, @01:52AM (#473170) Homepage

      Sure OK, but have you factored in the cost of these companies doing it all themselves, the expertise they'd have to hire, and the downtime they'd experience with their own servers compared to AWS? It's not like overworked devops folks can run a farm of their own servers, in addition to everything else, for 99.99% uptime.

      Our uptime has improved since moving to AWS. I'm a reasonably confident sysadmin (it's not my full-time job) and I inherited a project without the budget or resources to manage the servers. AWS also reduced our hosting costs, so I'd call that a win/win. Oh yeah, and better security, although I won't go into why. I don't run on us-west-1 so I thankfully wasn't hit by today's outage. We are stuck on one region though because even though it's "the cloud", it turns out it still costs more to set up geographical redundancy. Plus more to run the servers. Plus the complexity of managing it. I'd love to do that but I just don't have the budget, which means that if our region goes down, so do we and our clients are just going to have to be OK with that.

      We do at least have a warm backup in another region, so if things go really really wrong at least we can be back up in probably 2 hours.

      • (Score: 2, Insightful) by Anonymous Coward on Wednesday March 01 2017, @02:04AM (1 child)

        by Anonymous Coward on Wednesday March 01 2017, @02:04AM (#473173)

        we can be back up in probably 2 hours.
        Do yourself a *HUGE* favor. Test it.

        We thought we were 5 seconds. Turns out we were 12 hours. We got it down to about 30 seconds. It did however take a lot of testing to get right.

        • (Score: 2) by aclarke on Wednesday March 01 2017, @01:41PM

          by aclarke (2049) on Wednesday March 01 2017, @01:41PM (#473298) Homepage

          When I tested it, it took about 30 minutes. So I tell people two hours to give myself some time to pat myself on the back, or possibly freak out, depending on how things go.

      • (Score: 0) by Anonymous Coward on Wednesday March 01 2017, @09:02AM (2 children)

        by Anonymous Coward on Wednesday March 01 2017, @09:02AM (#473246)

        > Sure OK, but have you factored in the cost of these companies doing it all themselves, the expertise they'd have to hire, and the downtime they'd experience with their own servers compared to AWS?

        Yes. Usually still two times cheaper than AWS.

        • (Score: 0) by Anonymous Coward on Wednesday March 01 2017, @11:39AM

          by Anonymous Coward on Wednesday March 01 2017, @11:39AM (#473264)

          Usually still two times cheaper than AWS.

          I'm... not sure how to parse that. Do you mean it costs half as much? (Not a diss, honest question.)

          And the fortune rubs it in :)

          "Protozoa are small, and bacteria are small, but viruses are smaller than the both put together."

        • (Score: 2) by aclarke on Wednesday March 01 2017, @01:40PM

          by aclarke (2049) on Wednesday March 01 2017, @01:40PM (#473297) Homepage

          We saved about 70% of our hosting costs by moving from a data centre and Google App Engine to AWS. It was a few years ago, but AWS was about half the cost of our data centre, and we got newer/faster hardware and more redundancy along with it.

      • (Score: 2) by TheRaven on Wednesday March 01 2017, @12:23PM

        by TheRaven (270) on Wednesday March 01 2017, @12:23PM (#473276) Journal
        The problem is correlated failure. This was why Katrina caused a load of insurance companies to go out of business (and not pay out as a result), whereas the hurricane the next year did more damage but didn't kill any insurance companies. It's easy to understand the risk of service X going down and if services X, Y, and Z are independent then you can handle them as independent risks. If they all depend on (or, worse, provide some portion of) some common infrastructure then the probability of failure might be lower, but the probability of independent failure is zero: if one goes down then they all will. You see the same thing on a smaller scale when a small company gets two ISPs to provide connections so that they have high reliability Internet access, only to discover that they use the same back-haul provider and all of the downtime comes from there, so if one goes down the other one will as well.
        --
        sudo mod me up
      • (Score: 2) by tangomargarine on Wednesday March 01 2017, @04:20PM

        by tangomargarine (667) on Wednesday March 01 2017, @04:20PM (#473358)

        the expertise they'd have to hire

        Hey, sounds like more jobs! :)

        --
        "Is that really true?" "I just spent the last hour telling you to think for yourself! Didn't you hear anything I said?"
  • (Score: 3, Interesting) by rob_on_earth on Wednesday March 01 2017, @07:55AM (4 children)

    by rob_on_earth (5485) on Wednesday March 01 2017, @07:55AM (#473237) Homepage

    spent 3 hrs last night trying to get my Pebble watch to work on a new phone. All the watch faces had copied across except one(the only one I wanted). I must have just hit the S3 problems as the last one started copying. I Assumed its Pebble taking down servers but they have pledged not to do that until end of 2017 at the earliest. All working again this morning.

    Back in December 2013 I tried to use Google Docs for the first time. Created a test document saved OK so I stated writing. An hour later the first recorded significant downtime of Google apps occurred and I have been fearful of cloud/internet centralised services ever since.

    Guess I am just lucky.

    • (Score: 2, Funny) by Anonymous Coward on Wednesday March 01 2017, @08:31AM (2 children)

      by Anonymous Coward on Wednesday March 01 2017, @08:31AM (#473242)

      Ladies and gentlemen, we found the culprit.

      • (Score: 0) by Anonymous Coward on Wednesday March 01 2017, @02:47PM (1 child)

        by Anonymous Coward on Wednesday March 01 2017, @02:47PM (#473316)

        having a watch that doesnt work right because a data center thousands of miles away is broke... that's crazy.

        I think I m ay stick to winding the watch if I want reliable timekeeping. Even if it drifts a few minutes, it at least won't be inoperable if someone else thousands of miles away makes a mistake unrelated to my ability to wear a watch.

        • (Score: 2) by Bot on Wednesday March 01 2017, @11:22PM

          by Bot (3902) on Wednesday March 01 2017, @11:22PM (#473604) Journal

          I would also consider that, should some perturbation/cosmic ray apocalypse hit your region and fry electronics, the only way to coordinate with others, set meetings, is through old fashioned watches.

          --
          Account abandoned.
    • (Score: 0) by Anonymous Coward on Wednesday March 01 2017, @03:24PM

      by Anonymous Coward on Wednesday March 01 2017, @03:24PM (#473328)

      I Assumed its Pebble taking down servers

      Obviously they are very effective on that.

  • (Score: 1) by StarryEyed on Wednesday March 01 2017, @04:06PM

    by StarryEyed (2888) on Wednesday March 01 2017, @04:06PM (#473353)

    I'm sure that this will ALSO be a new "feature" provided by Wix to Deviantart. They too can go down by surprise, once properly migrated.

(1)