Stories
Slash Boxes
Comments

SoylentNews is people

posted by martyb on Thursday March 14 2019, @11:58PM   Printer-friendly
from the a-bit-of-an-overstatement? dept.

The Reality of SSD Capacity: No-One Wants Over 16TB Per Drive

One of the expanding elements of the storage business is that the capacity per drive has been ever increasing. Spinning hard-disk drives are approaching 20 TB soon, while solid state storage can vary from 4TB to 16TB or even more, if you're willing to entertain an exotic implementation. Today at the Data Centre World conference in London, I was quite surprised to hear that due to managed risk, we're unlikely to see much demand for drives over 16TB.

Speaking with a few individuals at the show about expanding capacities, storage customers that need high density are starting to discuss maximum drive size requirements based on their implementation needs. One message starting to come through is that storage deployments are looking at managing risk with drive size – sure, a large capacity drive allows for high-density, but in a drive failure of a large drive means a lot of data is going to be lost.

[...] Ultimately the size of the drive and the failure rate leads to element of risks and downtime, and aside from engineering more reliant drives, the other variable for risk management is drive size. 16TB, based on the conversations I've had today, seems to be that inflection point; no-one wants to lose 16TB of data in one go, regardless of how often it is accessed, or how well a storage array has additional failover metrics.

Related: Toshiba Envisions a 100 TB QLC SSD in the "Near Future"
Samsung Announces a 128 TB SSD With QLC NAND


Original Submission

 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 4, Insightful) by The Mighty Buzzard on Friday March 15 2019, @12:25AM (17 children)

    If you're relying on a non-backed-up drive to never fail in a big commercial environment, you have no business making tech decisions in the first place. If you're not, you aren't going to lose any data if it fails and you're going to save tons on power bills by going with the (almost) largest drives you can find.

    --
    My rights don't end where your fear begins.
    Starting Score:    1  point
    Moderation   +2  
       Insightful=1, Interesting=1, Total=2
    Extra 'Insightful' Modifier   0  
    Karma-Bonus Modifier   +1  

    Total Score:   4  
  • (Score: 4, Insightful) by NateMich on Friday March 15 2019, @12:35AM (9 children)

    by NateMich (6662) on Friday March 15 2019, @12:35AM (#814558)

    You may not "lose" the data, but the real issue they are probably talking about is that any given machine is out of service until the data is put back.
    Of course, a better system is one where these drives are in an array, and when one fails you just change it out and keep going. No downtime required.

    • (Score: 2) by PartTimeZombie on Friday March 15 2019, @12:41AM (5 children)

      by PartTimeZombie (4827) on Friday March 15 2019, @12:41AM (#814563)

      Is there any commercial application where a drive would not be part of an array? Maybe as an offline backup I suppose.

      • (Score: 3, Insightful) by The Mighty Buzzard on Friday March 15 2019, @01:08AM (4 children)

        Small businesses. Anyone who moves enough cash to afford someone who knows how to set up an office IT structure has no excuse for not doing so though.

        --
        My rights don't end where your fear begins.
        • (Score: 2) by PartTimeZombie on Friday March 15 2019, @01:35AM (3 children)

          by PartTimeZombie (4827) on Friday March 15 2019, @01:35AM (#814587)

          Now you point that out I've realised you're right of course.

          In fact I have turned down a job setting up a server for a small business because it very quickly became clear that they didn't want to spend enough to do it properly.

          I probably wouldn't have been paid enough to make it worth my while either.

          • (Score: 2) by The Mighty Buzzard on Friday March 15 2019, @03:52AM (2 children)

            Depends on how important the server was to their business. Or at least how important they thought it was. You can occasionally get a small business that has a genuine need for a server and knows it but either quality is somewhat rare. Most can get by just fine with like half a dozen low-end-workstation-spec boxes or less but the ones that can't will find that out sooner or later.

            --
            My rights don't end where your fear begins.
    • (Score: 3, Insightful) by The Mighty Buzzard on Friday March 15 2019, @01:12AM (1 child)

      Like I said, if it's important data that's only on one drive, you failed right from the start. RAID and two sets of backups should be the bare minimum for most anything.

      --
      My rights don't end where your fear begins.
      • (Score: 3, Interesting) by AthanasiusKircher on Friday March 15 2019, @02:03AM

        by AthanasiusKircher (5291) on Friday March 15 2019, @02:03AM (#814595) Journal

        Yes, and if you actually care about data integrity in RAID and in those backups, you probably want ECC RAM and a filesystem that can detect and correct random errors and bitrot, like ZFS for example.

        Once we're talking about data the size of 16 TB, data degradation is likely to happen over time statistically, even due to random bit flips.

    • (Score: 3, Informative) by darkfeline on Friday March 15 2019, @03:48AM

      by darkfeline (1030) on Friday March 15 2019, @03:48AM (#814649) Homepage

      If you're talking about personal machines, restoring 500 GB to a 16 TB drive isn't any slower than restoring 500 GB to a 1 TB drive. If the 16 TB isn't too expensive it would definitely be a good option for data hoarders.

      If you're talking about servers, it's all in the cloud and datacenters. No one cares if a drive goes down. Hell, regular disaster testing involves the simulated equivalent of knocking out random rows of racks of servers or entire datacenters and watching the entire system gracefully work around that. If you're one of those wondering what all this newfangled containers and Kubernetes and cloud fad is all about, THAT is what it's all about. Losing a 1 TB drive here, losing a few PB datacenter there, no big deal, the system shifts the containers and storage around, someone's going to be working harder to restore the lost redundancy, but otherwise business as usual.

      --
      Join the SDF Public Access UNIX System today!
  • (Score: 1, Insightful) by Anonymous Coward on Friday March 15 2019, @12:49AM (4 children)

    by Anonymous Coward on Friday March 15 2019, @12:49AM (#814567)

    This is all assuming that rebuild failures aren't a thing, and that the odds of them don't increase astronomically as the capacity of your drives increases.

    • (Score: 2) by The Mighty Buzzard on Friday March 15 2019, @01:10AM (3 children)

      There are viable redundancies to eliminate most any worries available for every level of complexity you can name. Not using them is bloody stupid.

      --
      My rights don't end where your fear begins.
      • (Score: 1, Insightful) by Anonymous Coward on Friday March 15 2019, @01:17AM (2 children)

        by Anonymous Coward on Friday March 15 2019, @01:17AM (#814581)

        There are viable redundancies to eliminate most any worries available for every level of complexity you can name. Not using them is bloody stupid.

        Unless of course you've already crunched the numbers and figured out running 2x8TB drives provides less risk of drive failure and subsequent downtime for your particular application. Redundancy is only as good as the time it takes to get your data back online, I can think of plenty of businesses where lower risk of drive failure and reliability is more important than drive density. I find it alarming that you think such a scenario doesn't exist.

        • (Score: 2) by The Mighty Buzzard on Friday March 15 2019, @04:01AM (1 child)

          There is no properly designed situation where losing a drive of any size is a significant annoyance. If you find yourself in such a situation, you built your shit wrong. Now most people are not going to want to buy the largest drives because they're pricey as all fuck, they want something about one or two steps down for the sweet spot on price/dollar. That is for people who need to worry about up front costs though, not those who have the luxury of looking at lifetime costs. And, let's face it, if you're worried about up front costs, you're going to put your shit on someone else's hardware anyway.

          --
          My rights don't end where your fear begins.
          • (Score: 1, Insightful) by Anonymous Coward on Friday March 15 2019, @02:12PM

            by Anonymous Coward on Friday March 15 2019, @02:12PM (#814763)

            There is no properly designed situation where losing a drive of any size is a significant annoyance.

            You are so wrong that it hurts and I really don't understand why you keep doubling down on such a stupid position. Even with perfect redundancy, losing a drive means decreased read/write speeds. There are several, very valid reasons why people who actually host large amounts of data would choose lower capacity drives; they are outlined in the article if you bothered to read it.

  • (Score: 3, Interesting) by Farkus888 on Friday March 15 2019, @02:57AM

    by Farkus888 (5159) on Friday March 15 2019, @02:57AM (#814616)

    The problem is raid rebuild times. There is a relationship between bus speed and size. SATA II had this problem at the end. Drives so slow but big you'd get additional failures and lose the array before a rebuild could complete. After all once one drive from a lot fails the others are likely close behind. Without doing the math I believe 16tb would be over that line on SATA III. As for NVMe, I doubt the size limit is that low. I particularly doubt it in an array type that supports 2 or more disks out safely. To be clear I follow tradition and did not RTFA.

  • (Score: 3, Funny) by DannyB on Friday March 15 2019, @03:46PM

    by DannyB (5839) Subscriber Badge on Friday March 15 2019, @03:46PM (#814825) Journal

    you have no business making tech decisions in the first place.

    Some people (not me thankfully) have PHBs to make tech decisions for them.

    --
    People today are educated enough to repeat what they are taught but not to question what they are taught.