SoylentNews

What is ZFS? Why are People Crazy About it?

posted by CoolHand on Tuesday September 11 2018, @03:29PM   
from the zed-eff-ess-or-zee-eff-ess dept.

canopic jug writes:

John Paul Wohlscheid over at It's FOSS takes a look at the ZFS file system and its capabilities. He mainly covers OpenZFS which is the fork made since Oracle bought and shut down Solaris which was the original host of ZFS. It features pooled storage with RAID-like capabilities, copy-on-write with snapshots, data integrity verification and automatic repair, and it can handle files up to 16 exabytes in size, with file systems of up to 256 quadrillion zettabytes in size should you have enough electricity to pull that off. Because it started development under a deliberately incompatible license, ZFS cannot be directly integrated in Linux. However, several distros work around that and provide packages for it. It has been ported to FreeBSD since 2008.

---

Original Submission

• ZFS adventurism (Score: 2) by hendrikboom on Tuesday September 11 2018, @09:33PM

  by hendrikboom (1125) on Tuesday September 11 2018, @09:33PM (#733331) Homepage Journal

  I found the following:

  https://www.reddit.com/r/DataHoarder/comments/5u3385/linus_tech_tips_unboxes_1_pb_of_seagate/ddrngar/ [reddit.com]

  Some well meaning people years ago thought that they could be helpful by making a rule of thumb for the amount of RAM needed for good write performance with data deduplication. While it worked for them, it was wrong. Some people then started thinking that it applied to ZFS in general. ZFS' ARC being reported as used memory rather than cached memory reinforced the idea that ZFS needed plenty of memory when in fact it was just used in an evict-able cache. The OpenZFS developers have been playing whack a mole with that advice ever since.

  I am what I will call a second generation ZFS developer because I was never at Sun and I postdate the death of OpenSolaris. The first generation crowd could probably fill you in on more details than I could with my take on how it started. You will not find any of the OpenZFS developers spreading the idea that ZFS needs an inordinate amount of RAM though. I am certain of that.

  And also https://www.reddit.com/r/DataHoarder/comments/5u3385/linus_tech_tips_unboxes_1_pb_of_seagate/ddrh5iv/ [reddit.com]

  A system with 1 GB of RAM would not have much trouble with a pool that contains 1 exabyte of storage, much less a petabyte or a terabyte. The data is stored on disk, not in RAM with the exception of cache. That just keeps an extra copy around and is evicted as needed.

  The only time when more RAM might be needed is when you are turn on data deduplication. That causes 3 disk seeks for each DDT miss when writing to disk and tends to slow things down unless there is enough cache for the DDT to avoid extra disk seeks. The system will still work without more RAM. It is just that the deduplication code will slow down writes when enabled. That 1GB of RAM per 1TB data stored "rule" is nonsense though. The number is a function of multiple variables, not a constant.

  So I now wonder what the *real* limits are on home-scale systems. In particular, suppose I have only a few terabytes. And a machine with only a half gigabyte of RAM. And used for nothing more bandwidth-intensive than streaming (compressed) video over a network to a laptop.

  What I like about ZFS is its extreme resistance to data corruption. That's essential for long-term storage. My alternative seems to be btrfs. Currently I'm using ext4 on software-mirrored RAID, which isn't great at detecting data corruption.

  -- hendrik

  Parent

  Starting Score:	1		point
  Karma-Bonus Modifier		+1
  Total Score:		2