SoylentNews is people
SoylentNews
A blog has a walkthrough of using ZFS encryption on Linux:
In order to have a simple way to play with the new features of ZFS, it makes sense to have a safe "sandbox". You can pick an old computer, but in my case I decide to use a VM. It is tempting to use docker, but it won't work because we need a special kernel module to be able to use the zfs tools.
For the setup, I've decide to use VirtualBox and Archlinux, since those are a few tools that I'm more familiar with. And modifying the zfs-dkms package to build from the branch that hosts the encryption PR is really simple.
[...] Finally we are able to enjoy encryption in zfs natively in linux. This is a feature that was long due. The good thing is that this new implementation improved a few of the problems that the original one had, especially around key management. It is not binary compatible, which is fine in most cases and still not ready to be used in production, but so far I really like what I see.
If you want to follow progress, you can watch the current PR in the official git repo of the project. If everything keeps going ok, I would hope this feature to land in version 0.7.1
(Score: 2) by jasassin on Sunday June 25 2017, @05:42AM (21 children)
Modded you up. I was going to ask the same question. ZFS eats RAM like a mofo, and a virtualzed system running ZFS WTF? You'd need a system with 64GB at least. The capabilities of ZFS are astounding. The ammount of resources it requires is also astounding.
How much RAM does the host have, and how much did you allocate to the VM? I roll a 20 sided die in an attempt to disbelieve this bullshit.
jasassin@gmail.com GPG Key ID: 0xE6462C68A9A3DB5A
(Score: 2) by kaszz on Sunday June 25 2017, @05:57AM (5 children)
Only the RAM allocated to the VM needs to be known. The virtualization overhead in terms of RAM isn't likely to be large. Actually ZFS shall be usable with 4 GB, and workable at 8 GB. So no 64 GB needed, but more is better (until any non-ECC ram gets a alpha emission.. flip).
(Score: 2) by jasassin on Sunday June 25 2017, @08:30AM (4 children)
Disputable.
It's an entire OS running in VirtualBox so it will use 1-2GB depending on the OS (not sure of how much RAM Arch Linux uses after booting a default install).
This is interesting. I'm not very familiar with ECC RAM but I seem to remember reading somewhere if a bit flips the system detects it and freezes anyway. The only difference is the flipped bit it detected forcing a system halt. (Been awhile since I read about ECC RAM.)
jasassin@gmail.com GPG Key ID: 0xE6462C68A9A3DB5A
(Score: 3, Informative) by kaszz on Sunday June 25 2017, @09:44AM
From the point of the operating system inside a VM. It's running a on real hardware. So even the "RAM" it uses may be swap in reality. Thus only the space allocated as "RAM" really matters.
With overhead, I meant the memory overhead for the OS running on bare metal that hosts a VM.
ECC can correct some bit errors and on the rest it will throw an error. The action taken depends. The general rule seems to be 1 bit error per gigabyte per hour.
(Score: 5, Informative) by zeigerpuppy on Sunday June 25 2017, @03:46PM (2 children)
ZFS is great. But beware ECC RAM is not optional, it is an absolute requirement for ZFS data consistency. See here for why: https://pthree.org/2013/12/10/zfs-administration-appendix-c-why-you-should-use-ecc-ram/ [pthree.org] .
Running ZFS in a VM is a good test but you get no benefit from doing this in production.
ZFS can only protect your data if it has direct access to the drive controller at hardware level. Then feel free to layer whatever you like on top of it. I run many Xen VMs using sparse zvols with ext4 format on top, works great.
Encryption is a feature that's long awaited in zfsonlinux, interesting to see it's close...
(Score: 2) by WillR on Monday June 26 2017, @04:20PM (1 child)
Using ECC on anything with parity RAID-like storage should be considered mandatory. There's nothing especially bad RAM sensitive about ZFS, the same stuck-bit scenario can trash data on RAID 5 just as easily. ZFS just gets shit on much more frequently in the blogosphere about it, for reasons that aren't clear to me.
(Score: 2) by zeigerpuppy on Tuesday June 27 2017, @09:50AM
a good point, parity checking is the main issue. the other thing that makes ZFS a little different is the scrub operation. random bit flips in RAM that are uncaught could actually corrupt good data on disk during a scrub.
I think most of the critiques are from people used to EXT or NFS file systems who want the ZFS special source but aren't willing to familarise themselves with its quirks!
At least ECC is cheap these days...
(Score: 5, Informative) by TheRaven on Sunday June 25 2017, @11:03AM (2 children)
sudo mod me up
(Score: 1, Interesting) by Anonymous Coward on Sunday June 25 2017, @05:10PM (1 child)
> ZFS needs about on 1GB of RAM per TB of disk
Only if you're using dedup (and possibly, RAIDZ) -- on Linux, it uses less RAM than the Solaris implementation.
(Score: 2) by fnj on Monday June 26 2017, @05:26AM
Incorrect. 1GB per TB is the minimum for comfort whether you have RAID or not. 5GB per TB at a minimum is the guideline if you use dedup.
You can make it work with less, but performance will suffer to an excruciating degree.
(Score: 1, Interesting) by Anonymous Coward on Sunday June 25 2017, @05:06PM (1 child)
You think 64GB on the host is required just for ZFS and encryption?? What kind of crack are you on??
Do your own tests. Leave deduplication OFF and run Linux+ZFS zRAID10 with compression=lz4 + the new encryption and let us know how that works out.
(Score: 2) by jasassin on Sunday June 25 2017, @06:32PM
My last test was a few years ago using FreeBSD on bare metal with 2GB of RAM. The performance was so terrible after a day or two I switched back to Linux with the extended filesystem. Before I switched back I went into the FreeBSD channel and asked about the performance I was experiencing, and when they heard I was trying to use ZFS on a system with 2GB of RAM I was basically laughed out of the IRC channel. They didn't even bother mentioning deduplication, I'll have to research that.
P.S. I don't smoke crack.
jasassin@gmail.com GPG Key ID: 0xE6462C68A9A3DB5A
(Score: 2) by LoRdTAW on Sunday June 25 2017, @09:19PM (9 children)
FreeNAS uses ZFS for storage and recommends at least 8GB. But the general memory recommendation is something like 4GB for the OS and then add 1GB per terabyte of ZFS storage. So the memory requirement is based on the array size. A smaller test array could easily run with 4-8 GB which is not hard to come by.
(Score: 3, Informative) by rleigh on Sunday June 25 2017, @10:18PM (8 children)
1 GB RAM / TB storage is only for the deduplication tables. Don't enable dedup and the requirements are far more modest. Some caches get enabled with more than 4GB RAM, but ZFS itself will run fine with less.
(Score: 2) by kaszz on Monday June 26 2017, @01:07AM (1 child)
How much RAM is needed for non-de-duplication ZFS ?
(per TB etc)
(Score: 2) by rleigh on Monday June 26 2017, @10:33AM
I'd suggest looking at the books I posted in my other reply, and looking at the existing guidance out there on the net. 4GB is a reasonable minimum; file-level prefetching is disabled on systems with less memory, but I have as little as 2GB in some of my virtual machines. The more you have, the bigger its caches can be, and you can instrument ZFS to determine what the cache utilisation is for your system, and if it's wasted or too small. None of the documentation or books I've read to date have suggested a certain amount of memory per TB storage is a requirement, unless you're using deduplication. There's quite a lot of advice and guidance suggesting it, but I have not seen a technical rationale to back it up (if there is one, I'd be interested to see it).
(Score: 2) by fnj on Monday June 26 2017, @05:33AM (5 children)
Incorrect. 1 GB per TB is the minimum for use without dedup. If you have dedup enabled, you need at least 5 GB per TB. But it's not a straight line. If you only have 0.1 TB, it's NOT going to work with 0.1 GB. Basically you need 4GB to get ZFS off the ground; then you add 1 GB per TB on top of that. It won't even work acceptably at all on ia32 because of virtual memory mapping limitations. It's got to be 64 bit.
It doesn't do anybody any good to spread false information.
(Score: 2) by rleigh on Monday June 26 2017, @09:24AM (4 children)
Umm... it's not false information. I've just finished reading two ZFS books ("FreeBSD Mastery: ZFS" and "FreeBSD Mastery: Advanced ZFS"; also applicable to other operating systems BTW for anyone wanting to know more about ZFS), and I've also read quite widely about the memory requirements before that, as well as using it for several years myself. They are all quite clear that it's quite possible to run ZFS on low memory systems; I run FreeBSD on ZFS on low memory virtual machines at work without any problems whatsoever.
The large memory requirements are a myth which continues to propagate like some bizarre meme. Deduplication requires large amounts of memory for its in-memory copy of the on-disc deduplication tables; that scales linearly with disc size (well, total allocated unique blocks) and leads to the aforementioned rule. This does not at all apply to the ARC and related in memory caches, which can be dramatically shrunk, and you can additionally work on performance tuning on individual datasets and zvols to adjust the block prefetching, commit intervals and other parameters.
It's true that extreme low memory usage requires sacrificing some performance, and that some optimisations are only enabled when a certain amount of memory is available, but this does not mean it's unusable. The cache sizes are not really related to the /quantity/ of storage, but the data access patterns you require for your workload, and this can vary wildly depending upon the specific cases. ZFS can be adjusted for throughput or interactive response on a per-dataset basis, and this also affects the amount of cache required. Caching can also be completely disabled for datasets which won't benefit from it.
I have no idea where the 32-bit comments came from because I didn't refer to it at all.
(Score: 2) by LoRdTAW on Monday June 26 2017, @12:48PM (3 children)
You know your file system is massively over complicated when you need to read two fucking books on how to correctly use it.
(Score: 2) by rleigh on Monday June 26 2017, @04:43PM (2 children)
Ha ha. It's not quite that bad. I'd been using it for 2½ years before getting those books. They have some useful guidance and best practices, and I picked up on some neat tools I hadn't used before, but they are not essential reading unless you want to manage a humongous storage array with hundreds of discs, where the more advanced disc management stuff would come into its own.
ZFS isn't just a file system though; it's doing RAID, volume management, block devices, replication, delegated administration and you can do all sorts of configuration and performance tuning of individual datasets. There's enough functionality that a couple of small books are a worthwhile addition to your library if you want to use it properly. I think it's justifiable; if you wanted to use RAID+LVM+xfs/ext4 there would be a good amount of complexity in setting up and ongoing maintenance of that as well.
(Score: 2) by fnj on Monday June 26 2017, @08:58PM (1 child)
You don't have a single goddam clue what you are talking about.
(Score: 2) by rleigh on Monday June 26 2017, @11:00PM
Then maybe you would care to point out exactly what I said which was wrong, with some references to the documentation, please?