The exFAT filesystem is coming to Linux:
When software and operating system giant Microsoft announced its support for inclusion of the exFAT filesystem directly into the Linux kernel back in August, it didn't get a ton of press coverage. But filesystem vendor Paragon Software clearly noticed this month's merge of the Microsoft-approved, largely Samsung-authored version of exFAT into the VFS for-next repository, which will in turn merge into Linux 5.7—and Paragon doesn't seem happy about it.
Yesterday, Paragon issued a press release about European gateway-modem vendor Sagemcom adopting its version of exFAT into an upcoming series of Linux-based routers. Unfortunately, it chose to preface the announcement with a stream of FUD (Fear, Uncertainty, and Doubt) that wouldn't have looked out of place on Steve Ballmer's letterhead in the 1990s.
Paragon described its arguments against open source software—which appeared directly in my inbox—as an "article (available for publication in any form) explaining why the open source model didn't work in 3 cases."
All three of Paragon's offered cases were curious examples, at best.
Case one: Android
Case two: MacOS
Case three: SMB
We congratulate Paragon on closing their timely exFAT deal with Sagemcom. Although there's good reason to believe that the Samsung-derived and Microsoft-approved exFAT implementation in Linux 5.7 will be secure, stable, and highly performant, it's not here yet—and it isn't even in the next upcoming Linux kernel, 5.6, which we expect to hit general availability in late April or early May.
(Score: 4, Informative) by bzipitidoo on Thursday March 26 2020, @01:49PM (7 children)
A long time ago, when btrfs was new, I checked file system performance. FAT stood out for being much slower than all the others. xfs, btrfs, ext2/3/4. all were within a few percent of one another. Except FAT, which was half the speed of the rest of the pack.
Each file system had a bad corner case. btrfs was poor at sync, and Firefox likes to sync frequently. I have read that the btrfs maintainers were aware of the problem and improved btrfs' performance on that operation. xfs can be very slow at deleting large directory trees, such as the source of a Linux kernel. That problem was greatly mitigated by tuning the sizes of the file system structures to match the hardware. The worst case I saw was the default sizes being the worst possible sizes for the hardware. Took 5 minutes (!) to delete the tree. I reformatted that drive with much better size settings, and that helped a lot, but it still took 15 seconds. I'm not used to seeing "rm -rf linux-4.0.0" go out to lunch like that. The xfs maintainers worked on the formatting utility to make it more intelligent about picking good sizes.
The main issue with the ext's is that they're a little wasteful of space. Expect to have as much as 10% less capacity. Worst case is to have a file system with thousands of tiny files of about 100 bytes each. ext2 has to allocate a minimum of 4k (I think that's still its default block size) even for a file that is just 1 byte. So the waste can be far worse than 10%. Contrive it right, and you can make ext2/3/4 waste over 90% of the available space. FAT has the same problem, worse. To stretch FAT's maximum capacity, have to make the block sizes as big as possible. Originally, FAT used 0.5k units. As hard drives grew in size, those had to get much larger to avoid running out of block numbers. Instead of wasting just 4k, you might waste 64k per tiny file.
FAT also stinks at data handling safety. If interrupted while writing, it can leave the disk so badly corrupted that data recovery might not be possible. It usually isn't that bad, but it can happen.
(Score: 4, Informative) by Bot on Thursday March 26 2020, @01:56PM (6 children)
wait, exfat!=fat. GNU/Linux distros have some I guess fuse-based implementation, stable, in my experience.
IIRC, exfat is not totally patent/IP unencumbered, so why paragon should object to it? conflict of interest? reverse psychology to actually help exfat?
Account abandoned.
(Score: 4, Informative) by Grishnakh on Thursday March 26 2020, @01:59PM (3 children)
Paragon objects to it because they sell a proprietary exFAT filesystem driver. Who's going to buy that when the kernel supports exFAT natively with a built-in driver?
(Score: 1, Funny) by Anonymous Coward on Thursday March 26 2020, @04:43PM (1 child)
apt search exfat
Sorting... Done
Full Text Search... Done
exfat-fuse/stable,now 1.3.0-1 amd64 [installed]
Paragon is ex-fucked anyway
(Score: 2) by loonycyborg on Thursday March 26 2020, @06:43PM
Kernel module provides a lot better performance than fuse implementation.Paragon have own kernel driver, and now kernel will acquire own builtin one which will be as fast as kernel hackers can make it. Paragon's one would become redundant then.
(Score: 0) by Anonymous Coward on Thursday March 26 2020, @11:35PM
Fuck 'em.
(Score: 0) by Anonymous Coward on Thursday March 26 2020, @02:10PM
they complain because they sell a exfat support for linux... with it being included in the kernel, that market will totally dry up (unless you are either stupid or using older kernels that will not have that support, , like in Sagemcom case)
(Score: 3, Informative) by bzipitidoo on Thursday March 26 2020, @02:14PM
True, but exFAT has a FAT gut.
(Score: 5, Interesting) by Runaway1956 on Thursday March 26 2020, @02:11PM
I pretty much have to use a file system that MS recognizes, if I want to move files among machines that use MS OS. I mean, you just don't have much choice, right?
But, I've found that NTFS is pretty robust. Someone grabs it and pulls it out without ejecting or dismounting it - NTFS generally comes back perfectly healthy after chkdsk. FAT file systems? My luck hasn't been so good. File system gets mounted read-only, chkdsk claims to have fixed the problem but you still have to select folders and change it to read/write. Worse, despite telling Windows to apply the changes to this folder and all subfolders, you still have to check properties of all subfolders.
I'll pass on all the FATs, thank you very much.
“I have become friends with many school shooters” - Tampon Tim Walz
(Score: 2) by DannyB on Thursday March 26 2020, @02:23PM (3 children)
Think of exFAT as "the rosetta stone" of file systems. (Not the language translation, but the actual stone.)
It is a common "language" between platforms. If you have an external USB stick/drive formatted exFAT you can probably plug it into any computer and expect to be able to use it. Although these days that is probably true of NTFS also.
For certain use cases this can outweigh performance considerations.
Poverty exists not because we cannot feed the poor, but because we cannot satisfy the rich.
(Score: 3, Interesting) by progo on Thursday March 26 2020, @02:39PM (1 child)
Also ExFAT is the best way I can think of to share data on physical media between separate Linux systems with different user IDs. ExFAT doesn't support ownership metadata, so when you plug it into a new system, any user IDs that created files, that don't match your current user ID, are irrelevant because the whole filesystem is owned CURRENT user.
Why can't you disable ownership in ext4?
(Score: 2, Interesting) by Bot on Thursday March 26 2020, @09:58PM
I remember having made a tar directly to the raw device file (using dd but maybe output redirection is sufficient) and extracting with --no-same-owner. Note it will erase any other file in the device. Maybe you can pass dd parameters to append other tars if you recall how many records the first tar ate up.
Did that because the device was ext4 and the target pc recognized up to ext3 and I saw no reason to spend time reformatting. I had to reformat specifying sector size later, though, because grub had barfed, probably related to the crap found in the first sectors.
Earlier debian and mx didn't complain and automounted unpartitioned drives too (that is mkfs and mount the dev file), not anymore.
Account abandoned.
(Score: 2) by turgid on Friday March 27 2020, @06:40PM
I format mine as ext4 and if someone can't read it I advise them to invest in a better computer.
I refuse to engage in a battle of wits with an unarmed opponent [wikipedia.org].
(Score: 3, Interesting) by progo on Thursday March 26 2020, @02:36PM (11 children)
Isn't ExFAT a non-free specification, encumbered by patents? Is that allowed in the Linux Kernel?
Is it okay because Microsoft, the owner of ExFAT, is pushing it, and because they use Linux in Windows and Azure is full of Linux VMs?
Why isn't any of the press coverage explaining the patent angle?
(Score: 2, Informative) by SvenErik on Thursday March 26 2020, @03:01PM (6 children)
Microsoft released the patents [wikipedia.org] to the Open Invention Network [wikipedia.org] in 2019.
"Every demand is a prison, and wisdom is only free when it asks nothing." Sir Bertrand Russell
(Score: 4, Interesting) by canopic jug on Thursday March 26 2020, @04:02PM (5 children)
Microsoft released the patents to the Open Invention Network in 2019.
[citation needed]
Please provide a site which authoritatively and conclusively shows that M$ has (past tense) released the necessary specifications and code under a royalty-free license.
That Wikipedia link goes to Phoronix, which while a very reliable site, writes only of a possible future event in which says only that M$ would probably release the patents at some future date. It makes no forward reference to that event having happened at all. One M$ engineer quoted there, alone, that he hopes that M$ will one day do that is not a commitment from the beast. It'd be a quick way to shut down all the distros by getting ExFAT everywhere and then demanding payment and or cross-licensing. It has been what they have been threatening quietly with Mono for years.
Money is not free speech. Elections should not be auctions.
(Score: 4, Insightful) by nitehawk214 on Thursday March 26 2020, @04:09PM
My guess is MS will make their implementation slightly incompatible with the free one. Thus making Linux systems using exFAT look like the bad guys for not providing a fully compatible filesystem.
Its like the reverse of embrace-extend-extinguish.
"Don't you ever miss the days when you used to be nostalgic?" -Loiosh
(Score: 1, Interesting) by Anonymous Coward on Thursday March 26 2020, @10:52PM (3 children)
https://cloudblogs.microsoft.com/opensource/2019/08/28/exfat-linux-kernel/ [microsoft.com]
https://lore.kernel.org/lkml/20190828160817.6250-1-gregkh@linuxfoundation.org/ [kernel.org]
(Score: 2) by canopic jug on Friday March 27 2020, @05:55AM (2 children)
https://cloudblogs.microsoft.com/opensource/2019/08/28/exfat-linux-kernel/ [microsoft.com] https://lore.kernel.org/lkml/20190828160817.6250-1-gregkh@linuxfoundation.org/ [kernel.org]
Thanks, those are interesting and are a clear warning. Both links are so close to showing something, anything, about the patents, but actually don't. It appears very much that M$ GregKH signed off on the patch from Valdis Klētnieks without verifying the patent licensing. The patch seems to be GPL 2.0 and that is a good thing. It also means that the patent situation is addressed unless done so separately, something which the patch has not done.
The first link, a blog, does not even mention patents. Both link onward to another page entitled, "exFAT file system specification", but that page links to neither the code nor mentions patents. It does have one line vaguely mentioning some unspecified changes to the licensing. However, it is a dead-end and does not link to any M$ code anywhere.
In addition to the distros, it sets up Android for a big fall. It'll be using that tainted kernel soon. Assuming that M$ would go after the manufacturers instead of the individual users, that would make expensive trouble for Samsung and the other phone makers, but not Google/Alphabet. They'd just use an attack from M$ over the patents as an excuse to drop Android lock a hot potato and roll out Fuchsia as a full replacement.
Money is not free speech. Elections should not be auctions.
(Score: 0) by Anonymous Coward on Friday March 27 2020, @08:12PM
That commit was signed off by "Sasha Levin" . As an agent of Microsoft with apparent authority to authorize commits to the Linux tree coupled with Microsoft having agreed to the OIN License, granted access under provisions 1.2 of the agreement, among others, to all other licensees of the OIN by allowing such a commit to the Linux kernel under the GPL 2 without objection.
(Score: 0) by Anonymous Coward on Saturday March 28 2020, @01:06PM
>They'd just use an attack from M$ over the patents as an excuse to drop Android lock a hot potato and roll out Fuchsia as a full replacement.
Actually, google stopping to churn out versions would be a good thing for Android...
(Score: 2) by canopic jug on Thursday March 26 2020, @04:12PM (3 children)
Isn't ExFAT a non-free specification, encumbered by patents? Is that allowed in the Linux Kernel?
It is now, with Linus out of the way and M$ Greg nominally in charge. Linus needs to pack up his trademarks, for the code, and move on like he had to do with OSDL.
Money is not free speech. Elections should not be auctions.
(Score: 0, Informative) by Anonymous Coward on Thursday March 26 2020, @05:21PM (2 children)
Linus himself is no RMS. He's already said "MS hate is a disease".
https://www.neowin.net/forum/topic/801240-linus-torvalds-microsoft-hatred-is-a-disease/ [neowin.net]
(Score: 4, Touché) by Bot on Thursday March 26 2020, @10:05PM (1 child)
Except rms is always right, when speaking about software freedom, Linus already got burned once, see bitkeeper.
Account abandoned.
(Score: 0) by Anonymous Coward on Saturday March 28 2020, @07:02PM
i wasn't agreeing with Linus. I agree with RMS too.
(Score: 2) by PinkyGigglebrain on Friday March 27 2020, @02:54AM (1 child)
I reformat most of my large USB flash drives as UDF. only exception is the microSD that I use in my phone and the older CFs my SLR camera uses.
It's supported by all the major OSs, supports long file names and files over 4GB, and is open.
works great for my uses but YMMV depending on your needs.
http://duncanlock.net/blog/2013/05/13/using-udf-as-an-improved-filesystem-for-usb-flash-drives/ [duncanlock.net]
"Beware those who would deny you Knowledge, For in their hearts they dream themselves your Master."
(Score: 2) by PinkyGigglebrain on Friday March 27 2020, @02:56AM
Forgot to mention that my Linux boxes uses ext3-4 and my single Windwos box, only used for 1 game, uses ntfs of course.
"Beware those who would deny you Knowledge, For in their hearts they dream themselves your Master."