Kernel Integrated exFAT Support Performance has saved my Chia farmer
In case you did not hear, great exFAT support has been integrated in the Linux Kernel 5.7 quite some time ago. Unfortunately, that Kernel is not the default kernel for all Linux Distributions, namely Ubuntu. In this article we will see how drastic the improvements compared to the FUSE implementation are, and how to upgrade Ubuntu 20.04 to use the Kernel integrated exFAT drivers.
The result on my Chia farm is actually spectacular. Previously I have missed many challenges and the system was partially slow. Upgrading to the latest Kernel with integrated exFAT support, this changed completely.
![]() |
Before and after migrating to kernel integrated exFAT. Yes, Seconds! |
Context
There have been multiple ways how to operate a exFAT filesystem with Linux. The most popular might still be the first one, as it is the default method in many major distros
- extfat Fuse relan/exfat: Free exFAT file system implementation (github.com)
- exfatNoFuse (with dkms): dorimanx/exfat-nofuse: Android ARM Linux non-fuse read/write kernel driver for exFat and VFat Android file systems (github.com)
- Mainline Kernel (since 5.7) with extFAT support.
- Backport: arter97/exfat-linux: exFAT filesystem module for Linux kernel (github.com)
I was using the "default" way, which has lead to quite some performance problems. A single drive has used up to 10% of my CPU core, just to provide read access and very low throttled write load (30MB/s). Having 4 USB drives connected to my mini PC, that meant 40% CPU usage, with little room for other CPU hungry services like SMB. After the migration, this dropped to below 10% for all 4 drives!
Interestingly is the availability of a backport of the Kernel integrated module. Their website list an interesting benchmark, which matches my experience
![]() |
random IO on a NVMe |
The second row is the Kernel Integrated version (5.7) compared to the backport (Bold). You see that the speed improvements are huge and almost on par with ext4. More benchmarks are available on their GitHub Repo
Some other real-life workloads, are also very positively affected, as this user mentions on Reddit: Anecdotal: Big performance difference in exfat-fuse vs exfat-nonfuse : archlinux (reddit.com)
How to Benefit?
Assuming that your running your extFat devices with Ubuntu 20.04, there are only a few steps required to bring the performance improvements to you.
Install the newest Kernel
mis@box:~$ uname -r
5.4.0-56-generic
Step 2: The next steps is to download a script maintained by Roy to automatically download and install a new kernel. Don't forget to make the file executable and move it to the /usr/local/bin folder
mis@box:~$ wget https://raw.githubusercontent.com/pimlie/ubuntu-mainline-kernel.sh/master/ubuntu-mainline-kernel.sh
mis@box:~$ chmod +x ubuntu-mainline-kernel.sh
mis@box:~$ sudo mv ubuntu-mainline-kernel.sh /usr/local/bin/
Step 3: Start the installation. This is as simple as executing the below command with root permissions
mis@box:~$ sudo ubuntu-mainline-kernel.sh –i
Step 4: Restart the system.
mis@box:~$ sudo restart
Uninstall FUSE Version
Step 1: Unmount all drives that are currently using exFat as filesystem. Check your /etc/fstab file for that and unmount all of them with sudo umount /mnt/mointpoint
Step 2: Uninstall the old FUSE drivers. sudo apt remove exfat-fuse exfat-utils
Step 3: Remount all drives. This time the Kernel Integrated driver should take over. sudo mount -a
You can check your success with the top command. There shouldn't be any process related to exFat.
Install User space Utilities
mis@box:~$ sudo apt install exfatprogs
Disclaimer (Error 0x8007003B)
All the FUSE and Kernel implementation lock of truncate, which is a serious problem with SAMBA as Windows clients will timeout when transferring very large files to a share on a exFAT Mount, for instance here: Support fast space allocation? · Issue #45 · relan/exfat (github.com) but the whole internet is full of timeout-errors related to that.
So if you plan to share a exFAT mount with Samba for Windows clients... Just don't. As soon as the file reaches a certain size (probably around 16GB), the copy process fill fail.
![]() |
Due to lack of truncate, the file is filled with Zeros before copying starts |
More of the same error
- Samba on Raspi copy large files gives error 0x8007003B solution so far: don't use ExFAT or FAT32 - Raspberry Pi Forums
- [ubuntu] Shared drive over Samba, error transferring large files (ubuntuforums.org)
- [OpenWrt Wiki] SMB / Samba share overview (Windows file sharing)
- DD-WRT Forum :: View topic - Copy big file to samba shared disk may cause router dies
- Can't copy large files (RPI + SMB + HDD) - Raspberry Pi Forums
A very delightful article that you have shared here about mini pc. Your blog is a valuable and engaging article for us, and also I will share it with my companions who need this info. optical audio to speaker wireThankful to you for sharing an article like this.
ReplyDelete