Kernel Integrated exFAT Support Performance has saved my Chia farmer

published:

In case you did not hear, great exFAT support has been integrated in the Linux Kernel 5.7 quite some time ago. Unfortunately, that Kernel is not the default kernel for all Linux Distributions, namely Ubuntu. In this article we will see how drastic the improvements compared to the FUSE implementation are, and how to upgrade Ubuntu 20.04 to use the Kernel integrated exFAT drivers.

The result on my Chia farm is actually spectacular. Previously I have missed many challenges and the system was partially slow. Upgrading to the latest Kernel with integrated exFAT support, this changed completely. 

Before and after migrating to kernel integrated exFAT. Yes, Seconds!

Context

There have been multiple ways how to operate a exFAT filesystem with Linux. The most popular might still be the first one, as it is the default method in many major distros

I was using the "default" way, which has lead to quite some performance problems. A single drive has used up to 10% of my CPU core, just to provide read access and very low throttled write load (30MB/s). Having 4 USB drives connected to my mini PC, that meant 40% CPU usage, with little room for other CPU hungry services like SMB. After the migration, this dropped to below 10% for all 4 drives!

Interestingly is the availability of a backport of the Kernel integrated module. Their website list an interesting benchmark, which matches my experience

random IO on a NVMe

The second row is the Kernel Integrated version (5.7) compared to the backport (Bold). You see that the speed improvements are huge and almost on par with ext4. More benchmarks are available on their GitHub Repo

Some other real-life workloads, are also very positively affected, as this user mentions on Reddit: Anecdotal: Big performance difference in exfat-fuse vs exfat-nonfuse : archlinux (reddit.com)

How to Benefit?

Assuming that your running your extFat devices with Ubuntu 20.04, there are only a few steps required to bring the performance improvements to you. 

Install the newest Kernel

You can find multiple options for upgrading your kernel on Install or Upgrade To Latest Linux Kernel on Ubuntu 20.04 & Linux Mint 20 (linuxhint.com). For me, the following worked like a charm:

Steo 1: First you might check what is your current Kernel version:

mis@box:~$ uname -r 5.4.0-56-generic

This indicates that you're running kernel version 5.4

Step 2: The next steps is to download a script maintained by Roy to automatically download and install a new kernel. Don't forget to make the file executable and move it to the /usr/local/bin folder

mis@box:~$ wget https://raw.githubusercontent.com/pimlie/ubuntu-mainline-kernel.sh/master/ubuntu-mainline-kernel.sh mis@box:~$ chmod +x ubuntu-mainline-kernel.sh mis@box:~$ sudo mv ubuntu-mainline-kernel.sh /usr/local/bin/

This will make the file executable and place it in the right folder

Step 3: Start the installation. This is as simple as executing the below command with root permissions

mis@box:~$ sudo ubuntu-mainline-kernel.sh –i

This will prompt for a confirmation during the installation, but runs fully automated otherwise.

Step 4: Restart the system.

mis@box:~$ sudo restart

This will prompt for a confirmation during the installation, but runs fully automated otherwise.

Uninstall FUSE Version

Even though the new Kernel is installed, all drives mentioned in /etc/fstab will still be mounted with the FUSE driver if installed. We need to uninstall that as well.

Step 1: Unmount all drives that are currently using exFat as filesystem. Check your /etc/fstab file for that and unmount all of them with sudo umount /mnt/mointpoint

Step 2: Uninstall the old FUSE drivers. sudo apt remove exfat-fuse exfat-utils

Step 3: Remount all drives. This time the Kernel Integrated driver should take over.  sudo mount -a
You can check your success with the top command. There shouldn't be any process related to exFat.

Install User space Utilities

This is optional if you only want to mount drives, but it contains the tools to run a filesystem check, or create a fresh filesystem on a block device. Source: exfatprogs/exfatprogs: exFAT filesystem userspace utilities (github.com)

mis@box:~$ sudo apt install exfatprogs

Done!. You will again find the tools for file-system checks and initialization as before.

Disclaimer (Error 0x8007003B)

All the FUSE and Kernel implementation lock of truncate, which is a serious problem with SAMBA as Windows clients will timeout when transferring very large files to a share on a exFAT Mount,  for instance here: Support fast space allocation? · Issue #45 · relan/exfat (github.com) but the whole internet is full of timeout-errors related to that.

So if you plan to share a exFAT mount with Samba for Windows clients... Just don't. As soon as the file reaches a certain size (probably around 16GB), the copy process fill fail.

Due to lack of truncate, the file is filled with Zeros before copying starts


After approximately 20s, you will see an error message on Windows, even while SAMBA is still allocating the space for the file.... A quick Google search unveils potentially hundreds of users that have the same problem.

Error 0x8007003B: An unexpected network error ocurred
Infamous Error 0x8007003B: An unexpected network error ocurred

More of the same error

Comments

  1. Sz TomatoNovember 3, 2022 at 3:53 PM

    A very delightful article that you have shared here about mini pc. Your blog is a valuable and engaging article for us, and also I will share it with my companions who need this info. optical audio to speaker wireThankful to you for sharing an article like this.

    ReplyDelete

Post a Comment

Popular posts from this blog

Synology Presto Review

published:
Image
I still run my NAS in Switzerland while I'm living abroad in Singapore. I frequently run backups of for my files where I leverage my Switzerland NAS as an off-side backup location. After all, keeping my NAS in Switzerland is cheaper than the costs for international shipping from Switzerland to Singapore. But it comes with two draw-backs: Average internet connection speed is still worse in Switzerland compared to other places, especially the upload speed Latency is typically around 370ms, which unfortunately has a huge impact on file transfer speeds. While the first issue can be solved with money, the second is more expensive. During some analysis on the topic, I found that Synology actually came up with an extension called Presto  for their DSM. It's based on the somewhat mysterious technology called "SITA" which should solve the issue of high latency transfers. According to their explanation on YouTube (video below), they are able to use the available bandwid
Read more

Home Assistant in Docker with Nginx and Let's Encrypt on Raspberry Pi

published:
Image
This guide will help to you to setup home assistant as a docker container and make it publicly available behind a ngix proxy, all running in containers. Additionally, we'll use certbot to keep your ssl certificates in sync and Duck DNS so that you access it even with a dynamic ip address. There we go... Dockerized Home Assistant behind Nginx with Let's Encrypt and DuckDNS :) This guide is partially based on the generic guide on how to install Ngix and Let's Encrypt by Philipp ( Nginx and Let’s Encrypt with Docker in Less Than 5 Minutes | by Philipp | Medium ) In order to start, please make sure you have the following setups done already Home Assistant installed as docker container and available under port 8123 Create a DuckDNS account and register your subdomain You know how to redirect a port from your router to the computer running docker We'll be using docker-compose, which is much simpler to use than the plain docker cli. If you are not familiar with docker-compose,
Read more

Use Bodmer TFT_eSPI Library with PlatformIO

published:
Image
If you start playing around with TFT on your ESP8266 or ESP32, you will arrive at the excellent TFT_eSPI Library by Bodmer . It's not only magnitudes faster than the Adafruit driver , it's also comes with support for a wide range of display drivers. Test results of the TFT_eSPI library Unfortunately it has one catch: The library is not really ready to be included as a library in your projects as it requires adjustments in the library itself. While it's not uncommon to define your settings with #define directives, this library expects you to patch files to configure it . The official way goes like this Locate the library folder (Windows would be Documents\Arduino\libraries\TFT_eSPI ) and create your own setup in the library sub folder User_Setups To use your configuration, patch the file User_Setup_Select.h  to include your configuration This has obviously some disadvantages. At latest when you wan't to update the library or work with different screens at the same tim
Read more