Boot Raspberry Pi 4 from USB SSD without SD Card

published:
The Raspberry foundation has recently published a new beta version of their firmware for the Raspberry Pi 4 Model B, which supports booting from USB. Finally. This means no SD Card is not required anymore, the bootloader can be loaded from an USB mass storage device! This is a major improvement! 


Note: This approach only works with the official Raspberry Pi OS, either 32bit or 64bit. Ubuntu, unfortunately does not yet work, as there are changes required to uBoot, uncompressing the kernel image, etc. to support booting from the USB Drive on Raspberry Pi.
This is a step by step instruction loosely based on Jeff Gerling's I'm booting my Raspberry Pi 4 from a USB SSD and James Chambers's Raspberry Pi 4 USB boot config guide for SSD flash drives. Please note that the second article was published before USB boot was possible and thus is using a workaround of loading the bootloader from SD and the root filesystem from the SSD.

In the following tutorial, we'll make sure that 
  1. you're using the latest (beta) version of the firmware, 
  2. use the correct settings to boot from USB 
  3. and change the configuration of your bootloader so that it loads the root filesystem from USB as well.
Let's get started!

Preparation

This is what you need to get you started. Obviously you need a Raspberry Pi Model 4 as well
I had great success with the Samsung T3 Portable SSD, and many other SSD's seem to work great as well. Please checkout the compatibility list and performance benchmark maintained by James Chambers. It was also recently discovered that USB SDD's without UASP support will not be as fast as your might expect. So make sure that your model supports UASP and does not run in Bulk Only Transport (BOT).

Update Firmware

The Raspberry Pi 4 is the first model that comes with an integrated EEPROM that contains the firmware and boot configuration. We need to update that internal firmware and make sure that the configuration is set properly. More information about the firmware can be found in the officialRaspberry documentation.

Updating the firmware can be done with any existing Raspberry OS/Raspian installation. If you don't want to mess with your current system, grab a empty SD Card, flash the latest Raspberry OS, enable SSH, login with pi/raspberry and install the latest updates with sudo apt update&&sudo apt full-upgrade.

The feature was not officially announced, but it seems that it became available in the version 2020-06-15. At the time of writing, this version was not yet promoted to the stable channel, but this might have changed in the meantime.

To check if a version equal or newer then 2020-06-15 is available, we can use the tool rpi-eeprom-update. When called without any parameters, it simply prints the current versions of the firmware and checks if there are any updates available.
pi@raspberrypi:~ $ sudo rpi-eeprom-update
BCM2711 detected
VL805 firmware in bootloader EEPROM
*** UPDATE REQUIRED ***
BOOTLOADER: update required
CURRENT: Thu 19 Mar 14:27:25 UTC 2020 (1584628045)
 LATEST: Thu 16 Apr 17:11:26 UTC 2020 (1587057086)
 FW DIR: /lib/firmware/raspberrypi/bootloader/critical
VL805: up-to-date
CURRENT: 000137ad
 LATEST: 000137ad
While this is telling us that there is a newer version available, its unlikely that this newer version contains the changes for USB Boot, as they were released somewhen in June. Your output might differ, but I had to change to the BETA channel to get the newer firmware. 

To change to beta channel, edit the file /etc/default/rpi-eeprom-update and change theFIRMWARE_RELEASE_STATUS to beta

After changing the channel to beta, I was able to see the newest beta version 2020-06-15. Again, it might be that you don't need to change to the beta channel at all if you get a version >= 2020-06-15 in the screen before.
pi@raspberrypi:~ $ sudo rpi-eeprom-update
BCM2711 detected
VL805 firmware in bootloader EEPROM
*** UPDATE REQUIRED ***
BOOTLOADER: update required
CURRENT: Thu 19 Mar 14:27:25 UTC 2020 (1584628045)
 LATEST: Mon 15 Jun 13:36:19 UTC 2020 (1592228179) <-- this is the version I wanted.
 FW DIR: /lib/firmware/raspberrypi/bootloader/beta
VL805: up-to-date
CURRENT: 000137ad
 LATEST: 000137ad
Updating the firmware happens on boot. To initiate this, we have to download the new firmware and place it on the boot partition of the SD Card. This can be done with the same tool, by applying the switch -a. Please note that the system has to be powered off and on again.
pi@raspberrypi:~ $ sudo rpi-eeprom-update -a
pi@raspberrypi:~ $ sudo shutdown now
After booting up the update should be reflected as below
pi@raspberrypi:~ $ sudo rpi-eeprom-update
BCM2711 detected
VL805 firmware in bootloader EEPROM
BOOTLOADER: up-to-date
CURRENT: Mon 15 Jun 13:36:19 UTC 2020 (1592228179)
 LATEST: Mon 15 Jun 13:36:19 UTC 2020 (1592228179)
 FW DIR: /lib/firmware/raspberrypi/bootloader/beta
VL805: up-to-date
CURRENT: 000137ad
 LATEST: 000137ad
If you would like to see the exact commit hash version of the version, issue the following command
pi@raspberrypi:~ $ vcgencmd bootloader_version
Jun 15 2020 14:36:19
version c302dea096cc79f102cec12aeeb51abf392bd781 (release)
timestamp 1592228179

Change boot order configuration

Let's also check if the bootloader is configured according to it's new possibilities. We have to make sure that the configuration for the boot order reflects our desire to load from USB at all. The commandvcgencmd bootloader_config is used to display the current configuration as it is flashed in the EEPROM.
pi@raspberrypi:~ $ vcgencmd bootloader_config
[all]
BOOT_UART=0
WAKE_ON_GPIO=1
POWER_OFF_ON_HALT=0
DHCP_TIMEOUT=45000
DHCP_REQ_TIMEOUT=4000
TFTP_FILE_TIMEOUT=30000
TFTP_IP=
TFTP_PREFIX=0
BOOT_ORDER=0x41
SD_BOOT_MAX_RETRIES=3
NET_BOOT_MAX_RETRIES=5
[none]
FREEZE_VERSION=0
We're particularly interested in the value for BOOT_ORDER. It reads from left to right right to left, and each byte refers to a boot source. For example: 0x1 means SD Card, 0x4 means USB, etc. The full list of possible options are available in the official documentation.

We should have a value of 0x41, which instructs the firmware to try SD first and then USB mass storage second. If you have a different configuration, you need to change it by creating a new firmware. Sounds more complex that it actually is, just follow the steps below

1. Extract the configuration from a firmware file
As we are using the latest firmware 2020-06-15.bin, we copy the whole firmware to our home directory and extract the configuration from there. Should you use a different version, please change the name of the file accordingly.
pi@raspberrypi:~ $ cp /lib/firmware/raspberrypi/bootloader/beta/pieeprom-2020-06-15.bin pieeprom.bin
pi@raspberrypi:~ $ rpi-eeprom-config pieeprom.bin > conf.txt
2. Change the configuration
To do that, we simply edit the extracted file bootconf.txt and set the value for BOOT_ORDER to 0x41

3. Built new firmware that includes the new configuration
We have to create a new binary that contains both the firmware and the configuration. The following command creates the binary pieeprom-new.bin based on the stock fimware pieeprom.bin and the applied configuration as in bootconf.txt.
pi@raspberrypi:~ $ rpi-eeprom-config --out pieeprom-new.bin --config conf.txt pieeprom.bin
4. Apply the update and reboot
Similar to the previous update command, but this time with a customized firmware with our settings
pi@raspberrypi:~ $ sudo rpi-eeprom-update -d -f ./pieeprom-new.bin
BCM2711 detected    
VL805 firmware in bootloader EEPROM
BOOTFS /boot
*** INSTALLING ./pieeprom-new.bin ***
BOOTFS /boot
EEPROM update pending. Please reboot to apply the update.
5. Restart and check the configuration
You can use the command vcgencmd bootloader_config again to validate if the changes got applied.

Create compatible Raspberry OS

Your Raspberry Pi is ready for our USB SSD, but we have to prepare the OS on that USB SSD as well. As a first step, flash the Raspberry OS to that device using the normal imager as you have done before.

After flashing the image to your SSD, unplug and attach it again to your computer, so that you can access the boot partition. We'll have to update some of the files there, as bootloader in the image does not support USB boot. Head over to the GitHub Repository that contains the necessary files in the boot-folder. You can either download the a copy of the repository via git checkout or download the master version as Zip file

Unpack the contents of the ZIP file copy the following files in the folder boot to the boot partition of your USB hard drive.
  • Everything that ends with .dat (8 files)
  • Everything that ends with .elf (8 files)
Please make sure that you only copy the files listed above. Overwriting any other file could lead in failures.
Files that need to be updated

         
Replace existing files in target

One last thing: Don't forget to add the ssh file if you're running the Raspberry Pin headless 🤣

Comments

  1. AnonymousDecember 31, 2020 at 7:14 AM

    BOOT_ORDER reads from far right to the left.

    ReplyDelete
    Replies
    1. Michael SchnyderJanuary 25, 2021 at 11:39 AM

      Thanks for the feedback, you're right. I change the section accordingly as per documentation "The bootmodes are attempted in lowest significant nibble to highest significant nibble order."

      Delete
  2. BroodleeApril 29, 2021 at 7:22 PM

    Awesome

    ReplyDelete
  3. @itx_FahadDecember 1, 2022 at 9:09 PM

    This comment has been removed by a blog administrator.

    ReplyDelete

Post a Comment

Popular posts from this blog

Home Assistant in Docker with Nginx and Let's Encrypt on Raspberry Pi

published:
Image
This guide will help to you to setup home assistant as a docker container and make it publicly available behind a ngix proxy, all running in containers. Additionally, we'll use certbot to keep your ssl certificates in sync and Duck DNS so that you access it even with a dynamic ip address. There we go... Dockerized Home Assistant behind Nginx with Let's Encrypt and DuckDNS :) This guide is partially based on the generic guide on how to install Ngix and Let's Encrypt by Philipp ( Nginx and Let’s Encrypt with Docker in Less Than 5 Minutes | by Philipp | Medium ) In order to start, please make sure you have the following setups done already Home Assistant installed as docker container and available under port 8123 Create a DuckDNS account and register your subdomain You know how to redirect a port from your router to the computer running docker We'll be using docker-compose, which is much simpler to use than the plain docker cli. If you are not familiar with docker-compose,
Read more

Use Bodmer TFT_eSPI Library with PlatformIO

published:
Image
If you start playing around with TFT on your ESP8266 or ESP32, you will arrive at the excellent TFT_eSPI Library by Bodmer . It's not only magnitudes faster than the Adafruit driver , it's also comes with support for a wide range of display drivers. Test results of the TFT_eSPI library Unfortunately it has one catch: The library is not really ready to be included as a library in your projects as it requires adjustments in the library itself. While it's not uncommon to define your settings with #define directives, this library expects you to patch files to configure it . The official way goes like this Locate the library folder (Windows would be Documents\Arduino\libraries\TFT_eSPI ) and create your own setup in the library sub folder User_Setups To use your configuration, patch the file User_Setup_Select.h  to include your configuration This has obviously some disadvantages. At latest when you wan't to update the library or work with different screens at the same tim
Read more

Migrating from Arduino IDE to Visual Studio Code to PlatformIO

published:
Image
Like every other beginner on ESP8266 I started with my first steps in the official Arduino IDE. I was impressed by the simplicity and easy to use examples that were build-in to the UI. Also the simple UI made it easy to start in uncharted land, especially for someone that did never develop embedded firmware. The library manager let me install additional libraries to play around with, and it also mostly worked. Standard Arduino Editor (IDE) Needless to say, I missed a lot of the features that I was used to from more powerful IDE's like IntelliJ or Visual Studio. I think even Visual Basic 6 had a better IntelliSense support than the Arduino IDE. In fact, Arduino IDE has none. Also, the lack of a dependency management (for libraries) makes it really hard to share code between multiple peers, as you have to manually install the correct version of a library in the global libraries folder. Guess what happens if you use different versions? But overall the Arduino IDE is a g
Read more