Arch Linux RISC-V on a Milk-V Mars
Introduction
RISC-V is an interesting departure from x86 and ARM CPU architectures. It has a lot of potential due to it being royalty free and having some thought out technical principles on which it is based. Whether RISC-V will be able to earn itself a significant market share next to the two giants has yet to be determined. It has a good start, though. Last year the Raspberry Pi Foundation introduced the RP2350 as an affordable RISC-V microcontroller together with a development board (Pico 2). While this is a small step for a (buggy) microcontroller, backing by an organization like the Raspberry Pi Foundation is a large step for RISC-V as a whole. There are also other (mostly Chinese) organizations who develop RISC-V microcontrollers and system on chips (SoCs). While the future of RISC-V is uncertain, it cannot be said that no attempt is made to make something out of it.
I wanted to experiment with RISC-V. To explore strange and new technology is fun, but it should ideally not break the bank. My eyes fell on the Milk-V Mars (specifications, schematic), a single-board computer (SBC) with a StarFive JH-7110 SoC (datasheet). The SoC is quite a popular RISC-V “starter” model. In terms of computing power it is slightly slower compared to a Raspberry Pi 3B. That is fine. For testing my use cases I rely more on general computing and not on raw power. The Mars is available in different memory configurations. I opted for 8 GB, the largest configuration.
Speaking of the Raspberry Pi 3B, the Milk-V Mars shares the same form factor, including many of its external ports. The 40-pin GPIO header is also quite compatible. This allows the Mars to use most accessories that were released for the Pi, including cases and the PoE hat. Neat! Unlike the Pi 3B, the Mars uses a USB-C connector for power. The power supply should be able to supply 3A or more at 5V.
Now on to the software. RISC-V is a relatively new architecture, but I expected that software would be readily available due to the Mars being out for a while and the SoC on which it is based even longer. Some of the choices I found in my search:
- An old and seemingly unmaintained Debian image, supported by Milk-V. Not really interested in that. Give me something newer.
- A Ubuntu Server image officially supported by Ubuntu. Better, but still not exactly what I am looking for.
- A community distribution known as Arch Linux RISC-V.
Long-term visitors of my website will guess that the last one would grab my attention, and they are right. I have used Arch Linux and derivatives in the past, and feel quite at home in the ecosystem.
I wish it was as easy as some of my endeavors with Arch Linux ARM (ALARM). Unfortunately, Arch Linux RISC-V (ALRV) is not as mature as its sister distribution. It seems to be mostly aimed at developers who use RISC-V emulators to prepare installations on actual hardware. That’s not why I buy hardware. For me everything must run on the hardware and be able to maintain itself. Setting up a virtual environment or cross compiling is overkill for my needs.
A saving grace of ALRV is that it leans towards mainline kernel support. similarly to ALARM. What works is stable and well supported by up-to-date software, and very likely will remain so. The device’s GPU capabilities are not mainlined (yet), but aside of that almost everything else is pretty much supported. That is fine for me, since I do not plan to use the GPU anyway.
Due to a lack of (official) ALRV images for the Mars I faced a chicken-and-egg problem for the installation process. There is also a lack of information concerning the installation of ALRV in general and for the Mars in particular. I could not fall back on the experience of other users for my specific use case, but there are enough snippets of useful information here and there for the Mars and similar hardware to avoid the wheel from needing complete reinvention. So…
Prerequisites
- A Mars Milk-V. I tested this with a V1.21 model with 8 GB of RAM. These instructions might also be useful as inspiration to get other RISC-V computers running on ALRV without relying on emulators or cross compilers.
- A microSD card (at least 8 GB recommended). An experienced Linux user can use this guide to get started with the Mars on microSD and later have the system boot using eMMC, USB or network.
- An SD card reader with a microSD adapter, or a microSD card reader. This can either be embedded in a notebook, or a USB (micro)SD card reader can be used.
- A computer (or virtual machine) running an up-to-date Linux distribution. For this guide Arch Linux was used, but any other modern Linux distribution should do.
- A USB UART adapter. Due to a lack of (reliable) GPU output, we will have to go old school with a serial connection for initial configuration. While it is possible to do everything described here headless, it is a lot harder to diagnose problems. Furthermore, a USB UART adapter can be used to debrick and boot the Mars if something really goes wrong.
- A wired internet connection for the Mars. The M.2 E-Key slot supports Wi-Fi and Bluetooth, but I have not tested this and it might not be well supported on older boot loader versions. Furthermore, a lot of wireless cards need proprietary blobs that are initially not included with ALRV, introducing another chicken-and-egg problem to solve in a chicken-and-egg problem. Ethernet is supported, so let’s stick with that initially.
Milk-V Mars boot capabilities
Code execution of the Mars starts with ROM code which loads and executes the boot loader. All revisions of the board can load the boot loader from a 16 MB SPI flash chip or from UART. The latter is a recovery method if everything else fails. The boot device can be changed to UART for a single boot by holding the boot button (next to the microSD slot) while powering on the device. The boot loader can continue the boot process from microSD, eMMC module, USB or network.
The default device to load the boot loader from was limited to the SPI flash chip in earlier revisions. With board revision V1.2 and newer, two DIP switches were added which can change this. The default can still be the SPI flash chip, but it can now also be UART instead (for regular boot, not just recovery). In addition, two new devices were added from which the boot loader can be read: microSD and eMMC. On these storage devices the different stages of the boot loader are loaded from special partitions.
This guide will first install ALRV on a microSD card and run it from there using the available bootloader on the SPI flash chip. This bootloader is most likely quite old and has some compatibility issues. After ALRV is installed, the OS will be used to build a new boot loader. The new boot loader will first be installed to the microSD card on which ALRV is installed for testing. If it runs correctly, the original boot loader on the SPI flash chip will be upgraded to solidify the change.
Create an ALRV SD card
As noted earlier, ALRV does not provide images for specific devices. It does provide a generic rootfs, an archive containing a basic version of the operating system minus the boot files needed to run it. This will be the starting point.
A microSD card needs to be prepared. A special partition configuration is chosen due to the various needs of the default boot loader configuration (which will not be changed). Four partitions will be created:
The SPL partition contains the first stage of the boot loader (OpenSBI) to run from microSD (instead of SPI flash). It is identified by ROM code by its partition type (GUID). The location on the storage medium and partition number are not important, but it makes sense to have it at the beginning of the storage medium due to it containing the first stage boot loader.
The Das U-Boot partition contains the second stage of the boot loader (Das U-Boot, unsurprisingly) to run from microSD. It is identified by the partition number, which in the default configuration of the first stage boot loader must be 2. The location on the storage medium location and its partition type are not important, although a recommended GUID is given for the latter. It makes sense to have this partition right after the SPL partition at the beginning of the medium since it concerns the second stage boot loader.
The boot partition (FAT-formatted) containing the operating system kernel boot configuration (in extlinux/extlinux.conf) is identified by its partition number by the second stage boot loader, which must be 1. The default boot loader configuration only checks the first partition on the medium for extlinux.conf. This is the simplest boot option without reconfiguring the boot loader for fancy stuff such as EFI.
The root partition is not used by the boot loader. The configuration in extlinux.conf points at it so the Linux kernel will be able to find its root volume. Therefore there are no special requirements for this partition, which can simply be added after all the others and span the rest of the storage volume.
Clear, partition and format the SD and install the ALRV rootfs
This is mostly self-explanatory. The partition type codes for the boot loader partitions are provided by Das U-Boot’s documentation.
Make sure to replace /dev/mmcblk0 with the proper block device for the microSD card! Verify that you have the correct device (e.g. using lsblk, df, dmesg, …).
These instructions must be run as root since they involve raw access to a block device and a mounted root volume.
# Get the latest rootfs of Arch Linux RISC-V
curl -JLO https://archriscv.felixc.at/images/archriscv-latest.tar.zst
# Partition
sgdisk --zap-all \
--set-alignment=2 \
--new=3:4096:8191 --change-name=3:spl --typecode=3:2E54B353-1271-4842-806F-E436D6AF6985 \
--new=2:8192:16383 --change-name=2:uboot --typecode=2:BC13C2FF-59E6-4262-A352-B275FD6F7172 \
--new=1:16384:540671 --change-name=1:boot --typecode=1:EBD0A0A2-B9E5-4433-87C0-68B6B72699C7 \
--new=4:540672: --change-name=4:root --typecode=4:0FC63DAF-8483-4772-8E79-3D69D8477DE4 \
/dev/mmcblk0
# Format
mkfs.vfat -F 32 -n SDBOOT /dev/mmcblk0p1
mkfs.btrfs -f -L SDROOT /dev/mmcblk0p4
# Mount
mkdir -p /mnt/sd
mount /dev/mmcblk0p4 /mnt/sd
mkdir -p /mnt/sd/boot
mount /dev/mmcblk0p1 /mnt/sd/boot
# Extract
zstd -dc archriscv-latest.tar.zst | bsdtar -xp -C /mnt/sd
Prepare for initial boot
The ALRV rootfs misses boot files. For the initial boot the files from Ubuntu Server will be used. These will later be replaced by ALRV equivalents.
Run these instructions as root, since changes are made to a mounted boot partition.
# Get Ubuntu Server (current LTS version)
curl -JLO https://cdimage.ubuntu.com/releases/24.04.2/release/ubuntu-24.04.2-preinstalled-server-riscv64+milkvmars.img.xz
xz -d ubuntu-24.04.2-preinstalled-server-riscv64+milkvmars.img.xz
# Mount the Ubuntu server root partition from the image
mkdir -p /mnt/image
fdisk -lu ubuntu-24.04.2-preinstalled-server-riscv64+milkvmars.img
# Look up the start sector of the largest (=root) partition. For this example it is 253952.
mount -o loop,offset=$((253952*512)) ubuntu-24.04.2-preinstalled-server-riscv64+milkvmars.img /mnt/image
# Copy the required boot files to the ALRV SD card
cp -L /mnt/image/boot/vmlinuz /mnt/sd/boot/vmlinuz-linux
cp -L /mnt/image/boot/initrd.img /mnt/sd/boot/initramfs-linux.img
cp -L /mnt/image/boot/dtb /mnt/sd/boot
umount /mnt/image
# Create the boot configuration
mkdir -p /mnt/sd/boot/extlinux
cat << EOF > /mnt/sd/boot/extlinux/extlinux.conf
default linux
prompt 0
label linux
linux /vmlinuz-linux
initrd /initramfs-linux.img
fdt /dtb
append root=LABEL=SDROOT rw console=tty0 console=ttyS0,115200
EOF
Remove the SD card safely
Now we have a microSD card that should be able to boot the Milk-V Mars. Unmount the microSD from your computer by running as root:
sync && umount -R /mnt/sd
Now remove the microSD card.
Boot the Milk-V Mars and finalize the ALRV installation
If you have a V1.2 or newer board revision of the Mars, ensure that the DIP switches are configured to boot from SPI flash. The switches should both be in the off/Low position.
Insert the microSD card in the Mars. Connect a USB-to-UART adapter to the proper pins of the GPIO connector. The UART TX, UART RX and ground pins connect to the Mars the same way as they would be connected to a Raspberry Pi and use the same voltage levels (3.3V). Connection settings: a baud rate of 115,200, 8 data bits, no parity, 1 stop bit, and no flow control.
Use the Arch Linux RISC-V default root credentials. These are:
Username: root
Password: archriscv
Unless stated otherwise, all following commands are executed as root on the Mars.
Enable network support
The rootfs image is very bare. Network support must be enabled manually. Run:
cat << EOF > /etc/systemd/network/en-dhcp.network
[Match]
Name=en*
[Network]
DHCP=yes
EOF
ln -sf ../run/systemd/resolve/stub-resolv.conf /etc/resolv.conf
systemctl enable --now systemd-networkd.service systemd-resolved.service systemd-timesyncd.service
Update ALRV and enable remote access
A local user must be made which can login remotely. Doing so directly as root is insecure and is therefore not allowed in the default OpenSSH configuration of ALRV.
# Update the OS and install some extra packages
pacman -Syu --needed --noconfirm openssh sudo screen nano
# Enable sudo support
sed -Ei 's/^# (%wheel ALL=\(ALL:ALL\) NOPASSWD: ALL)$/\1/' /etc/sudoers
# Create a user with permission to use sudo
useradd -m -G wheel -s /bin/bash user
passwd user
# Enable SSH access
systemctl enable --now sshd.service
From now on SSH can be used to configure the system further. Get the system’s IP address with ip addr or by monitoring your DHCP server. It is still recommended to keep the UART adapter connected to verify that the system boots correctly in the later parts of this guide.
Switch to mainline kernel
Replace the crudely installed Ubuntu Server kernel with the ALRV-supported mainline kernel. This will also allow future kernel updates to be installed through the regular update process.
# Make the OS aware of its boot partition
echo "LABEL=SDBOOT /boot vfat rw,relatime 0 0" >> /etc/fstab
systemctl daemon-reload
mount /boot
# Replace the kernel
umount --force --lazy /usr/lib/modules
pacman -S --needed --noconfirm --overwrite '*' linux mkinitcpio
# Disable and remove the unused initramfs fallback image
sed -Ei "s/(PRESETS=\('default') 'fallback'\)/\1\)/" /etc/mkinitcpio.d/linux.preset
rm /boot/initramfs-linux-fallback.img
# Enable update of device tree blob (used during boot by the kernel) when initramfs and the blob are renewed
cat << "EOF" > /etc/initcpio/post/cp-dtb
#!/usr/bin/env bash
dtb=/usr/lib/modules/${KERNELVERSION}/dtb/starfive/jh7110-milkv-mars.dtb
dest=/boot/dtb
if ! diff $dtb $dest 2>&1 >/dev/null; then
cp $dtb $dest
fi
EOF
chmod +x /etc/initcpio/post/cp-dtb
# Regenerate initramfs
mkinitcpio -P
# Boot into the new kernel
reboot
If everything went correct, the system will restart using the mainline kernel. Run uname -a to verify this.
Update the boot loader
One problem I had with my Mars was that only 4 GB of RAM was addressable by the operating system, which I verified by running cat /proc/meminfo. The amount of RAM is not easy to physically read from the board, but the included U-Boot boot loader did tell me 8 GB was available. That made sense since I did order the 8 GB model.
It turns out that the boot loader included on the SPI flash chip is very old and does not initiate the hardware correctly to support 8 GB of RAM. While an official update for the boot loader is available, this update is quite old as well. We can do better than that.
Updating the boot loader is a procedure consisting of these steps:
- Build a new version of Das U-Boot.
- Install and test the boot loader on the SD card.
- Install the boot loader to SPI flash.
Build a new boot loader
Run as a normal user:
# Install and enable prerequisites
sudo pacman -S --needed --noconfirm base-devel git python python-pip swig
mkdir -p ~/src
cd ~/src
python -m venv python-venv/marsboot
source python-venv/marsboot/bin/activate
pip install setuptools
# Build OpenSBI (first stage boot loader)
git clone https://github.com/riscv/opensbi.git
cd opensbi
make PLATFORM=generic FW_TEXT_START=0x40000000 FW_OPTIONS=0
# This build takes around 3 minutes
# Build Das U-Boot (second stage boot loader)
cd ~/src
git clone https://github.com/u-boot/u-boot.git
cd u-boot
make starfive_visionfive2_defconfig
make OPENSBI=../opensbi//build/platform/generic/firmware/fw_dynamic.bin
# This build takes around 14 minutes
Test the new boot loader on the SD card
Only do this if your Milk-V Mars is revision V1.2 or newer. Only newer boards are capable of loading the boot loader from microSD since these have the required DIP switches to change the boot device.
Run as a normal user with sudo access:
# Install the boot loader stages to their respective partitions
cd ~/src/u-boot
sudo dd if=spl/u-boot-spl.bin.normal.out of=/dev/mmcblk1p3
sudo dd if=u-boot.itb of=/dev/mmcblk1p2
Shutdown and power off the Milk-V Mars. Switch the boot mode from “Flash” to “SD-card” (flip GPIO0 from off/Low to on/High) and power it on again. You can use the UART connection to verify that the new boot loader is loaded from SD. Once booted to ALRV, all 8 GB of RAM should be available according to cat /proc/meminfo if you have an 8 GB model.
Install the new boot loader to SPI flash
First, examine the SPI flash layout by running:
cat /proc/mtd
Note that /dev/mtd0 concerns the full SPI flash chip. This first segment fully overlaps all other segments and is therefore not necessary to do anything with.
Start by Installing some prerequisites and by making a backup. Run as a normal user with sudo authorization:
sudo pacman -S mtd-utils
# Make a backup
mkdir ~/mtdbackup
cd ~/mtdbackup
sudo nanddump -f mtd1-spl.dump /dev/mtd1
sudo nanddump -f mtd2-uboot-env.dump /dev/mtd2
sudo nanddump -f mtd3-uboot.dump /dev/mtd3
Now flash both stages of the previously build boot loader:
cd ~/src/u-boot
sudo flashcp -v spl/u-boot-spl.bin.normal.out /dev/mtd1
sudo flashcp -v u-boot.itb /dev/mtd3
Shutdown and power off the Milk-V Mars. Switch the boot mode back from “SD-card” to “Flash” if you previously tested the boot loader using the SD card (flip GPIO0 to off/Low). Now power on the device. You should see the new boot loader running from SD using the UART connection. Once booted to ALRV, all 8 GB of RAM should be available according to cat /proc/meminfo if you have an 8 GB model.
Concluding remarks
Getting Arch Linux RISC-V up and running on a Milk-V Mars was a fun exercise. Throughout this I learned that RISC-V has many quirks similar to ARM. I’m looking forward to future RISC-V development in general, and better support by the mainline kernel in particular.