Overlayroot on Debian Trixie and custom kernel 6.19.13

General Discussion
1

I’m going to outline the steps I took to get overlayroot functioning as it was a little bit of a journey.

Step one is to make sure the kernel supports the overlay root files system. For my case I have a custom kernel with pinmux mods and wifi support for RTW88_8821CU

make menuconfig

enable kernel overlay parameters, compile kernel
<*> Overlay filesystem support                                              │ │  
  │ │      [ ]   Overlayfs: turn on redirect directory feature by default              │ │  
  │ │      [*]   Overlayfs: follow redirects even if redirects are turned off          │ │  
  │ │      [ ]   Overlayfs: turn on inodes index feature by default                    │ │  
  │ │      [ ]   Overlayfs: auto enable inode number mapping (NEW)                     │ │  
  │ │      [ ]   Overlayfs: turn on metadata only copy up feature by default           │ │  
  │ │      [*]   Overlayfs: turn on extra debugging checks
2

After compiling the new kernel step two was to

sudo apt install busybox-static
sudo nano /etc/initramfs-tools/initramfs.conf
BUSYBOX=y
sudo update-initramfs -u -k $(uname -r)

after this step three


sudo nano /etc/overlayroot.conf
# uncomment to disable overlay root
# overlayroot=""

# to enable overlay root uncomment here
overlayroot_cfgdisk="disabled"
overlayroot="tmpfs:swap=1,recurse=0"
overlayroot_exclude="/mnt"
3

and step 4

sudo nano /etc/fstab

# The root file system has fs_passno=1 as per fstab(5) for automatic fsck.
/dev/mmcblk0p3  /  ext4  noatime,errors=remount-ro  0  1

# mount for lplc data
/dev/mmcblk0p4  /mnt   ext4  noatime,errors=remount-ro      0  2
/mnt /data none bind 0 0

# All other file systems have fs_passno=2 as per fstab(5) for automatic fsck.
/dev/mmcblk0p1  /boot/firmware vfat user,uid=1000,gid=1000,defaults 0 2
/dev/mmcblk0p2       none    swap    sw      0       0
debugfs  /sys/kernel/debug  debugfs  mode=755,uid=root,gid=gpio,defaults  0  0

I have a partition on my sd card that I want to read and write so this is the reason for the
overlayroot_exclude="/mnt"above

4

step 5, reboot and check for success


Begin: Running /scripts/local-bottom ... done.
Begin: Running /scripts/init-bottom ... Warning: overlayroot: configuring overlayroot
 with driver=overlay mode=tmpfs opts='swap=1,recurse=0' per /dev/mmcblk0p3/etc/overla
yroot.conf
Success: overlayroot: configured root with 'tmpfs:swap=1,recurse=0' using overlay per
 /dev/mmcblk0p3/etc/overlayroot.conf
done.


mount | grep -E "overlay|data"
overlayroot on / type overlay (ro,relatime,lowerdir=/media/root-ro,upperdir=/media/root-rw/overlay,workdir=/media/root-rw/overlay-workdir/_,uuid=on)
/dev/mmcblk0p4 on /data type ext4 (rw,noatime,errors=remount-ro)

I hope I have all of the steps above.

In the past overlayroot allowed write access to the root file system. You just lost the write info when you rebooted.

For some reason this has changed and the root file system is read only. I would like the old method back because of issues like this,

sudo beagle-version
[sudo] password for debian: 
eeprom:[A335BNLTEID02547SBI04226]
model:[TI_AM335x_BeagleBone_Black]
dogtag:[BeagleBoard.org Debian Trixie Base Image 2026-04-23]
/usr/bin/beagle-version: 12: cannot create /tmp/SPL.tmp: Read-only file system

Any suggestions?

5

to disable overlayroot

sudo overlayroot-chroot

nano /etc/overlayroot.conf
# uncomment to disable overlay root
overlayroot=""

# to enable overlay root uncomment here
#overlayroot_cfgdisk="disabled"
#overlayroot="tmpfs:swap=1,recurse=0"
#overlayroot_exclude="/mnt"
6

BTW, I tried overlayroot with the standard 6.19.13 kernel, I’m not sure the overlay file system is compiled in, but I could be wrong here, I was tying so many things to get this working.

7 
6.19.x/bb-kernel$ cat patches/defconfig | grep OVERLAY_FS
CONFIG_OVERLAY_FS=y
# CONFIG_OVERLAY_FS_REDIRECT_DIR is not set
CONFIG_OVERLAY_FS_REDIRECT_ALWAYS_FOLLOW=y
# CONFIG_OVERLAY_FS_INDEX is not set
# CONFIG_OVERLAY_FS_METACOPY is not set
# CONFIG_OVERLAY_FS_DEBUG is not set

Yeap, It is built-in by default with the shipping images..

Regards,

8

the fix is here https://unix.stackexchange.com/questions/800673/overlayroot-behaving-differently-on-debian-13
Patch systemd service

sudo nano /usr/lib/systemd/system/systemd-remount-fs.service
Add to the [Service] section
Environment=LIBMOUNT_FORCE_MOUNT2=always

reload the daemon
sudo systemctl daemon-reexec

then reboot
LIBMOUNT_FORCE_MOUNT2 is an environment variable used by libmount (the library systemd uses for mounting).
Setting it to always tells systemd to force mount2() calls even if it normally would refuse, which is what makes tmpfs overlays RW.
On Debian 12, systemd defaulted to a more permissive behavior; Debian 13 tightened it.

basically systemd is a mess right now, timing issues everywhere, things are disabled that were not disabled before, etc…

mount | grep overlay
overlayroot on / type overlay (rw,relatime,lowerdir=/media/root-ro,upperdir=/media/root-rw/overlay,workdir=/media/root-rw/overlay-workdir/_,uuid=on)
9

Just checking what you are doing here - as it isn’t quite clear.

It sounds like you want the main root file system to be read only, and you then mount a far smaller overlay file system, which is used for all changes to the file system. This is often used on rooters, but not usually on beagle devices. So really just checking that this is what you are trying to do?

If you want to do device tree overlays, thats a different set up …

10

You might want to take a look at composefs,
especially in memory-constrained environments, like Beaglebones.

1 Like
11

When overlayroot is enabled I have the root file system overlayed excluding a single large partition /mnt for storing data and user config files. /data is just a copy of /mnt for my application.

I had several problems,

  1. My custom kernel did not support overlay file systems by default (fixed)
  2. Busybox was missing from the initramfs toolbox, scripting failed looking for grep (fixed)
  3. Overlayroot finally worked but mounted the upper overlayed files system read only (fixed)
12

That’s disappointing, and looking at: ~cloud-initramfs-tools/cloud-initramfs-tools - [no description] nothing to useful for newer systemd, wonder if more 26.04 LTS users will start seeing issues, so ubuntu will fix it..

Regards