OSTree on Beaglebone Boards

General Discussion
1

I’ve taken my first concrete steps towards trying to get an OSTree managed Beaglebone image, but there are definitely many pieces I don’t fully understand so it’s difficult to make progress. I’m hoping to not have to dig too deeply into U-Boot, the kernel or ostree source code, but we’ll see how that goes. Any info anyone has is appreciated!

I’ve hobbled together a script that takes a Beaglebone console image, conditions the rootfs into a structure that it can be checked into an ostree repository, then creates a new rootfs with an OSTree structure for managing the full system using that repository. Some things are hardcoded that shouldn’t and other shortcuts were taken, with the goal of getting to a bootable image as soon as possible. Improvements can be made later. I’ve gotten to a point that I have an image that can be flashed to a microSD card, but it fails to boot during the initramfs.

I put together a Docker image so it should be easy to try for anyone who wants to give it a go (you could also build the docker image yourself using this Dockerfile as a starting point):

docker pull pocketnc/build-bbb-ostree-console
docker run --rm --privileged -v /tmp:/host pocketnc/build-bbb-ostree-console
# Outputs /tmp/bone-debian-10.9-console-2021-07-23-4gb.img

# If you want to run the script manually/make edits/debug
# The script is /tmp/bbb-ostree-helper-scripts/ostree_prep_rootfs.sh
docker run -ti --entrypoint /bin/bash --privileged -v /tmp:/host pocketnc/build-bbb-ostree-console

Here is the output from the boot process:

U-Boot SPL 2019.04-00002-gbb4af0f50f (Jul 08 2019 - 11:44:39 -0500)
Trying to boot from MMC2
Loading Environment from EXT4... ** File not found /boot/uboot.env **

** Unable to read "/boot/uboot.env" from mmc0:1 **


U-Boot 2019.04-00002-gbb4af0f50f (Jul 08 2019 - 11:44:39 -0500), Build: jenkins-github_Bootloader-Builder-128

CPU  : AM335X-GP rev 2.1
I2C:   ready
DRAM:  512 MiB
No match for driver 'omap_hsmmc'
No match for driver 'omap_hsmmc'
Some drivers were not found
Reset Source: Power-on reset has occurred.
RTC 32KCLK Source: External.
MMC:   OMAP SD/MMC: 0, OMAP SD/MMC: 1
Loading Environment from EXT4... ** File not found /boot/uboot.env **

** Unable to read "/boot/uboot.env" from mmc0:1 **
Board: BeagleBone Black
<ethaddr> not set. Validating first E-fuse MAC
BeagleBone Black:
BeagleBone: cape eeprom: i2c_probe: 0x54:
BeagleBone: cape eeprom: i2c_probe: 0x55:
BeagleBone: cape eeprom: i2c_probe: 0x56:
BeagleBone: cape eeprom: i2c_probe: 0x57:
Net:   eth0: MII MODE
cpsw, usb_ether
Press SPACE to abort autoboot in 2 seconds
board_name=[A335BNLT] ...
board_rev=[000C] ...
switch to partitions #0, OK
mmc0 is current device
SD/MMC found on device 0
gpio: pin 56 (gpio 56) value is 0
gpio: pin 55 (gpio 55) value is 0
gpio: pin 54 (gpio 54) value is 0
gpio: pin 53 (gpio 53) value is 1
starting USB...
USB0:   Port not available.
gpio: pin 54 (gpio 54) value is 1
Checking for: /boot/uEnv.txt ...
switch to partitions #0, OK
mmc0 is current device
gpio: pin 56 (gpio 56) value is 0
gpio: pin 55 (gpio 55) value is 0
gpio: pin 54 (gpio 54) value is 0
gpio: pin 53 (gpio 53) value is 1
switch to partitions #0, OK
mmc0 is current device
gpio: pin 56 (gpio 56) value is 0
gpio: pin 55 (gpio 55) value is 0
gpio: pin 54 (gpio 54) value is 0
gpio: pin 53 (gpio 53) value is 1
switch to partitions #0, OK
mmc0 is current device
gpio: pin 54 (gpio 54) value is 1
Checking for: /uEnv.txt ...
Checking for: /boot.scr ...
Checking for: /boot/boot.scr ...
Checking for: /boot/uEnv.txt ...
gpio: pin 55 (gpio 55) value is 1
2158 bytes read in 42 ms (49.8 KiB/s)
Loaded environment from /boot/uEnv.txt
Checking if uname_r is set in /boot/uEnv.txt...
gpio: pin 56 (gpio 56) value is 1
Running uname_boot ...
loading /boot/vmlinuz-4.19.94-ti-r62 ...
10166784 bytes read in 770 ms (12.6 MiB/s)
debug: [enable_uboot_overlays=1] ...
debug: [enable_uboot_cape_universal=1] ...
debug: [uboot_base_dtb_univ=am335x-boneblack-uboot-univ.dtb] ...
uboot_overlays: [uboot_base_dtb=am335x-boneblack-uboot-univ.dtb] ...
uboot_overlays: Switching too: dtb=am335x-boneblack-uboot-univ.dtb ...
loading /boot/dtbs/4.19.94-ti-r62/am335x-boneblack-uboot-univ.dtb ...
164234 bytes read in 179 ms (895.5 KiB/s)
uboot_overlays: [fdt_buffer=0x60000] ...
uboot_overlays: unable to find [mmc 0:1 /lib/firmware/BB-BONE-eMMC1-01-00A0.dtbo]...
uboot_overlays: uboot loading of [/lib/firmware/BB-HDMI-TDA998x-00A0.dtbo] disabled by /boot/uEnv.txt [disable_uboot_overlay_video=1]...
uboot_overlays: unable to find [mmc 0:1 /lib/firmware/BB-ADC-00A0.dtbo]...
uboot_overlays: unable to find [mmc 0:1 AM335X-PRU-UIO-00A0.dtbo]...
loading /boot/initrd.img-4.19.94-ti-r62 ...
7433990 bytes read in 599 ms (11.8 MiB/s)
debug: [console=ttyO0,115200n8 bone_capemgr.uboot_capemgr_enabled=1 root=/dev/mmcblk0p1 ro rootfstype=ext4 rootwait coherent_pool=1M net.ifnames=0 lpj=1990656 rng_core.default_quality=100 quiet ostree=/ostree/deploy/debian/deploy/170d5ee24c41d63c7bbe595352cdb2404d258ba890ad5e655742248e8e09c829.0] ...
debug: [bootz 0x82000000 0x88080000:716f06 0x88000000] ...
## Flattened Device Tree blob at 88000000
   Booting using the fdt blob at 0x88000000
   Loading Ramdisk to 8f8e9000, end 8fffff06 ... OK
   Loading Device Tree to 8f8bd000, end 8f8e8189 ... OK

Starting kernel ...

[    0.002165] timer_probe: no matching timers found
[    0.188179] l4_wkup_cm:clk:0010:0: failed to disable
[    1.524570] omap_voltage_late_init: Voltage driver support not added
run-init: can't execute '/sbin/init': No such file or directory
Target filesystem doesn't have requested /sbin/init.
run-init: can't execute '/sbin/init': No such file or directory
run-init: can't execute '/etc/init': No such file or directory
run-init: can't execute '/bin/init': No such file or directory
run-init: can't execute '/bin/sh': No such file or directory
run-init: can't execute '': No such file or directory
No init found. Try passing init= bootarg.


BusyBox v1.30.1 (Debian 1:1.30.1-4) built-in shell (ash)
Enter 'help' for a list of built-in commands.

(initramfs)

It looks like it has problems locating overlays in /lib as well as the init script as that information is buried in the /ostree deployment. I was under the impression that ostree is supposed to chroot to that deployment in the kernel, but that doesn’t seem to be happening (or perhaps hasn’t happened yet). If anyone has any knowledge about ostree details or Beaglebone U-boot or kernel details, please let me know!

1 Like
2

I also posted an issue on the OSTree GitHub repository: ostree on Beaglebone Black · Issue #2357 · ostreedev/ostree · GitHub

3

Is anyone familiar with the initramfs that is generated for the Beaglebone? It sounds like I’ll need to modify it to handle performing bind mounts for the ostree deployment and other directories within it.

4

It booted! I had to insert the switchroot.sh script in between the overlayroot and udev scripts in the ORDER file in /scripts/init-bottom. Making progress…

5

I changed some things around so it’s easier to develop the OSTree conversion script locally and to choose any Beaglebone image:

How to build docker image

git clone https://github.com/PocketNC/bbb-ostree-helper-scripts
cd bbb-ostree-helper-scripts/docker
docker build . --tag pocketnc/build-bbb-ostree-image

How to use the docker image

git clone https://github.com/PocketNC/bbb-ostree-helper-scripts
cd bbb-ostree-helper-scripts

# pick an image to convert and download it
curl -L -O https://rcn-ee.com/rootfs/bb.org/testing/2021-07-26/buster-console/bone-debian-10.10-console-armhf-2021-07-26-1gb.img.xz

# Either build the image using the commands above or:
docker pull pocketnc/build-bbb-ostree-image

# specify that image on the docker run command
docker run --rm --privileged -v $PWD:/host pocketnc/build-bbb-ostree-image bone-debian-10.10-console-armhf-2021-07-26-1gb.img.xz

# A bone-debian-10.10-console-armhf-2021-07-26-1gb-ostree.img file will be created (no xz to save time while developing)
2 Likes
6

I have a Beaglebone booting, but certain services/devices are having trouble. Anyone know any details about these services? I imagine most of the issues stem from having a read only filesystem for certain locations. Right now I have /opt as read-only (it doesn’t have to be, but I’d like to limit how many folders are writable as updates could become a problem).

hostapd continually errors because it keeps restarting and failing. I can manually disable it after the system boots up to stop the errors. On a normal Beaglebone, though, it seems to automatically disable it. I’m sure I could remove that service from the build, but ideally this would be handled similarly to a normal Beaglebone image. Any ideas?

I’m unable to connect over ssh to the 192.168.6.2 or 192.168.7.2 addresses.

/dev/ttyGS0 times out. I’m not sure how to get more info about this. All I see is a message in journalctl: " Timed out waiting for device /dev/ttyGS0.".

dnsmasq.service also fails. It claims an address is already in use. Perhaps it’s trying to write to a read only location?

I’ve attached the output of dmesg and journalctl. Anything else that would be helpful debugging this?
dmesg.txt (21.8 KB)
journalctl.txt (57.4 KB)

7

I have a process that converts a standard Beaglebone image to an ostree managed version of it. It uses a version of ostree built from source with modifications I made specifically for getting this to work. There’s an open pull request on the official ostree repository for those changes. We’ll see if the maintainers have any thoughts on those changes. It also installs dracut onto the system for creating an initramfs with ostree support in it. Besides that, the resulting ostree image should largely behave the same as a normal image. You can’t use apt-get to install anything, as /usr is mounted read-only, but the system can be atomically upgraded to later builds.

If anyone wants to give it a shot to help shake out any problems with it, I’d appreciate it! @RobertCNelson any interest?

I put a README together here.

BTW, this has only really been tested with Buster. I ran it once on a Bullseye image and it didn’t work and haven’t had a chance to look into why.

8

I also want to note that if you want to jump right into seeing what OSTree can do, there is a client image I generated here.

If you flash that onto a microSD card, it should boot right up, and be in a comparable state to this image. You can then run the following commands to upgrade it to a comparable version of this image

export REMOTE=debian
export REMOTE_URL=http://pocketnc-ostree-test-repo.s3-website-us-east-1.com
sudo ostree remote add --no-gpg-verify --no-sign-verify $REMOTE $REMOTE_URL
sudo ostree pull $REMOTE bb.org/testing/2021-08-23/buster-iot
sudo ostree admin deploy bb.org/testing/2021-08-23/buster-iot
sudo reboot

You can free up some space and get rid of the previous version by running these commands:

sudo ostree admin undeploy 1
sudo ostree refs --delete debian:bb.org/testing/2020-06-01/buster-iot
sudo ostree admin cleanup

You can then upgrade to a comparable version of this LXQt image using these commands:

sudo ostree pull debian bb.org/testing/2021-08-23/buster-lxqt
sudo ostree admin deploy bb.org/testing/2021-08-23/buster-lxqt

You should have about 100MB of free space still with both the LXQt image and the iot image files (note that the LXQt image itself only has 250MB of free space, so the fact that the LXQt image is mostly a superset of the iot image and ostree doesn’t duplicate files that are the same is what makes this possible). You can of course grow the partition to fill your microSD card, the same as you do on a normal image:

sudo /opt/script/tools/grow_partition.sh
sudo reboot
9

@pocketnc_john ,

Thanks for posting this and getting a working example going. I was able to successfully use your script and Docker image to create an ostree image for both my BeagleBone Green and BeagleBone Green Gateway.

I couldn’t get the eMMC flasher images to work out of the box (and didn’t dive into it) but it worked when running the OS off the microSD card.