Compressed btrfs root on eMMC

For my application, the space on the eMMC is beginning to get a little tight. I decided to write a new “flash” script that would flash the eMMC, but set it up using btrfs with compression for the root. It’s a bit tricky since the uboot stuff doesn’t support it so the /boot needs to be ext2/4 and all the device tree files it needs also needs to be on that partition. Anyway, I did get it all working. I’ve attached my script.

Basically, it creates a 128MB /boot (/dev/mmcblk1p1) to hold the kernel and initial filesystem and a /lib/firmware that just houses the various Beaglebone devicetree files. It then uses the rest of the eMMC (/dev/mmcblk1p2) as a btrfs filesystem with zlib compression for the flash. It then rsyncs everything over and creates a new /etc/fstab file, updates the boot/uEnv.txt to mark the root as btrfs, and builds and copies all the bb overlay dts things over. For the fstab, I have it mounting with compression=lzo as the data for my app works very well for that (I need the extra CPU when loading/saving my files), but you could keep that as zlib as well.

For my app, this leaves well over 2GB of free space on the eMMC as compared to about 500MB before. Performance is about the same. Anyway, I thought I’d share it with others in case they want to try it out, play with it, suggest enhancements, etc…

BBB-FlashMMC.sh (2.78 KB)

For my application, the space on the eMMC is beginning to get a little
tight. I decided to write a new "flash" script that would flash the eMMC,
but set it up using btrfs with compression for the root. It's a bit tricky
since the uboot stuff doesn't support it so the /boot needs to be ext2/4 and
all the device tree files it needs also needs to be on that partition.
Anyway, I did get it all working. I've attached my script.

btrfs got added in u-boot v2017.11... :wink:

Basically, it creates a 128MB /boot (/dev/mmcblk1p1) to hold the kernel and
initial filesystem and a /lib/firmware that just houses the various
Beaglebone devicetree files. It then uses the rest of the eMMC
(/dev/mmcblk1p2) as a btrfs filesystem with zlib compression for the flash.
It then rsyncs everything over and creates a new /etc/fstab file, updates
the boot/uEnv.txt to mark the root as btrfs, and builds and copies all the
bb overlay dts things over. For the fstab, I have it mounting with
compression=lzo as the data for my app works very well for that (I need the
extra CPU when loading/saving my files), but you could keep that as zlib as
well.

For my app, this leaves well over 2GB of free space on the eMMC as compared
to about 500MB before. Performance is about the same. Anyway, I
thought I'd share it with others in case they want to try it out, play with
it, suggest enhancements, etc....

Careful around v4.14.x+ "I" dropped btrfs, it saves a good second or
two on bootup, from all the raid6 calculations that are done. (if we
could force it to always use one mode, and thus not waste bootup time,
i'll gladly reenable it)

Regards,

btrfs got added in u-boot v2017.11… :wink:

https://github.com/u-boot/u-boot/commit/0c936ee3194ac5b533a18fcd08e5d285853af5fe#diff-09d710108823b360adf225b17cef57f9

Hmm… I couldn’t get it to work. I could never get it to actually find any of the files on the btrfs root. I’ll try again. It would certainly simplify things.

For my app, this leaves well over 2GB of free space on the eMMC as compared
to about 500MB before. Performance is about the same. Anyway, I
thought I’d share it with others in case they want to try it out, play with
it, suggest enhancements, etc…

Careful around v4.14.x+ “I” dropped btrfs, it saves a good second or
two on bootup, from all the raid6 calculations that are done. (if we
could force it to always use one mode, and thus not waste bootup time,
i’ll gladly reenable it)

Ick… That would be REALLY bad for me. I’ve switch over to btrfs on pretty much all my beaglebones, even on the SD cards. The files I have are large (>500MB) but highly compressible so it’s much faster to read them from btrfs than ext4. That said, I’m still on 4.4 as we use uio-pruss for working with the PRU’s and have not plans/desire to change that (and last I looked, there were issues with it on 4.9+, maybe those issues are now fixed).

For me, an extra second or two on a bootup that takes almost 75 seconds is irrelevant when the beaglebone is up for weeks/months on end.

btrfs got added in u-boot v2017.11... :wink:

fs: btrfs: Add U-Boot fs handlers. · u-boot/u-boot@0c936ee · GitHub

Hmm... I couldn't get it to work. I could never get it to actually find
any of the files on the btrfs root. I'll try again. It would certainly
simplify things.

No Worries, they posted that to u-boot the same week i disabled it in
the kernel..

It's something i'd like to personally test too, just haven't had a
moment to see what the u-boot version could do..

>
> For my app, this leaves well over 2GB of free space on the eMMC as
> compared
> to about 500MB before. Performance is about the same. Anyway, I
> thought I'd share it with others in case they want to try it out, play
> with
> it, suggest enhancements, etc....

Careful around v4.14.x+ "I" dropped btrfs, it saves a good second or
two on bootup, from all the raid6 calculations that are done. (if we
could force it to always use one mode, and thus not waste bootup time,
i'll gladly reenable it)

Ick... That would be REALLY bad for me. I've switch over to btrfs on
pretty much all my beaglebones, even on the SD cards. The files I have
are large (>500MB) but highly compressible so it's much faster to read them
from btrfs than ext4. That said, I'm still on 4.4 as we use uio-pruss for
working with the PRU's and have not plans/desire to change that (and last I
looked, there were issues with it on 4.9+, maybe those issues are now
fixed).

For me, an extra second or two on a bootup that takes almost 75 seconds is
irrelevant when the beaglebone is up for weeks/months on end.

Yeah, i'd like to bring it back, just be careful with the v4.14.x from
today, so here's what i see on bootup:

v4.9.x

[ 0.551792] omap4_sram_init:Unable to allocate sram needed to
handle errata I688
[ 0.551810] omap4_sram_init:Unable to get sram pool needed to
handle errata I688
[ 0.661277] raid6: int32x1 gen() 257 MB/s
[ 0.729310] raid6: int32x1 xor() 229 MB/s
[ 0.797283] raid6: int32x2 gen() 322 MB/s
[ 0.865257] raid6: int32x2 xor() 283 MB/s
[ 0.933293] raid6: int32x4 gen() 328 MB/s
[ 1.001258] raid6: int32x4 xor() 267 MB/s
[ 1.069373] raid6: int32x8 gen() 300 MB/s
[ 1.089205] random: fast init done
[ 1.137287] raid6: int32x8 xor() 233 MB/s
[ 1.205231] raid6: neonx1 gen() 1454 MB/s
[ 1.273231] raid6: neonx1 xor() 843 MB/s
[ 1.341225] raid6: neonx2 gen() 1921 MB/s
[ 1.409220] raid6: neonx2 xor() 1203 MB/s
[ 1.477241] raid6: neonx4 gen() 1088 MB/s
[ 1.545216] raid6: neonx4 xor() 791 MB/s
[ 1.613209] raid6: neonx8 gen() 803 MB/s
[ 1.681241] raid6: neonx8 xor() 732 MB/s
[ 1.681251] raid6: using algorithm neonx2 gen() 1921 MB/s
[ 1.681259] raid6: .... xor() 1203 MB/s, rmw enabled
[ 1.681267] raid6: using intx1 recovery algorithm
[ 1.692396] edma 49000000.edma: TI EDMA DMA engine driver

v4.14.x:

[ 0.186505] omap4_sram_init:Unable to allocate sram needed to
handle errata I688
[ 0.186529] omap4_sram_init:Unable to get sram pool needed to
handle errata I688
[ 0.206935] edma 49000000.edma: TI EDMA DMA engine driver

the "raid6" part of the btrfs driver is required, it always picks
"neonx2 gen()" i'd like to patch btrfs to always just force that mode
for us..

On the devices such as the PocketBeagle, we need fast bootup, as
there's no good indicator of bootup, till the usb gadget device comes
alive.

Regards,

I'm goint to give this a try:

https://gist.github.com/RobertCNelson/c632031e77cb17b667b2a3aa99a735ea

Regards,

Doesn’t seem to work…

`

U-Boot 2018.01-rc1-00002-g87ef84e094 (Dec 06 2017 - 14:58:04 -0600), Build: jenkins-github_Bootloader-Builder-16

CPU : AM335X-GP rev 2.1
I2C: ready
DRAM: 512 MiB
No match for driver ‘omap_hsmmc’
No match for driver ‘omap_hsmmc’
Some drivers were not found
Reset Source: Power-on reset has occurred.
MMC: OMAP SD/MMC: 0, OMAP SD/MMC: 1
Using default environment

Board: BeagleBone Black
not set. Validating first E-fuse MAC
BeagleBone Black:
Model: Element14 BeagleBone Black Industrial:
BeagleBone: cape eeprom: i2c_probe: 0x54:
BeagleBone: cape eeprom: i2c_probe: 0x55:
BeagleBone: cape eeprom: i2c_probe: 0x56:
BeagleBone: cape eeprom: i2c_probe: 0x57:
Net: eth0: MII MODE
cpsw, usb_ether
Press SPACE to abort autoboot in 2 seconds
board_name=[A335BNLT] …
board_rev=[EIA0] …
Card did not respond to voltage select!
mmc_init: -95, time 13
Card did not respond to voltage select!
mmc_init: -95, time 12
Card did not respond to voltage select!
mmc_init: -95, time 13
gpio: pin 56 (gpio 56) value is 0
gpio: pin 55 (gpio 55) value is 0
gpio: pin 54 (gpio 54) value is 0
gpio: pin 53 (gpio 53) value is 1
Card did not respond to voltage select!
mmc_init: -95, time 13
Card did not respond to voltage select!
mmc_init: -95, time 13
switch to partitions #0, OK
mmc1(part 0) is current device
Failed to mount ext2 filesystem…
** Unrecognized filesystem type **
gpio: pin 56 (gpio 56) value is 0
gpio: pin 55 (gpio 55) value is 0
gpio: pin 54 (gpio 54) value is 0
gpio: pin 53 (gpio 53) value is 1
switch to partitions #0, OK
mmc1(part 0) is current device
gpio: pin 54 (gpio 54) value is 1
Failed to mount ext2 filesystem…
** Unrecognized filesystem type **
Checking for: /uEnv.txt …
Failed to mount ext2 filesystem…
** Unrecognized filesystem type **
Checking for: /boot.scr …
Failed to mount ext2 filesystem…
** Unrecognized filesystem type **
Checking for: /boot/boot.scr …
Failed to mount ext2 filesystem…
** Unrecognized filesystem type **
Checking for: /boot/uEnv.txt …
Failed to mount ext2 filesystem…

`

Looks like more testing is needed! :wink:

PS, btrfs is now back:

https://github.com/RobertCNelson/bb-kernel/commit/0a59eaa178b69cd3dc0a489656d359c33e632188

Depending on KERNEL_MODE_NEON (RT deselects KERNEL_MODE_NEON), you'll
either get: raid6_neonx2 or raid6_intx4 (which are the calculated
defaults) it would be nicer to just pass which one we want thru
cmdline, but that'll need more work for a later time..

Regards,

Got it!

u-boot just needs:

CONFIG_FS_BTRFS=y

and we need to pass:

mmcrootfstype=btrfs rootwait

thru /boot/uEnv.txt

and it just works:

https://gist.github.com/RobertCNelson/d2ba5d74c8bc7407ce7bd8a521424b20

I'll add that config to u-boot by default...

Regards,

Uhoh..

Running uname_boot ...
Failed to mount ext2 filesystem...
loading /boot/vmlinuz-4.14.5-bone8 ...
Failed to mount ext2 filesystem...
data abort
pc : [<9ff6fa90>] lr : [<00000000>]
reloc pc : [<80825a90>] lr : [<e08b6000>]
sp : 9df29148 ip : 82020000 fp : 0001f000
r10: 000000cf r9 : 9df29ed8 r8 : 82001000
r7 : 0001d62f r6 : 00001000 r5 : 0001df29 r4 : 9df542b7
r3 : 00001000 r2 : 82000000 r1 : 00001000 r0 : 00000000
Flags: nzCv IRQs off FIQs on Mode SVC_32
Resetting CPU ...

resetting ...

Don't compress files under /boot/*....

Regards,

So btrfs in u-boot should handle both zlib and lzo

So while lzo has a bug at the moment, zlib works!

btrfs filesystem defragment -r -v -czlib /

root@beaglebone:~# df -h | grep mmc
/dev/mmcblk0p1 15G 956M 13G 8% /

(originally 1.7G)

I'm going to submit a bug to u-boot for lzo..

Regards,

That’s awesome. Will definitely help breath some new life into the little 4GB eMMC. :slight_smile:

Thanks for looking into this.

Dan

I've been toying with the idea of making btrfs default for our buster
images (still about a 1.5 years away).. onboard compression and cow
might make the eMMC more reliable for end users who like to yank the
power cord...

Regards,

Quick update, a patch has been posted to u-boot's mainling list that
fixes the issue with lzo compress btrfs root partition.

https://lists.denx.de/pipermail/u-boot/2018-January/317777.html

The patch has been integrated into our u-boot, on jan 20th:

U-Boot SPL 2018.01-00002-g9aa111a004 (Jan 20 2018 - 12:45:29)
Trying to boot from MMC1

Regards,