Unable to boot after 'apt upgrade' to Debian 9.13

Hello all,

I had an old Debian 9.9 console image on my BBB and did an apt upgrade that bumped it to Debian 9.13. Now the board does not boot.

If I hook up a serial console I see it keeps stopping at:

Starting kernel …

[ 0.001033] clocksource_probe: no matching clocksources found
[ 1.312908] wkup_m3_ipc 44e11324.wkup_m3_ipc: could not get rproc handle
[ 1.487471] omap_hsmmc 481d8000.mmc: prop pinctrl-0 index 0 invalid phandle
[ 1.522392] omap_voltage_late_init: Voltage driver support not added
[ 1.529573] PM: Cannot get wkup_m3_ipc handle
[ 1.807980] bone_capemgr bone_capemgr: slot #0: No cape found
[ 1.839062] bone_capemgr bone_capemgr: slot #1: No cape found
[ 1.871759] bone_capemgr bone_capemgr: slot #2: No cape found
[ 1.904436] bone_capemgr bone_capemgr: slot #3: No cape found
Gave up waiting for root file system device. Common problems:

  • Boot args (cat /proc/cmdline)
  • Check rootdelay= (did the system wait long enough?)
  • Missing modules (cat /proc/modules; ls /dev)
    ALERT! /dev/mmcblk1p1 does not exist. Dropping to a shell!
    (initramfs)

Is there any way to fix this other than reflashing?

Thanks

This may help:

https://forum.digikey.com/t/beablebone-black-boot-issue-dev-mmcblk1p1-does-not-exist/7717

Cheers,

Jon

Hi Jon,

I have already seen that post. It doesn’t help since it requires reflashing the board.

I had another board that I did the same ‘apt update/upgrade’ on and had the same problem.

BTW, If have not been able to get that board to reflash, it keeps giving me some strange error message about the SD-Card, still have not figured that out. The SD-Card worked fine on a different BBB.

I would like to know if there is a way to fix this without reflashing the board.

I guess a bigger question is - Does upgrading the console image from Debian 9.9 to 9.13 actually work?

Hartley

Could you post the strange message you are seeing regarding the uSD card?

For me, going from 9.9 to 9.13 and beyond, I would just do a reflash of the image.

Which rev of BBB do you have? Rev B or Rev C?

Robert is the master of this so I would think his suggestions would clear the issue.

Cheers,

Jon

Let’s see your serial boot log from j1 header to see where it’s getting stick…

Hi Robert,

Quick background on this.

I have a “standard” image I use that is based on the Console Debian 9 image. I usually keep it pretty up-to-date but have not updated it since Debian 9.9.

I was trying to update it to the latest Debian 9.13 using apt update/upgrade. This has always worked in the past.

But going from 9.9 to 9.13 I can no longer get the BeagleBone Black to boot. This has happened on 2 different BeagleBone Black’s so far.

Here is the serial log:

Want to remove "quiet" so we can see what really failed?

Regards,

Robert,

Glad to… How?

I know I can edit /boot/uEnv.txt to remove the “quiet” but the board is not booting…

How do I remove it from uBoot?

Hartley

Ok, this happened again. I forgot that an ‘apt upgrade’ breaks my setup.

I reinstalled my default image on the BBB. For reference:

~$ cat /etc/dogtag

BeagleBoard.org Debian Image 2018-01-21
~$ lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux 9.9 (stretch)
Release: 9.9
Codename: stretch

Not sure what other information may be helpful.

This install has worked for me with no issues for quite a while. But, it’s pretty behind and should be upgraded:

~$ sudo apt upgrade

127 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 86.5 MB of archives.
After this operation, 1,395 kB disk space will be freed.

But it appears every time I try an ‘apt upgrade’ it seems to go ok but when I reboot I get the error about:

[ 0.001030] clocksource_probe: no matching clocksources found
[ 1.313075] wkup_m3_ipc 44e11324.wkup_m3_ipc: could not get rproc handle
[ 1.487481] omap_hsmmc 481d8000.mmc: prop pinctrl-0 index 0 invalid phandle
[ 1.522508] omap_voltage_late_init: Voltage driver support not added
[ 1.529663] PM: Cannot get wkup_m3_ipc handle
[ 1.807970] bone_capemgr bone_capemgr: slot #0: No cape found
[ 1.839041] bone_capemgr bone_capemgr: slot #1: No cape found
[ 1.871724] bone_capemgr bone_capemgr: slot #2: No cape found
[ 1.904358] bone_capemgr bone_capemgr: slot #3: No cape found

Gave up waiting for root file system device. Common problems:

  • Boot args (cat /proc/cmdline)
  • Check rootdelay= (did the system wait long enough?)
  • Missing modules (cat /proc/modules; ls /dev)
    ALERT! /dev/mmcblk1p1 does not exist. Dropping to a shell!
    (initramfs)

Does anyone have an suggestions or ideas about why this is not working?

Thanks,
Hartley

Can you share the "file" name of this image? Such that i can test.

Regards,

Robert,

I really can’t remember what the original image file was that I started with. I only have the /etc/dogtag info.

The image “file” I am using is one I created using the /opt/scripts/tools/eMMC/beaglebone-black-make-microSD-flasher-from-eMMC.sh script. This was done after installing the original image and then installing just what I needed for some Qt development.

I did remember to remove the “quiet” from the command line after installing the image again. I then tried doing an ‘apt install …’ for individual packages that needed updated. It appears one of the openssh packages is what is breaking the boot.

I got this after updating openssh-client, openssh-server, openssh-sftp-server, and openssl:

[ 3.270685] Freeing unused kernel memory: 1024K
Loading, please wait…
[ 3.311056] usb 1-1.1: new full-speed USB device number 3 using musb-hdrc
starting version 232
[ 3.367799] random: systemd-udevd: uninitialized urandom read (16 bytes read)
[ 3.384045] random: systemd-udevd: uninitialized urandom read (16 bytes read)
[ 3.393865] random: udevadm: uninitialized urandom read (16 bytes read)
[ 3.425391] usb 1-1.1: New USB device found, idVendor=046d, idProduct=c534
[ 3.439037] usb 1-1.1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 3.455036] usb 1-1.1: Product: USB Receiver
[ 3.463011] usb 1-1.1: Manufacturer: Logitech
[ 3.492331] input: Logitech USB Receiver as /devices/platform/ocp/47400000.usb/47401c00.usb/musb-hdrc.1/usb1/1-1/1-1.1/1-1.1:1.0/0003:046D:C534.0001/input/input1
[ 3.587187] hid-generic 0003:046D:C534.0001: input,hidraw0: USB HID v1.11 Keyboard [Logitech USB Receiver] on usb-musb-hdrc.1-1.1/input0
[ 3.629415] input: Logitech USB Receiver as /devices/platform/ocp/47400000.usb/47401c00.usb/musb-hdrc.1/usb1/1-1/1-1.1/1-1.1:1.1/0003:046D:C534.0002/input/input2
[ 3.722058] hid-generic 0003:046D:C534.0002: input,hiddev0,hidraw1: USB HID v1.11 Mouse [Logitech USB Receiver] on usb-musb-hdrc.1-1.1/input1
Begin: Loading essential drivers … done.
Begin: Running /scripts/init-premount … done.
Begin: Mounting root file system … Begin: Running /scripts/local-top … done.
Begin: Running /scripts/local-premount … Scanning for Btrfs filesystems
done.
Begin: Waiting for root file system … Begin: Running /scripts/local-block … done.
Begin: Running /scripts/local-block … done.
Begin: Running /scripts/local-block … done.
Begin: Running /scripts/local-block … done.
Begin: Running /scripts/local-block … done.
Begin: Running /scripts/local-block … done.
Begin: Running /scripts/local-block … done.
Begin: Running /scripts/local-block … done.
Begin: Running /scripts/local-block … done.
Begin: Running /scripts/local-block … done.
Begin: Running /scripts/local-block … done.
Begin: Running /scripts/local-block … done.
Begin: Running /scripts/local-block … done.
Begin: Running /scripts/local-block … done.
Begin: Running /scripts/local-block … done.
Begin: Running /scripts/local-block … done.
Begin: Running /scripts/local-block … done.
Begin: Running /scripts/local-block … done.
Begin: Running /scripts/local-block … done.
Begin: Running /scripts/local-block … done.
Begin: Running /scripts/local-block … done.
Begin: Running /scripts/local-block … done.
Begin: Running /scripts/local-block … done.
Begin: Running /scripts/local-block … done.
Begin: Running /scripts/local-block … done.
Begin: Running /scripts/local-block … done.
Begin: Running /scripts/local-block … done.
Begin: Running /scripts/local-block … done.
Begin: Running /scripts/local-block … done.
Begin: Running /scripts/local-block … done.
Begin: Running /scripts/local-block … done.
done.
Gave up waiting for root file system device. Common problems:

  • Boot args (cat /proc/cmdline)
  • Check rootdelay= (did the system wait long enough?)
  • Missing modules (cat /proc/modules; ls /dev)
    ALERT! /dev/mmcblk1p1 does not exist. Dropping to a shell!
    (initramfs)

A quick google search says that there may be a problem with the UUID. Isn’t the SDcard flasher supposed to set that?

Hartley

I tried adding the UUID (found with blkid) to my uEnv.txt. Same result.

  1. Install SDcard image
  2. Boot
  3. sudo apt update
  4. sudo apt upgrade
  5. Reboot, system hangs

It does change the message:

Begin: Running /scripts/local-block … done.
done.
Gave up waiting for root file system device. Common problems:

  • Boot args (cat /proc/cmdline)
  • Check rootdelay= (did the system wait long enough?)
  • Missing modules (cat /proc/modules; ls /dev)
    ALERT! UUID=0159e561-b327-492f-a73a-b188b2217267 does not exist. Dropping to a shell!
    (initramfs)

Hartley

It was around the v4.4.x era we stopped using uuid, as we had a kernel fix..

Do you have a serial cable you can plug into J1 so we can see what
u-boot is doing?

Regards,

OK, another annoyance…

I jut reinstalled my SDcard image and the UUID changed.

The last install (that broke after the apt upgrade) was 0159e561-b327-492f-a73a-b188b2217267.
After this install its now 60fc6cad-2e3f-408d-a2f8-5f2320240203.

Is it possible that something during the apt upgrade is changing the mmcblk1p1 UUID?

Aside… I can kind of understand why the system will not boot if the UUID changes but prior to adding the UUID= to uEnv.txt my system was booting from /dev/mmcblk1p1. That should have still worked but it didn’t.

Getting rather annoyed by this…

Hartley

The serial cable is plugged in. What do you want me to do?

Here is the uBoot output before I do an ‘apt upgrade’

U-Boot SPL 2018.01-00002-g9aa111a004 (Jan 20 2018 - 12:45:29)
Trying to boot from MMC2

U-Boot 2018.01-00002-g9aa111a004 (Jan 20 2018 - 12:45:29 -0600), Build: jenkins-github_Bootloader-Builder-32

CPU : AM335X-GP rev 2.1
I2C: ready
DRAM: 512 MiB
No match for driver ‘omap_hsmmc’
No match for driver ‘omap_hsmmc’
Some drivers were not found
Reset Source: Global warm SW reset has occurred.
Reset Source: Power-on reset has occurred.
MMC: OMAP SD/MMC: 0, OMAP SD/MMC: 1
Using default environment

Board: BeagleBone Black
not set. Validating first E-fuse MAC
BeagleBone Black:
BeagleBone: cape eeprom: i2c_probe: 0x54:
BeagleBone: cape eeprom: i2c_probe: 0x55:
BeagleBone: cape eeprom: i2c_probe: 0x56:
BeagleBone: cape eeprom: i2c_probe: 0x57:
Net: eth0: MII MODE
cpsw, usb_ether
Press SPACE to abort autoboot in 2 seconds
board_name=[A335BNLT] …
board_rev=[00C0] …
Card did not respond to voltage select!
mmc_init: -95, time 12
Card did not respond to voltage select!
mmc_init: -95, time 13
Card did not respond to voltage select!
mmc_init: -95, time 13
gpio: pin 56 (gpio 56) value is 0
gpio: pin 55 (gpio 55) value is 0
gpio: pin 54 (gpio 54) value is 0
gpio: pin 53 (gpio 53) value is 1
Card did not respond to voltage select!
mmc_init: -95, time 13
Card did not respond to voltage select!
mmc_init: -95, time 13
switch to partitions #0, OK
mmc1(part 0) is current device
Scanning mmc 1:1…
gpio: pin 56 (gpio 56) value is 0
gpio: pin 55 (gpio 55) value is 0
gpio: pin 54 (gpio 54) value is 0
gpio: pin 53 (gpio 53) value is 1
switch to partitions #0, OK
mmc1(part 0) is current device
gpio: pin 54 (gpio 54) value is 1
Checking for: /uEnv.txt …
Checking for: /boot.scr …
Checking for: /boot/boot.scr …
Checking for: /boot/uEnv.txt …
gpio: pin 55 (gpio 55) value is 1
2295 bytes read in 15 ms (149.4 KiB/s)
Loaded environment from /boot/uEnv.txt
Checking if uname_r is set in /boot/uEnv.txt…
gpio: pin 56 (gpio 56) value is 1
Running uname_boot …
loading /boot/vmlinuz-4.9.105-ti-r114 …
9818560 bytes read in 638 ms (14.7 MiB/s)
uboot_overlays: [uboot_base_dtb=am335x-boneblack-uboot.dtb] …
uboot_overlays: Switching too: dtb=am335x-boneblack-uboot.dtb …
loading /boot/dtbs/4.9.105-ti-r114/am335x-boneblack-uboot.dtb …
61638 bytes read in 42 ms (1.4 MiB/s)
uboot_overlays: [fdt_buffer=0x60000] …
uboot_overlays: loading /lib/firmware/BB-BONE-eMMC1-01-00A0.dtbo …
1440 bytes read in 165 ms (7.8 KiB/s)
uboot_overlays: loading /lib/firmware/BB-HDMI-TDA998x-00A0.dtbo …
4896 bytes read in 58 ms (82 KiB/s)
uboot_overlays: loading /lib/firmware/BB-ADC-00A0.dtbo …
711 bytes read in 134 ms (4.9 KiB/s)
uboot_overlays: loading /lib/firmware/univ-bbb-EVA-00A0.dtbo …
62064 bytes read in 235 ms (257.8 KiB/s)
loading /boot/initrd.img-4.9.105-ti-r114 …
5801348 bytes read in 383 ms (14.4 MiB/s)
debug: [console=ttyO0,115200n8 bone_capemgr.uboot_capemgr_enabled=1 root=/dev/mmcblk1p1 ro rootfstype=ext4 rootwait coherent_pool=1M net.ifnames=0] …
debug: [bootz 0x82000000 0x88080000:588584 88000000] …

Flattened Device Tree blob at 88000000

Booting using the fdt blob at 0x88000000
Loading Ramdisk to 8fa77000, end 8ffff584 … OK
reserving fdt memory region: addr=88000000 size=7e000
Loading Device Tree to 8f9f6000, end 8fa76fff … OK

Starting kernel …

Okay that's better, nothing wrong going on their..

can you share the rest of the boot?

Regards,

Here’s the rest. This is all before the ‘apt upgrade’.

Starting kernel …

[ 0.000000] Booting Linux on physical CPU 0x0
[ 0.000000] Linux version 4.9.105-ti-r114 (root@a5-imx6q-wandboard-2gb) (gcc version 6.3.0 20170516 (Debian 6.3.0-18+deb9u1) ) #1 SMP PREEMPT Fri Aug 17 11:05:53 UTC 2018
[ 0.000000] CPU: ARMv7 Processor [413fc082] revision 2 (ARMv7), cr=10c5387d
[ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
[ 0.000000] OF: fdt:Machine model: TI AM335x BeagleBone Black
[ 0.000000] efi: Getting EFI parameters from FDT:
[ 0.000000] efi: UEFI not found.
[ 0.000000] cma: Reserved 48 MiB at 0x9c800000
[ 0.000000] Memory policy: Data cache writeback
[ 0.000000] CPU: All CPU(s) started in SVC mode.
[ 0.000000] AM335X ES2.1 (sgx neon)
[ 0.000000] percpu: Embedded 15 pages/cpu @df902000 s31820 r8192 d21428 u61440
[ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 129412
[ 0.000000] Kernel command line: console=ttyO0,115200n8 bone_capemgr.uboot_capemgr_enabled=1 root=/dev/mmcblk1p1 ro rootfstype=ext4 rootwait coherent_pool=1M net.ifnames=0
[ 0.000000] PID hash table entries: 2048 (order: 1, 8192 bytes)
[ 0.000000] Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
[ 0.000000] Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
[ 0.000000] Memory: 440220K/522240K available (12288K kernel code, 1139K rwdata, 4324K rodata, 1024K init, 459K bss, 32868K reserved, 49152K cma-reserved, 0K highmem)
[ 0.000000] Virtual kernel memory layout:
[ 0.000000] vector : 0xffff0000 - 0xffff1000 ( 4 kB)
[ 0.000000] fixmap : 0xffc00000 - 0xfff00000 (3072 kB)
[ 0.000000] vmalloc : 0xe0000000 - 0xff800000 ( 504 MB)
[ 0.000000] lowmem : 0xc0000000 - 0xdfe00000 ( 510 MB)
[ 0.000000] pkmap : 0xbfe00000 - 0xc0000000 ( 2 MB)
[ 0.000000] modules : 0xbf000000 - 0xbfe00000 ( 14 MB)
[ 0.000000] .text : 0xc0008000 - 0xc0d00000 (13280 kB)
[ 0.000000] .init : 0xc1300000 - 0xc1400000 (1024 kB)
[ 0.000000] .data : 0xc1400000 - 0xc151cd58 (1140 kB)
[ 0.000000] .bss : 0xc151e000 - 0xc1590ff0 ( 460 kB)
[ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[ 0.000000] Preemptible hierarchical RCU implementation.
[ 0.000000] Build-time adjustment of leaf fanout to 32.
[ 0.000000] RCU restricting CPUs from NR_CPUS=2 to nr_cpu_ids=1.
[ 0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=32, nr_cpu_ids=1
[ 0.000000] NR_IRQS:16 nr_irqs:16 16
[ 0.000000] IRQ: Found an INTC at 0xfa200000 (revision 5.0) with 128 interrupts
[ 0.000000] OMAP clockevent source: timer2 at 24000000 Hz
[ 0.000023] sched_clock: 32 bits at 24MHz, resolution 41ns, wraps every 89478484971ns
[ 0.000051] clocksource: timer1: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 79635851949 ns
[ 0.000066] OMAP clocksource: timer1 at 24000000 Hz
[ 0.002095] clocksource_probe: no matching clocksources found
[ 0.002417] Console: colour dummy device 80x30
[ 0.002452] WARNING: Your ‘console=ttyO0’ has been replaced by ‘ttyS0’
[ 0.002461] This ensures that you still see kernel messages. Please
[ 0.002469] update your kernel commandline.
[ 0.002496] Calibrating delay loop… 995.32 BogoMIPS (lpj=1990656)
[ 0.020746] pid_max: default: 32768 minimum: 301
[ 0.021016] Security Framework initialized
[ 0.021034] Yama: becoming mindful.
[ 0.021093] AppArmor: AppArmor initialized
[ 0.021181] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
[ 0.021194] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
[ 0.022359] CPU: Testing write buffer coherency: ok
[ 0.022433] ftrace: allocating 39262 entries in 116 pages
[ 0.133533] CPU0: thread -1, cpu 0, socket -1, mpidr 0
[ 0.133613] Setting up static identity map for 0x80100000 - 0x80100060
[ 0.167804] EFI services will not be available.
[ 0.180740] Brought up 1 CPUs
[ 0.180760] SMP: Total of 1 processors activated (995.32 BogoMIPS).
[ 0.180770] CPU: All CPU(s) started in SVC mode.
[ 0.182399] devtmpfs: initialized
[ 0.221055] VFP support v0.3: implementor 41 architecture 3 part 30 variant c rev 3
[ 0.221635] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[ 0.221668] futex hash table entries: 256 (order: 2, 16384 bytes)
[ 0.225444] xor: measuring software checksum speed
[ 0.264759] arm4regs : 1217.000 MB/sec
[ 0.304710] 8regs : 1091.000 MB/sec
[ 0.344709] 32regs : 1088.000 MB/sec
[ 0.384709] neon : 1665.000 MB/sec
[ 0.384720] xor: using function: neon (1665.000 MB/sec)
[ 0.384742] pinctrl core: initialized pinctrl subsystem
[ 0.386235] NET: Registered protocol family 16
[ 0.390094] DMA: preallocated 1024 KiB pool for atomic coherent allocations
[ 0.459089] omap_hwmod: debugss: _wait_target_disable failed
[ 0.508757] cpuidle: using governor ladder
[ 0.520738] cpuidle: using governor menu
[ 0.537011] OMAP GPIO hardware version 0.1
[ 0.563807] No ATAGs?
[ 0.563839] hw-breakpoint: debug architecture 0x4 unsupported.
[ 0.564659] omap4_sram_init:Unable to allocate sram needed to handle errata I688
[ 0.564679] omap4_sram_init:Unable to get sram pool needed to handle errata I688
[ 0.676779] raid6: neonx2 gen() 1882 MB/s
[ 0.744731] raid6: neonx2 xor() 1157 MB/s
[ 0.744743] raid6: using algorithm neonx2 gen() 1882 MB/s
[ 0.744753] raid6: … xor() 1157 MB/s, rmw enabled
[ 0.744762] raid6: using intx1 recovery algorithm
[ 0.755709] edma 49000000.edma: TI EDMA DMA engine driver
[ 0.761939] SCSI subsystem initialized
[ 0.762379] usbcore: registered new interface driver usbfs
[ 0.762459] usbcore: registered new interface driver hub
[ 0.762650] usbcore: registered new device driver usb
[ 0.763783] omap_i2c 44e0b000.i2c: could not find pctldev for node /ocp/l4_wkup@44c00000/scm@210000/pinmux@800/pinmux_i2c0_pins, deferring probe
[ 0.764863] omap_i2c 4802a000.i2c: bus 1 rev0.11 at 100 kHz
[ 0.766334] omap_i2c 4819c000.i2c: bus 2 rev0.11 at 100 kHz
[ 0.766627] media: Linux media interface: v0.10
[ 0.766700] Linux video capture interface: v2.00
[ 0.766793] pps_core: LinuxPPS API ver. 1 registered

I bet it's bb-cape-overlays, just lock it:

sudo apt-mark hold bb-cape-overlays

Regards,

OK, I did the hold on the bb-cape-overlays.

Running the ‘sudo apt upgrade’ now. I’ll let you know how it goes shortly.

It it fails to boot again to you want the full serial port log?

Hartley