First boot with debian bone 40 on beaglebone black issues

General Discussion
1

I am battling with initial boot of micro-sd card on BBB.

I have download the bone35 and run update/upgrade placing mark holds for bone40 (versions are updating faster than I can keep up with).

I edit the sysconf file with passwords/users etc and insert into beaglebone.

When I boot up (holding boot) button with lcd4 connected the device doesn’t start up on first power up - I haven’t dug into this. Normally second attempt works.

I then tried with just a plain BBB with nothing connected except for serial->usb cable so I can watch logs.

While booting I see the following Failed message:

systemd[1]: Failed to populate /etc with preset unit settings, ignoring: Read-only file system

I have not been able to figure out what is was trying to write.

I did see a reboot halfway through the logs which then takes me to the login prompt.

This may be a none issue but seeing as I am having an issue on initial boot (with LCD connected) I thought I would ask.

2

you should serial log these.. my guess the eMMC is some old state..

Regards,

3

Apologies Robert I am not sure what you mean with your response.

I saw the Failed while booting with serial->usb attached.

deb_log.txt (93.0 KB)

4

Just to clarify I am not flashing to eMMC, I am simply booting from and running on the micro-sd card.

5

So looking at: bbbio-set-sysconf

         Starting bbbio-set-sysconf.service…oard Set up system configuration...
[  112.366847] bbbio-set-sysconf[475]: Reading the system configuration settings from /boot/firmware/sysconf.txt
[  112.559701] bbbio-set-sysconf[478]: Resetting root password
[  120.914419] bbbio-set-sysconf[596]: Resetting debian password
[  123.907976] bbbio-set-sysconf[612]: Changing debian to debian
[  124.076979] bbbio-set-sysconf[614]: usermod: no changes
[  124.264710] bbbio-set-sysconf[624]: Setting hostname to 'BeagleBone'
[  124.311626] bbbio-set-sysconf[627]: Adding 'BeagleBone' to /etc/hosts
[  124.351676] bbbio-set-sysconf[629]: **********************************************
[  124.391182] bbbio-set-sysconf[631]: Rebooting after setting up sysconf.txt options
[  124.426866] bbbio-set-sysconf[633]: **********************************************

So looks like root and debian got a new password, that forced a reboot..

if the eMMC is blank, you don’t need to hold the boot button…

Normally with a fresh install, it should be 3 reboots..

1st boot: - setup user, Grow Root Partition
(reboot)
2nd boot - Resize Root Partition
(reboot)
3rd boot
Done - login…

Regards,

6

Oh this is expected… there was an bug issue with lightdm “auto-login” when the user was getting a new username (debian → not-debian) and new password (old-password → new-password) so lightdm is ‘disabled’ on first boot up and enabled after user/pass is changed..

Regards,

8 
sync
ssh-keygen -A -v || true
sync

You’d think this would block… IDK will have to add a check..

On the first boot up, ssh is shutdown and the keys are wiped.. one option is to mirror lightdm and disable ssh till keys are generated

9

What happens if you erase the dir. journal by calling it something different? Something like journal.bak or journal.old? As far as I can tell, it updates fine but one may want it back to debug a bit at times.

10

@RobertCNelson @silver2row — apologies for the deleted post; I pulled it because I’d misdiagnosed part of it, but you’d clearly already seen it. Here’s the corrected picture, which I think also explains your “you’d think this would block” — there are two different key generators in play, and the one that got killed is not the one you quoted:

  1. bb-regenerate-ssh-host-keys(via/etc/bbb.io/ssh_regenerate) runs *inside* bbbio-set-sysconf`, sequentially, before its reboot — so it can’t be interrupted by that reboot. And it never ran on my card at all: we archive a prepared image, and the trigger file had already been consumed on the build card’s own first boot.
  2. Debian’s sshd-keygen.service (ConditionFirstBoot=yes). Our image prep had deleted /etc/ssh/ssh_host_* and set /etc/machine-id to uninitialized so each flashed card generates unique identity — that combination hands key generation to this unit, running in parallel with bbbio-set-sysconf. Journal from the failed boot (monotonic):slight_smile:
[   22.85] systemd[1]: Detected first boot.
[   41.83] systemd[1]: Starting sshd-keygen.service - Generate sshd host keys on first boot...
[   63.62] bbbio-set-sysconf[613]: Rebooting after setting up sysconf.txt options
[   68.80] systemd-shutdown[1]: Sending SIGTERM to remaining processes...
[   68.86] ssh-keygen[426]: ssh-keygen: generating new host keys: RSA

That last line was ssh-keygen’s only output — the reboot (which had applied my passwords/hostname perfectly) caught it still on the first key, leaving an orphaned ssh_host_rsa_key.XXXXXXXXXX temp file. And since ConditionFirstBoot only fires once, the keys never got another chance — sshd dead on every later boot.

So stock images as shipped are fine — the exposure is only for derived images that delete host keys and reset machine-id, like ours did. Our fix was to mirror what the stock image actually ships: keep the host keys in the archived image and re-create /etc/bbb.io/ssh_regenerate before archiving. Then your sequential path replaces the keys before its reboot, and sshd-keygen.service no-ops because keys are present. Worth stating explicitly for anyone else deriving images: with this arrangement the baked-in keys aren’t optional — if they’re missing, sshd-keygen.service does real generation in parallel and the race comes back.

Your “disable ssh until keys are generated” guard would additionally harden things for people who strip keys the way we did — happy to test on real hardware if you add it.

11

Hey there…

I do not know the answer for now. I do not, for now, know too much about bbbio-set-sysconf outside of it working for setting random variables for users.

Some help to debug may actually help a bit:

  1. journalctl -xe | grep YOUR DIAGNOSIS
    a. where YOUR DIAGNOSIS are the ideas regarding systemd and its failures to review
  2. dmesg | grep YOUR DIAGNOSIS
    b. same
  3. TTY to USB (UART) debug cable…
    c. I have currently not had this issue. I can look in bbbio-set-sysconf to try
12

All’s good. Apologies if you saw my post describing the issue — I did delete the post but guess it was still seen.