BeagleBone-Black crashes when restarting chrony.service when using custom kernel

Hi,
I am currently working on a project with a GPS-Module which synchronizes the BB-Black using its PPS signal. I am using chrony to synchronize the bbb. Everything was working fine with the 5.10.162-ti-r58 Kernel.

But recently I had to use a custom build kernel (based on 5.10.162-ti-r58) to add pps-gen support to measure how well two different BeagleBones are synchronized. At the moment, when I try to start the chrony service the whole beaglebone locks up after 1 second. If I dont start the chrony.service the beaglebone works fine. The strange thing is, that it was working with the custom build kernel previously.

I tried to reinstall chrony using
´´´
sudo apt purge chrony
sudo apt install chrony
´´´

Does anyone know how I can debug this? The logs and dmesg offer no information before locking up. The only information I can see is this
cat /var/log/syslog :

.....
<restarting chrony.service>
Aug  7 13:31:49 BeagleBoneBlackSD systemd[1]: Starting chrony, an NTP client/server...
Aug  7 13:31:50 BeagleBoneBlackSD chronyd[1304]: chronyd version 4.0 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SIGND +ASYNCDNS +NTS +SECHASH +IPV6 -DEBUG)
Aug  7 13:31:50 BeagleBoneBlackSD chronyd[1304]: Using right/UTC timezone to obtain leap second data
Aug  7 13:31:50 BeagleBoneBlackSD chronyd[1304]: Loaded seccomp filter
Aug  7 13:31:50 BeagleBoneBlackSD systemd[1]: Started chrony, an NTP client/server.
Aug  7 13:31:51 BeagleBoneBlackSD kernel: [  422.570081] pps_gen_gpio: We are late this time [0.1659879110]
Aug  7 13:31:55 BeagleBoneBlackSD chronyd[1304]: Selected source 80.153.195.191 (2.debian.pool.ntp.org)

I am using ssh to interact with the beagle bone black. Could I see more using the HDMI output?
Are there any other files that might offer information why the system halted when I reboot?

Changing back to the original 5.10.162-ti-r58 Kernel does not have this problem.

So the kernel has to be the problem. Is there anything I might have forgotten to do? I have cross compiled the kernel and move the zImage from <Desktop>/linux/arch/arm/boot/zImage to bbblack:/boot/vmlinuz-5.10.162-ti-r58
I have installed the modules on the sdcard using

make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- INSTALL_MOD_STRIP=1  INSTALL_MOD_PATH=/mnt/rootfs modules_install

Is there something else to consider when using another Kernel Image? I was using this guide.

What have you changed between the 2 kernels. Whatever you are adding or removing is causing the problem. That’s going to be the first thing to work out.

You can also do zcat /proc/config.gz (think that is the right path) on both working and non working systems to check exactly what config options were set in the kernels. Maybe something is missing in the non working build.

Hi Benedict :),
I have compared the two configs. The compiler is different, because I used the “arm-linux-gnueabi-gcc” crosscompiler instead of “gcc (Debian 10.2.1-6)”. There are additional flags in the new kernel, including the pps-gen flag, but everything from the original is in the new config.
Having renamed and changed between the kernels again, I have once again tried to reinstall chrony using while the custom Kernel was loaded

sudo apt install --reinstall chrony

Even though I did this previously aswell, chrony is now working.

But when I shutdown the device and then boot using the custom kernel the error returns.
Booting the normal kernel (and restarting chrony.service ! Without restarting chrony.service with the normal kernel, it does not work) and then switching to the custom kernel works and then rebooting(no shutdown) seems to work.

Appearently when powercycling the device, it does not work when I boot directly into the custom kernel. So is something in the volatile memory involved?

I will try to build the kernel with the crosscompiler using the exact same config as the working example to see if that works.

So my current workaround is booting the normal kernel, restarting the chrony service, then rebooting into the custom kernel and restarting chrony.service again. Atleast I got it working atleast two times using the steps above.

The two configs are the following:
original.config (186.5 KB)
selfmade_kernel.config (186.9 KB)

Additionally, I did the same on the BBAI-64 and there it s working fine. (Apart from the pps-gen signal, which is not procuded, the pps-gen module is loaded.)

Okay… unloading the pps_gen driver and then restarting chrony works with the custom kernel. But this is somewhat the only reason I am using the custom kernel.
I guess that it has to do with the pps_gen driver.

So I will close this thread, since another thread was trying to figure out a similar thing. Help with Device Tree, Trying to expose kernel pps - #27 by NotAGnelf

Another alternative is to unload the pps_gen_gpio kernel module before restarting chrony.service and then reload the pps_gen_gpio module.