What would I need to get/do to enable me to single-step through kernel code on BBB/G? JTAG header, some kind of interface, and a bunch of software installed on Ubuntu? Can anyone make specific recommendations? Is it even possible?
Thanks,
You can do this with CCSV6 with a USB200 JTAG adapter, but CCSV6 is no longer kernel aware. CCSV4 was kernel aware, but won’t work with the current kernels. You can still debug with single stepping, software and hardware breakpoints, read/write memory, etc. Trying to debug a kernel module without a kernel aware debugger is very difficult. Since you don’t know the load address of the KM, it is difficult to set a breakpoint on INIT or probe. Best to make the driver a kernel builtin so that you can break on the start of the driver.
For the most part, most kernel developers use printk to display status at strategic locations in their driver. Also, ftrace and dynamic trace tools are very useful. Read Documentation/trace on how to use these tools. They are very powerful. I have never use KDBG, but that is another solution that might work.
I use Lauterbach, which is like the gold standard for Kernel debugging and it is kernel aware and has a trace buffer.
Regards,
John
Thanks, John. I've tried printk and other logging, but it's virtually impossible to trace something like audio, which spans many modules.
I'll look into Lauterbach.
For audio, your best solution is ftrace. If you use function_graph option, then you can see the hierarchy of the call sequence. You can also use trace_printk to display variables at specific locations.
BTW, Lauterbach is about $5K investment for the base tool with USB3 connection. You might be able to get something less expensive on eBay, but I doubt if you can get anything less than $1.5.
Regards,
John
If this is a latency problem, ftrace is probably your best bet, but
function_graph and the other tracers can induce even more latency.
I've had to write my own latency tracer on occasion to measure
specific code paths (most recently with UART DMA timeouts from
softirq latency). I cribbed the basic functionality from the
irqsoff tracer; let me know if that's something you'd find useful.
If you really need to single-step, KGDB can be made to work but
there are some limitations, the most crucial being that kgdboc
runs over the console so you can't debug earlier than when the
built-in 8250 driver probes. That's like 3 secs into boot so lots
of init has already happened.
FWIW, I did push an earlycon OMAP bootconsole for v4.6 but -rc2
is early days, and there's already been one problem with
MMC block device numbering. But if you're desperate and don't
have access to JTAG, earlycon does printk() output from even
before page table setup up to console initialization.
Regards,
Peter Hurley
What would I need to get/do to enable me to single-step through
kernel code on BBB/G? JTAG header, some kind of interface, and a
bunch of software installed on Ubuntu? Can anyone make specific
recommendations? Is it even possible?Thanks, John. I've tried printk and other logging, but it's virtually
impossible to trace something like audio, which spans many modules.If this is a latency problem, ftrace is probably your best bet, but
function_graph and the other tracers can induce even more latency.
My main purpose for using function_graph option was to understand program flow, so latency wasn’t an issue for me. I haven’t had any latency issues with my drivers, so I haven’t used the latency tracers that much. Mostly I use them to measure interrupt latency of the kernel. Other than that, my Lauterbach debugger gives me ns timing resolution in the trace buffer.
I've had to write my own latency tracer on occasion to measure
specific code paths (most recently with UART DMA timeouts from
softirq latency). I cribbed the basic functionality from the
irqsoff tracer; let me know if that's something you'd find useful.If you really need to single-step, KGDB can be made to work but
there are some limitations, the most crucial being that kgdboc
runs over the console so you can't debug earlier than when the
built-in 8250 driver probes. That's like 3 secs into boot so lots
of init has already happened.
I’m wondering if late init might work for the driver you want to debug? I haven’t tried this, but maybe this is a workaround for this issue.
If it’s a company project the Lauterbach is well worth the money
If it’s a company project the Lauterbach is well worth the money
Definitely. Lauterbach is one of my favorite tools and I cannot imagine doing some work without it. It is a difficult tool to learn, but it is very powerful. I’m sure I don’t use 90% of the features available. Best part, it is so fast for a JTAG debugger.
Regards,
John
Nothing so esoteric. I'm just trying to get the damn thing to work at all. It's worked in previous kernels, but not in the latest. Seeing what code gets executed will help a lot. But I'm currently stuck with the default diagnostics available (e.g. "Unable to set hw params for playback: Invalid argument" NOT HELPFUL).
In the past, when I spent so much time with printk, I was trying to understand how all the modules worked together to actually configure and drive the CODEC. I found that to be virtually impossible to do in any comprehensive way, not already understanding it. Can't use printk() if you don't know where to put it.
OK, so you have to rebuild the entire ALSA library and tools with debug symbols. Also, your Kernel must have debug symbols turned on. You will need to enable various ftrace features with make config. Here is a typical script I use to capture program flow.
That looks like something that would be useful to figure out how to do. The only part that's not immediately clear is "rebuild the entire ALSA library and tools with debug symbols. Also, your Kernel must have debug symbols turned on."
Can I do this with RCN's kernel build script setup?
Thanks.
yeah, double check your kernel's version (uname -r) grab the "yakbuild"
https://github.com/RobertCNelson/yakbuild
follow the readme.md, picking a version of gcc and setting the kernel tag
you want to build..
Run:
./build_kernel.sh
when menuconfig pop's up, enable CONFIG_DEBUG_INFO...
Kernel Hacking -> Compile-Time checks and compiler options -> [x] Compile
the kernel with debug info "CONFIG_DEBUG_INFO"
Regards,
I did this years ago. From what I can find, I believe this is how you get this done:
sudo apt-get build-dep libasound2
apt-get source libasound2
cd alsa-lib-1.0.22/
export DEB_BUILD_OPTIONS=nostrip,noopt
dpkg-buildpackage -rfakeroot -uc -us
After that, you will have some *.deb files created, which can be installed using:
dpkg -i libasound*.deb
The you have to do the same with alsa-tools.
Regards,
John
Reviewing my previous work, the two package you need are:
alsa-lib
alsa-utils
apt-get source alsa-utils
cd alsa-utils-1.0.25
export DEB_BUILD_OPTIONS=“nostrip noopt debug"
dpkg-buildpackage -rfakeroot -uc -us
make
make install
or
dpkg -i libasound2*.deb
or
apt-get install libasound2-dbg
apt-get install libasound2.dev
Anyway, as I said I did this a long time ago, so some of this could be wrong.
Regards,
John