Kernel-hacking workflow with RobertCNelson's linux-dev

Hi all,

I used to tweak and compile the kernel for my beaglebone in a very old fashioned way I guess. I downloaded the source, tweaked it, compiled them (make … uImage) and copied the generated uImage over to the boot partition of the SD card and was good to go.

Yesterday I tried to do it the “GIT way” and got stuck pretty bad. Using Robert C Nelson’s linux-dev repo I can compile and deploy his kernel, but I have no clue how I make my changes to kernel sources permanent.

Here’s what I did:

  1. cloned RobertCNelson’s linux-dev repo
  2. checked out the am33x-v3.2 branch
  3. cloned the linux-stable.git
  4. edited the system.sh file in linux-dev/ defining the cross compiler, LINUX_GIT, ZRELADDR and MMC
  5. executed build_kernel.sh → KERNEL directory is built
  6. edited the kernel sources in the generated KERNEL directory in linux-dev
  7. added and committed my changes inside the KERNEL directory
  8. executed the build_kernel.sh script

Obviously this script applies all patches from the linux-dev repo to the stable kernel and thus generates new kernel sources - overwriting all changes a just made.

Can someone outline the intended workflow please? I figure that this is a rooky question, but I really couldn’t figure it out.

Thanks,
Chris

Step 8 was your mistake.. Run "./tools/rebuild.sh" instead....
"./build_kernel.sh" will do everything possible to nuke the KERNEL
directory and setup the tree just as specified in patch.sh..
"./tools/rebuild.sh" will only 'rebuild' what is in the KERNEL
directory, thus allowing you to work on your new patch.. (btw, before
running ./tools/rebuild.sh any changes to 'patches/defconfig' will
also be ignored and reset, if your looking to also change the config
options edit "KERNEL/.config" first or via menuconfig when
./tools/rebuild.sh is ran..)

I updated/cleaned-up the readme late last week with more details on
this so it should help users going forward..

Regards,

Robert Nelson <robertcnelson@gmail.com> [12-10-18 18:04]:

> Hi all,
>
> I used to tweak and compile the kernel for my beaglebone in a very old
> fashioned way I guess. I downloaded the source, tweaked it, compiled them
> (make .... uImage) and copied the generated uImage over to the boot
> partition of the SD card and was good to go.
>
> Yesterday I tried to do it the "GIT way" and got stuck pretty bad. Using
> Robert C Nelson's linux-dev repo I can compile and deploy his kernel, but I
> have no clue how I make my changes to kernel sources permanent.
>
> Here's what I did:
>
> 1) cloned RobertCNelson's linux-dev repo
> 2) checked out the am33x-v3.2 branch
> 3) cloned the linux-stable.git
> 4) edited the system.sh file in linux-dev/ defining the cross compiler,
> LINUX_GIT, ZRELADDR and MMC
> 5) executed build_kernel.sh -> KERNEL directory is built
> 6) edited the kernel sources in the generated KERNEL directory in linux-dev
> 7) added and committed my changes inside the KERNEL directory

> 8) executed the build_kernel.sh script
Step 8 was your mistake.. Run "./tools/rebuild.sh" instead....
"./build_kernel.sh" will do everything possible to nuke the KERNEL
directory and setup the tree just as specified in patch.sh..
"./tools/rebuild.sh" will only 'rebuild' what is in the KERNEL
directory, thus allowing you to work on your new patch.. (btw, before
running ./tools/rebuild.sh any changes to 'patches/defconfig' will
also be ignored and reset, if your looking to also change the config
options edit "KERNEL/.config" first or via menuconfig when
./tools/rebuild.sh is ran..)

I updated/cleaned-up the readme late last week with more details on
this so it should help users going forward..

Regards,

--
Robert Nelson
http://www.rcn-ee.com/

--

Hi,

...that means: everytime I update my copy with git pull I need to
run build_kernel.sh instead of tools/rebuild.sh to insert the updated
files (and all other) in KERNEL?

Is this correct?

Best regards,
mcc

Hi,

...that means: everytime I update my copy with git pull I need to
run build_kernel.sh instead of tools/rebuild.sh to insert the updated
files (and all other) in KERNEL?

Is this correct?

Correct... ./tools/rebuild.sh only operates inside KERNEL ignoring
everything outside..

Regards,

Hi Chris,

This is my workflow for making customizations:

  1. cloned RobertCNelson’s linux-dev repo

  2. checked out the am33x-v3.2 branch

  3. cloned the linux-stable.git

  4. edited the system.sh file in linux-dev/ defining the cross compiler, LINUX_GIT, ZRELADDR and MMC

  5. executed build_kernel.sh → KERNEL directory is built

  6. create a new local branch that contains your customizations
    cd KERNEL
    git branch #tell you what branch you are in, let’s say it is v3.2-psp24
    git branch chris-v3.2-psp24 #create new branch with the name chris-v3.2-psp24
    git checkout chris-v3.2-psp24

  7. edit the kernel source files, for example:
    vi arch/arm/mach-omap2/board-am335xevm.c

  8. rebuild kernel
    in linux-dev, run $ tools/rebuild_deb.sh

  9. if the customization is good, commit the changes
    (in KERNEL directory)
    git status #tell you which files you have changed
    git commit -a #commit those changes with a 1-liner to say what you have done
    git log #verify that your latest commit is there

At this point I would also create a patch (in the dir above linux-dev) of my customizations. This is useful for sharing with collaborators or replaying the changes on a new git repo.

  1. create a patch for future kernels
    git format-patch v3.2-psp24 --stdout > …/…/mycustom.patch24

For example, when Robert releases psp25:
11) in linux-dev, $ git pull
12) build_kernel.sh to get the “off-the-shelf” kernel image built
13) create a local branch for psp25 in KERNEL
cd KERNEL
git branch chris-v3.2-psp25
git checkout chris-v3.2-psp25
14) apply your customization patch
patch -p1 < …/…/mycustom.patch24
15) rebuild kernel
in linux-dev, run $ tools/rebuild_deb.sh

Also want to say a big “THANK YOU” to Robert for his contribution that makes Ubuntu easy to work with on beaglebone and beagleboard.

regards,
George

Thanks a lot! That really helps me out!

However, I now have another problem that’s probably to do with the .config:

When I run rebuild.sh on my modified kernel I get the error:
arch/arm/mach-versatile/core.c:42:22: fatal error: asm/leds.h: No such file or directory

When I looked up this error I found a post of yours in this group saying:
" It looks like you forgot to copy the defconfig
from patches/beagleboard/defconfig and save it as .config in kernel
directory right… "

As I’m compiling not for beagleboard but for beaglebone I looked in patches/beaglebone, but no defconfig there. So I tried all config files I found in the patches directory (current_defconfig, defconfig, ref_omap2plus_defconfig) - but not luck.
I still get either the mach-versatile/core.c error or errors in mach-omap2/board-omap3evm.c.

So I figured that I messed up the kernel sources somehow and ran rebuild_kernel.sh again. But oddly this also terminates with the mach-versatile/core.c error!?

What is it that I’m missing?

Thanks,
Chris

Thanks a lot! That really helps me out!

However, I now have another problem that's probably to do with the .config:

When I run rebuild.sh on my modified kernel I get the error:
arch/arm/mach-versatile/core.c:42:22: fatal error: asm/leds.h: No such file
or directory

"mach-versatile" ? you really messed up the .config...

When I looked up this error I found a post of yours in this group saying:
" It looks like you forgot to copy the defconfig
from patches/beagleboard/defconfig and save it as .config in kernel
directory right.. "

That's a completely different git repo... The config being used in
"linux-dev" is patches/defconfig

As I'm compiling not for beagleboard but for beaglebone I looked in
patches/beaglebone, but no defconfig there. So I tried all config files I
found in the patches directory (current_defconfig, defconfig,
ref_omap2plus_defconfig) - but not luck.

Well, "origin/master" doesn't support the Bone, as it's more of a
staging ground for my other repo "stable-kernel"..

For the Bone use the: "am33x-v3.2" or "am33x-v3.6" branches..

I still get either the mach-versatile/core.c error or errors in
mach-omap2/board-omap3evm.c.

So I figured that I messed up the kernel sources somehow and ran
rebuild_kernel.sh again. But oddly this also terminates with the
mach-versatile/core.c error!?

What is it that I'm missing?

Yeah, it sounds like you did..

git checkout master -f
git reset HEAD --hard
git pull

Then for the Bone:

git checkout origin/am33x-v3.2 -b tmp-am33x-v3.2
or
git checkout origin/am33x-v3.6 -b tmp-am33x-v3.6

and run "./build_kernel.sh" to populate KERNEL..

Regards,

I was on am33x-v3.2 but something strange must have happened to the config.
Anyway, I restored the kernel the way you suggested and applied the same modifications to the kernel as before.
Now it compiles like a charm…

Thanks very much!

Pardon my newbie-ness. I cannot find

arch/arm/mach-omap2/board-am335xevm.c

anywhere. The most closely named file I see is “arch/arm/mach-omap2/board-am3517evm.c”, in either the source for stable 3.6.3 from kernel.org, or 3.2.17 source. I have done a ‘git clone’ of RCNelson’s patches and scripts and have that available.

This relates to this post of mine from a short while back:
https://groups.google.com/forum/#!topic/beagleboard/Iyy4MF55zj4

My goal is to configure the kernel to enable the serial port using UART 4, not the default of UART 2, which will be needed for something else.

What have I misunderstood?

Thanks

You'll never find "arch/arm/mach-omap2/board-am335xevm.c" in the pure
mainline kernel source..

You'll find it in the v3.2.x community kernel patchset..

Regards,

You’ll never find “arch/arm/mach-omap2/board-am335xevm.c” in the pure
mainline kernel source…

You’ll find it in the v3.2.x community kernel patchset…

Again, pardon my ignorance, but I have no idea where this is. Looking in the obvious (to me) places, and Googling has not helped so far.

Thanks,

Mickey

Since you've cloned my linux-dev tree, it's under:

git checkout origin/am33x-v3.2 -b am33x-v3.2-tmp
./build_kernel.sh

(then under KERNEL directory)

or, from:

https://github.com/beagleboard/kernel

git checkout origin/beaglebone-3.2 -b beaglebone-3.2
./patch.sh

(then under kernel directory)

Regards,

Since you’ve cloned my linux-dev tree, it’s under:

git checkout origin/am33x-v3.2 -b am33x-v3.2-tmp
./build_kernel.sh

(then under KERNEL directory)

or, from:

https://github.com/beagleboard/kernel

git checkout origin/beaglebone-3.2 -b beaglebone-3.2
./patch.sh

(then under kernel directory)

Still struggling here I’m afraid.

Made sure cross compiler was Installed by running :
sudo apt-get install gcc-arm-linux-gnueabihf

Edited system.sh, pointed to my local copy of linux-src, and set CC=gcc-arm-linux-gnueabihf, as I understood from the README. If all I want to do at the moment is create a set of kernel source files to study and understand how to tweak the UARTS, I should be able to ignore ZRELADDR and MMC for now? When I am ready to actually create an image those will matter - correct?

When I try to run build_kernel.sh I get a complaint about the CC variable, and “gcc-arm-linux-gnueabihfgcc: command not found” Not sure where that gcc on the end is from.

I’m new to git and don’t know it’s subtleties yet. Trying to “git checkout origin/beaglebone-3.2 -b beaglebone-3.2” gets me an error message about “updating paths is incompatible with switching branches”

I’m thinking that I should start from scratch with a clean system, and that knowing the exact commands used for steps 1 through 3 (in particular) of George Lu’s process above would help.

Thanks in advance to all.

Since you've cloned my linux-dev tree, it's under:

git checkout origin/am33x-v3.2 -b am33x-v3.2-tmp
./build_kernel.sh

(then under KERNEL directory)

or, from:

https://github.com/beagleboard/kernel

git checkout origin/beaglebone-3.2 -b beaglebone-3.2
./patch.sh

(then under kernel directory)

Still struggling here I'm afraid.

Made sure cross compiler was Installed by running :
sudo apt-get install gcc-arm-linux-gnueabihf

Edited system.sh, pointed to my local copy of linux-src, and set
CC=gcc-arm-linux-gnueabihf, as I understood from the README. If all I want

In that case you would use:
CC=arm-linux-gnueabihf-

to do at the moment is create a set of kernel source files to study and
understand how to tweak the UARTS, I should be able to ignore ZRELADDR and
MMC for now? When I am ready to actually create an image those will matter -
correct?

Correct, hence they are listed under "OPTIONAL:"

When I try to run build_kernel.sh I get a complaint about the CC variable,
and "gcc-arm-linux-gnueabihfgcc: command not found" Not sure where that gcc
on the end is from.

It's just detected your CC was inproper..

I'm new to git and don't know it's subtleties yet. Trying to "git checkout
origin/beaglebone-3.2 -b beaglebone-3.2" gets me an error message about
"updating paths is incompatible with switching branches"

Your not going to find that in the linux-dev repo, it's in:
https://github.com/beagleboard/kernel

Regards,

btw: This "CC" hint is now in the README as of sometime last week.. :wink:
It's a common-ish problem for new users..

https://github.com/RobertCNelson/linux-dev/blob/master/README#L33

Regards,

In that case you would use:

CC=arm-linux-gnueabihf-

Aha. apt-get wants “gcc-arm-linux-gnueabihf” but the compiler as actually installed is arm-linux-gnueabihf-gcc, and the trailing hyphen is needed in system.sh.

Couple of speed bumps still:

I ran build_kernel.sh, and it started patch.sh which got pretty far, but then :

“fatal: Unable to create ‘/home/mfeldman/stable-kernel/KERNEL/.git/index.lock’: File exists.
If no other git process is currently running, this probably means a
git process crashed in this repository earlier. Make sure no other git
process is running and remove the file manually to continue.”

I can well believe I crashed git earlier, but there was no index.lock to be found.
Re-ran it, since there was no lock. Got as far as “Your display is too small to run Menuconfig!”
Enlarged terminal window & re-ran.
Finally ran to completion, creating …/deploy and apparently updating files in KERNEL/arch/arm/mach-omap2, but still no sign of “board-am335xevm.c”.

Guessing that I still don’t have the required source files, I try

git clone https://github.com/beagleboard/kernel
cd kernel
./patch.sh

Got as far as "cannot access blah…blah…/kernel/export/dirs/*.patch: No such file or directory

I feel like I’m missing some basic concept here. I’m understanding (rightly or wrongly) that the main branch source files get patched to make a Beagle-specific set, but it’s not 100% clear to me which files I need to start with (or if there are alternatives) or where they need to come from. I now have a couple of different sets of kernel files, none of which has the particular one I think I care about.

I’m ready to blow them away and start from square one, but I’m not sure where square one is.

Then your still on "master" switch to the am33x-v3.2 branch as
mentioned earlier:

git checkout origin/am33x-v3.2 -b am33x-v3.2-tmp

Regards,

Then your still on “master” switch to the am33x-v3.2 branch as

mentioned earlier:

git checkout origin/am33x-v3.2 -b am33x-v3.2-tmp

This generates the same error I had earlier with this. I don’t understand git enough to get around it.

“fatal: git checkout: updating paths is incompatible with switching branches.
Did you intend to checkout ‘origin/am33x-v3.2’ which can not be resolved as commit?”

First are you on master?
voodoo@a53t:/opt/github/linux-dev$ git branch
* master

if not:
git checkout master -f

2nd, do the branches show up? (hit tab after "origin/" to show options)
voodoo@a53t:/opt/github/linux-dev$ git checkout origin/
origin/am33x-v3.1 origin/am33x-v3.6 origin/master
origin/v2.6.39.x origin/v3.1.x
origin/am33x-v3.2 origin/HEAD origin/v2.6.38.x
origin/v3.0.x origin/v3.3.x

if not:
git pull --all
git fetch --all

then:
voodoo@a53t:/opt/github/linux-dev$ git checkout origin/am33x-v3.2 -b am33x-v3.2
Branch am33x-v3.2 set up to track remote branch am33x-v3.2 from origin.
Switched to a new branch 'am33x-v3.2'

Beyond that, i have no idea.. either you system is just old, or some
other random thing..

Regards,

First are you on master?
voodoo@a53t:/opt/github/linux-dev$ git branch

  • master

if not:
git checkout master -f

2nd, do the branches show up? (hit tab after “origin/” to show options)
voodoo@a53t:/opt/github/linux-dev$ git checkout origin/
origin/am33x-v3.1 origin/am33x-v3.6 origin/master
origin/v2.6.39.x origin/v3.1.x
origin/am33x-v3.2 origin/HEAD origin/v2.6.38.x
origin/v3.0.x origin/v3.3.x

if not:
git pull --all
git fetch --all

then:
voodoo@a53t:/opt/github/linux-dev$ git checkout origin/am33x-v3.2 -b am33x-v3.2
Branch am33x-v3.2 set up to track remote branch am33x-v3.2 from origin.
Switched to a new branch ‘am33x-v3.2’

Beyond that, i have no idea… either you system is just old, or some
other random thing…

git branch did show that I was on master. I did a pull --all, and a fetch --all.

git checkout origin/ showed a number of branches, but am33x-v3.2 was not among them. What I did see was:

origin/3.0.x-rt2870sta origin/v2.6.37.x origin/v3.2-camera origin/v3.4.x
origin/HEAD origin/v2.6.38.x origin/v3.2-merge origin/v3.5.x
origin/master origin/v2.6.39.x origin/v3.2-spitest origin/v3.6.x
origin/sgx_4.05.00.03 origin/v3.0.x origin/v3.3-merge
origin/v2.6.35.x origin/v3.1.x origin/v3.3.x

…just in case this offers some clues to what I’m beginning to suspect I may have initially checked out incorrectly…