Custom kernel (smaller) takes longer to uncompress?

I’ve noticed something strange, and I’m hoping there is a simple explaination for what I am seeing. If I take the default image from beaglebone on Angstrom’s website, it takes about 0.3 seconds to uncompress the image during boot. However, when I create a custom kernel which is actually a little smaller because I pulled some of the configurations out that I’m not using, and I use a different boot logo, it takes over 3 seconds to uncompress during boot. What is going on? Why does it take so much longer to uncompress?

I do notice in the rootfs/boot folder there are the following files:
config-3.2.34
Module.symvers-3.2.34
System.map-3.2.34

Do these help speed up the uncompression during loading of the kernel?

Ok, so I need to do a little more explaining here… I was using a newer version of u-boot that I had customized and u-boot takes much longer to copy and uncompress the kernel image in v2013.01 than it did in v2011.09. The default image from beaglebone is using v2011.09 and it is much quicker than the newer stuff. Not sure why yet, but I need to look into this further.