new beaglebone image-builder script fails when downloading Cloud9

Dennis_Cote · January 15, 2014, 7:00pm

It looks like xz just died, it should remove
“BBB-eMMC-flasher-debian-7.3-2014-01-14-2gb.img” when done…

That explains what happened to the debian-7.3-lxde-armhf-2014-01-14.tar file that ship.sh is looking for to start with.

I changed that this week, seemed kinda silly calling it “console” when
obviously an ‘lxde’ image…

Good to know. That difference is explained away.

Where does the armhf… file come from? I tried tracing through the build
scripts, but I couldn’t see where it was created.

The “armhf” is the name of the debian port we are using…

https://wiki.debian.org/ArmHardFloatPort

Yes, I understand what “armhf” means (after I looked it up a few days ago), but I was asking where does the file armhf-rootfs-debian-wheezy-2014-01-14-src.tar come from? I couldn’t see where it was created in the scripts.

Thanks again.

Dennis Cote

RobertCNelson · January 15, 2014, 7:15pm

Yes, I understand what "armhf" means (after I looked it up a few days ago),
but I was asking where does the file
armhf-rootfs-debian-wheezy-2014-01-14-src.tar come from? I couldn't see
where it was created in the scripts.

Oh that, it gets dumped out here:

github.com

beagleboard/image-builder/blob/master/scripts/chroot.sh#L719


      
          	dpkg_check () {
          		unset pkg_is_not_installed
          		LC_ALL=C dpkg --list | awk '{print \$2}' | grep "^\${pkg}$" >/dev/null || pkg_is_not_installed="true"
          	}
          
          	dpkg_package_missing () {
          		echo "Log: (chroot) package [\${pkg}] was not installed... (add to deb_include if functionality is really needed)"
          	}
          
          	is_this_qemu () {
          		unset warn_qemu_will_fail
          		if [ -f /usr/bin/qemu-arm-static ] ; then
          			warn_qemu_will_fail=1
          		fi
          		if [ -f /usr/bin/qemu-aarch64-static ] ; then
          			warn_qemu_will_fail=1
          		fi
          		if [ -f /usr/bin/qemu-riscv64-static ] ; then
          			warn_qemu_will_fail=1
          		fi
          	}

just un-comment the "chroot_ENABLE_DEB_SRC=enable" line..

Regards,

Dennis_Cote · January 15, 2014, 7:58pm

Thanks, that did it (after a little copying and editing to update my existing ship.sh script without rebuilding everything).

I now have what appear to be a complete set of image files.

Dennis Cote

Dennis_Cote · January 15, 2014, 8:00pm

Is it used for anything in this build, or is it simply an extraneous file that doesn’t need to be produced?

Dennis Cote

RobertCNelson · January 15, 2014, 8:06pm

It serves only one purpose. If someone was to ask you.

Do you have the source to package xyz which was installed in release_xyz?

You'd be able to give it to them..

It's just a tar file with the source package for everything that was installed.

Regards,

Dennis_Cote · January 15, 2014, 8:17pm

I see, *-src.tar now its clear.

Thanks again for all your help.

Dennis Cote

Dennis_Cote · January 15, 2014, 11:45pm

Still no joy.

I am getting an error when writing the 4GB image to a 4 GB SD card.

root@dennis-VirtualBox:/home/dennis/BBB/image-builder/deploy# xz -cd bone-debian-7.3-2014-01-14-4gb.img.xz > /dev/sdd
xz: (stdout): Write error: No space left on device

I expanded the image to check its size and it comes back as 3.7GB which should fit.

So I then unmounted the two partitions and copied the image file to the SD card (/dev/sdd) again. This also failed

root@dennis-VirtualBox:/home/dennis/BBB/image-builder/deploy# cp bone-debian-7.3-2014-01-14-4gb.img /dev/sdd
cp: writing ‘/dev/sdd’: No space left on device
cp: failed to extend ‘/dev/sdd’: No space left on device

I then removed and reinserted the SD card. It mounted the BOOT partition on /dev/sdd1, but failed to mount the rootfs partition from /dev/sdd2 with the following error:

Error mounting /dev/sdd2 at /media/dennis/rootfs: Command-line `mount -t “ext4” -o “uhelper=udisks2,nodev,nosuid” “/dev/sdd2” “/media/dennis/rootfs”’ exited with non-zero exit status 32: mount: wrong fs type, bad option, bad superblock on /dev/sdd2

Next I checked the size of the SD card.

dennis@dennis-VirtualBox:~$ cat /sys/block/sdd/size
7626752

I believe this is units of 512B blocks. This corresponds to 3.904 GB or 3.637 GiB.

The image file size is:

root@dennis-VirtualBox:/home/dennis/BBB/image-builder/deploy# ls -l bone-debian-7.3-2014-01-14-4gb.img
-rw-r–r-- 1 dennis dennis 3932160000 Jan 15 12:26 bone-debian-7.3-2014-01-14-4gb.img

This size is 3.932 GB or 3.662 GiB (i.e. the 3.7GB reported earlier).

Therefore this image is slightly too large for my nominal 4GB SD card.

Can this image be resized so that it is small enough to fit?

I know that the armhf.com images are 2 GB. These can be copied to a larger SD card and then fdisk is used to resize the second partition to fill the SD card, and finally the filesystem is resized to match the new partition size. This seems like a better approach than arbitrarily reducing the image size to fit on undersized SD cards, though it is more complicated.

Dennis Cote

RobertCNelson · January 16, 2014, 12:10am

Still no joy.

I am getting an error when writing the 4GB image to a 4 GB SD card.

root@dennis-VirtualBox:/home/dennis/BBB/image-builder/deploy# xz -cd
bone-debian-7.3-2014-01-14-4gb.img.xz > /dev/sdd
xz: (stdout): Write error: No space left on device

I expanded the image to check its size and it comes back as 3.7GB which
should fit.

So I then unmounted the two partitions and copied the image file to the SD
card (/dev/sdd) again. This also failed

root@dennis-VirtualBox:/home/dennis/BBB/image-builder/deploy# cp
bone-debian-7.3-2014-01-14-4gb.img /dev/sdd
cp: writing ‘/dev/sdd’: No space left on device
cp: failed to extend ‘/dev/sdd’: No space left on device

I then removed and reinserted the SD card. It mounted the BOOT partition on
/dev/sdd1, but failed to mount the rootfs partition from /dev/sdd2 with the
following error:

Error mounting /dev/sdd2 at /media/dennis/rootfs: Command-line `mount -t
"ext4" -o "uhelper=udisks2,nodev,nosuid" "/dev/sdd2" "/media/dennis/rootfs"'
exited with non-zero exit status 32: mount: wrong fs type, bad option, bad
superblock on /dev/sdd2

Next I checked the size of the SD card.

how about just plain old:

sudo dd if=./bone-debian-7.3-2014-01-14-4gb.img of=/dev/sdd

dennis@dennis-VirtualBox:~$ cat /sys/block/sdd/size
7626752

I believe this is units of 512B blocks. This corresponds to 3.904 GB or
3.637 GiB.

The image file size is:

root@dennis-VirtualBox:/home/dennis/BBB/image-builder/deploy# ls -l
bone-debian-7.3-2014-01-14-4gb.img
-rw-r--r-- 1 dennis dennis 3932160000 Jan 15 12:26
bone-debian-7.3-2014-01-14-4gb.img

This size is 3.932 GB or 3.662 GiB (i.e. the 3.7GB reported earlier).

Other than the fact i HATE debugging VMware/VirtualBox with a
passion.. What brand are these 4GB microSD cards?

The magic number I've been using is 3750, which is working on
SanDisk/Kingston/Samsung 4GB

github.com

RobertCNelson/omap-image-builder/blob/master/tools/setup_sdcard.sh#L1540


      
          				echo "#connmanctl> enable wifi" >> ${wfile}
          				echo "#connmanctl> scan wifi" >> ${wfile}
          				echo "#connmanctl> services" >> ${wfile}
          				echo "#connmanctl> agent on" >> ${wfile}
          				echo "#connmanctl> connect wifi_*_managed_psk" >> ${wfile}
          				echo "#connmanctl> quit" >> ${wfile}
          			fi
          		fi
          		fi
          		fi
          
          		if [ -f ${TEMPDIR}/disk/var/www/index.html ] ; then
          			rm -f ${TEMPDIR}/disk/var/www/index.html || true
          		fi
          
          		if [ -f ${TEMPDIR}/disk/var/www/html/index.html ] ; then
          			rm -f ${TEMPDIR}/disk/var/www/html/index.html || true
          		fi
          		sync
          
          	fi #RootStock-NG

Therefore this image is slightly too large for my nominal 4GB SD card.

Can this image be resized so that it is small enough to fit?

I know that the armhf.com images are 2 GB. These can be copied to a larger
SD card and then fdisk is used to resize the second partition to fill the SD
card, and finally the filesystem is resized to match the new partition size.
This seems like a better approach than arbitrarily reducing the image size
to fit on undersized SD cards, though it is more complicated.

If you want a 2Gb image, pass the "--img" option vs the "img-4gb"
option to setup_sdcard.sh..

Regards,

William_Hermans · January 16, 2014, 9:38am

Whats all this talk of “debugging vmware/virtualbox” all about ?

Dennis_Cote · January 16, 2014, 4:06pm

how about just plain old:

sudo dd if=./bone-debian-7.3-2014-01-14-4gb.img of=/dev/sdd

I can try but I doubt if it will be any different than cp or xz -cd >.

dennis@dennis-VirtualBox:~$ cat /sys/block/sdd/size
7626752

I believe this is units of 512B blocks. This corresponds to 3.904 GB or
3.637 GiB.

The image file size is:

root@dennis-VirtualBox:/home/dennis/BBB/image-builder/deploy# ls -l
bone-debian-7.3-2014-01-14-4gb.img
-rw-r–r-- 1 dennis dennis 3932160000 Jan 15 12:26
bone-debian-7.3-2014-01-14-4gb.img

This size is 3.932 GB or 3.662 GiB (i.e. the 3.7GB reported earlier).

Other than the fact i HATE debugging VMware/VirtualBox with a
passion… What brand are these 4GB microSD cards?

Kingston Technology.

To rule out virtualbox’s involvment I inserted the same SD card into my BBB and checked its size as reported by Angstrom linux.

root@beaglebone:~# cat /sys/block/mmcblk1/size
7626752

As you can see it reports exactly the same size.

The magic number I’ve been using is 3750, which is working on
SanDisk/Kingston/Samsung 4GB
https://github.com/RobertCNelson/omap-image-builder/blob/master/tools/setup_sdcard.sh#L1540

Yes, 375010241024 is 3.932 GB, the size of the image file.

Reducing this to 3750 * 3.904/3.932 or 3723 would make it fit this SD card. I would suggest reducing it slightly more, to 3700 to allow for other smaller cards.

I know that the armhf.com images are 2 GB. These can be copied to a larger
SD card and then fdisk is used to resize the second partition to fill the SD
card, and finally the filesystem is resized to match the new partition size.
This seems like a better approach than arbitrarily reducing the image size
to fit on undersized SD cards, though it is more complicated.

If you want a 2Gb image, pass the “–img” option vs the “img-4gb”
option to setup_sdcard.sh…

I think I’ll do that. I will also look at creating a script to automate the expansion process so that the result automatically fills the SD card to its full capacity, regardless of its exact size.

Thanks again.

Dennis Cote

RobertCNelson · January 16, 2014, 4:22pm

The magic number I've been using is 3750, which is working on
SanDisk/Kingston/Samsung 4GB

omap-image-builder/tools/setup_sdcard.sh at master · RobertCNelson/omap-image-builder · GitHub

Yes, 3750*1024*1024 is 3.932 GB, the size of the image file.

Reducing this to 3750 * 3.904/3.932 or 3723 would make it fit this SD card.
I would suggest reducing it slightly more, to 3700 to allow for other
smaller cards.

Okay, just dropped it to 3700..

> I know that the armhf.com images are 2 GB. These can be copied to a
> larger
> SD card and then fdisk is used to resize the second partition to fill
> the SD
> card, and finally the filesystem is resized to match the new partition
> size.
> This seems like a better approach than arbitrarily reducing the image
> size
> to fit on undersized SD cards, though it is more complicated.

If you want a 2Gb image, pass the "--img" option vs the "img-4gb"
option to setup_sdcard.sh..

I think I'll do that. I will also look at creating a script to automate the
expansion process so that the result automatically fills the SD card to its
full capacity, regardless of its exact size.

What i'd really like to see is a script that can be run on target to
automatically re-size it, even on bootup...

Because, if your already running linux, instead of resizing the *.img
for your microSD, you can just already call "--mmc /dev/sdX" with
setup_sdcard.sh and it'll auto partition your microSD using all the
available space.

Of course the "--img" option allows you to easily share a completed
image that doesn't need network access to pull in the extra bits
setup_sdcard.sh..

Regards,

Dennis_Cote · January 16, 2014, 10:04pm

Okay, just dropped it to 3700…

https://github.com/beagleboard/image-builder/commit/b6288e9ebbe3a688b4c9623ce3c94bfdb9e5e305

Success… finally.

It took a while to figure out ship.sh was using the setup_sdcard.sh inside the image file. I had pulled the new version into the image_builder dir, but I didn’t rerun the entire build. I was just rerunning the ship.sh script. After editing the setup_sdcard.sh in the image it worked.

I have booted my BBB using the new 4GB wheezy SD card. Everything seems to be working as expected.

What i’d really like to see is a script that can be run on target to
automatically re-size it, even on bootup…

Because, if your already running linux, instead of resizing the *.img
for your microSD, you can just already call “–mmc /dev/sdX” with
setup_sdcard.sh and it’ll auto partition your microSD using all the
available space.

Of course the “–img” option allows you to easily share a completed
image that doesn’t need network access to pull in the extra bits
setup_sdcard.sh…

I was thinking more along the lines of always building a standard image the exact size needed for the BBB eMMC (i.e. 2 GB) since they are all the same size. Then using a minimal system and a compressed copy of this standard image to build the eMMC_flasher image which I think would also fit into 2 GB. The same compressed standard image could be distributed for use with a small script that would expand the image onto an SD card of any larger size (i.e. 4 GB, 8 GB, 16 GB, etc) and then expand the partition and filesystem to match the size of the SD card.

This would work well for linux host systems, but perhaps not so well for Windows or OS X hosts. So perhaps the larger images will still be needed for these systems.

I noticed that the ship.sh script seems to build each image from scratch rather than basing one image on the other (i.e. it calls setup_sdcard.sh once for each image). The only thing I saw that is done for the eMMC_flasher image is the creation of a file called flash-eMMC.txt. It wasn’t obvious to me how that file is used. I assume the startup code looks for that file and runs the flashing commands if it’s found as the system boots the flasher image.

Are there other differences between a normal image (say the 4GB image) and a eMMC_flasher image as they are currently being built?

Dennis Cote

RobertCNelson · January 16, 2014, 10:27pm

Success... finally.

That's good..

It took a while to figure out ship.sh was using the setup_sdcard.sh inside
the image file. I had pulled the new version into the image_builder dir, but
I didn't rerun the entire build. I was just rerunning the ship.sh script.
After editing the setup_sdcard.sh in the image it worked.

Yeah, it's just years of bolt-on's.. I originally wrote
setup_sdcard.sh to automate the task of installing the output of
"project-rootstock". Then i needed to manage what options we called
"project-rootstock" with. Eventually re-writing 'project-rootstock'..
Then finally adding the ship.sh file as i couldn't remember what i
shipped the month before.

It was much easier when i just pushed out:
"debian-7.3-lxde-armhf-2014-01-16.tar.xz" and made everyone run
"setup_sdcard.sh"...

What i'd really like to see is a script that can be run on target to
automatically re-size it, even on bootup...

Because, if your already running linux, instead of resizing the *.img
for your microSD, you can just already call "--mmc /dev/sdX" with
setup_sdcard.sh and it'll auto partition your microSD using all the
available space.

Of course the "--img" option allows you to easily share a completed
image that doesn't need network access to pull in the extra bits
setup_sdcard.sh..

I was thinking more along the lines of always building a standard image the
exact size needed for the BBB eMMC (i.e. 2 GB) since they are all the same
size. Then using a minimal system and a compressed copy of this standard
image to build the eMMC_flasher image which I think would also fit into 2
GB. The same compressed standard image could be distributed for use with a
small script that would expand the image onto an SD card of any larger size
(i.e. 4 GB, 8 GB, 16 GB, etc) and then expand the partition and filesystem
to match the size of the SD card.

If you consider "armhf-rootfs-debian-wheezy.tar" to be the "minimal
system" you've just described what "setup_sdcard.sh --mmc /dev/sdX"
does already.. But i'm not going to stop you from writing a script to
dump the *.img and then expanding it..

I noticed that the ship.sh script seems to build each image from scratch
rather than basing one image on the other (i.e. it calls setup_sdcard.sh
once for each image). The only thing I saw that is done for the eMMC_flasher
image is the creation of a file called flash-eMMC.txt. It wasn't obvious to
me how that file is used. I assume the startup code looks for that file and
runs the flashing commands if it's found as the system boots the flasher
image.

Are there other differences between a normal image (say the 4GB image) and a
eMMC_flasher image as they are currently being built?

I've limited the flasher to 2GB in size.. No sense in flashing more then 2GB's..

The extra flash file triggers this call in the board init script:

github.com

RobertCNelson/boot-scripts/blob/master/boot/am335x_evm.sh#L76


      
          detected_capes=$(cat /proc/cmdline | sed 's/ /\n/g' | grep uboot_detected_capes= || true)
          if [ ! "x${detected_capes}" = "x" ] ; then
          	got_DLPDLCR2000=$(echo ${detected_capes} | grep DLPDLCR2000 || true)
          	if [ ! "x${got_DLPDLCR2000}" = "x" ] ; then
          		echo "${log} found: DLPDLCR2000 init display"
          		i2cset -y 2 0x1b 0x0b 0x00 0x00 0x00 0x00 i || true
          		i2cset -y 2 0x1b 0x0c 0x00 0x00 0x00 0x1b i || true
          	fi
          fi
          
          usb_gadget="/sys/kernel/config/usb_gadget"
          
          #  idVendor           0x1d6b Linux Foundation
          #  idProduct          0x0104 Multifunction Composite Gadget
          #  bcdDevice            4.04
          #  bcdUSB               2.00
          
          usb_idVendor="0x1d6b"
          usb_idProduct="0x0104"
          usb_bcdDevice="0x0404"
          usb_bcdUSB="0x0200"

Regards,