Setting up TFTP and NFS

GoogleGroups
1

I am attempting to setup a BBB to boot via TFTP and mount a rootfs via NFS.

I am using Robert Nelson’s 2014-08-05 Debian lxde release image. I’ve examined the nfs-uEnv.txt in the boot partition and had a brief look at how the various environment variables interact with the default uboot setup. With the latest BBB uboot setup, it appears that TFTP boot and NFS rootfs functionality are now “packaged” together.

I’ve also read William Herman’s guides to setting up to boot via TFTP and mount the rootfs using NFS. Whilst helpful, these guides are now a bit dated (published June 2013) with respect to the latest uboot default setup.

I have setup a VirtualBox image running Linux Mint on my MAC. I have configured this image to be a NFS server and have arranged to export a rootfs (copied from a uSD card flashed with a disk image or created directly from RCN’s rootfs tarball). I have also configured TFTP on this image.

My questions:

  • Where/how do I obtain a “zImage” to boot using TFTP? Is this simply the vmlinuz-xxx image in the /boot partition of the rootfs? (Running the “file” command on a vmlinuz image reports it to be a " Linux kernel ARM boot executable zImage (little-endian)".)

  • Unlike the normal so-called “uname” boot, the nfsboot setup does not appear to use an initrd ramfs image.

  • What function does this file system provide?

  • What impact, if any, does the lack of an initrd image have on a system booted via TFTP and running a rootfs via NFS?- In this new setup, what are the best practices for updating the kernel?

  • Given this setup, is it advisable to arrange to mount a user home directory via another NFS share to support development? Or would it be better to include the user home directory as part of a single (rootfs) NFS mount to keep things simple?
    Thanks in advance!

ba

2

I am attempting to setup a BBB to boot via TFTP and mount a rootfs via NFS.

I am using Robert Nelson's 2014-08-05 Debian lxde release image. I've
examined the nfs-uEnv.txt in the boot partition and had a brief look at how
the various environment variables interact with the default uboot setup.
With the latest BBB uboot setup, it appears that TFTP boot and NFS rootfs
functionality are now "packaged" together.

I've also read William Herman's guides to setting up to boot via TFTP and
mount the rootfs using NFS. Whilst helpful, these guides are now a bit
dated (published June 2013) with respect to the latest uboot default setup.

I have setup a VirtualBox image running Linux Mint on my MAC. I have
configured this image to be a NFS server and have arranged to export a
rootfs (copied from a uSD card flashed with a disk image or created directly
from RCN's rootfs tarball). I have also configured TFTP on this image.

My questions:

Where/how do I obtain a "zImage" to boot using TFTP? Is this simply the
vmlinuz-xxx image in the /boot partition of the rootfs? (Running the "file"
command on a vmlinuz image reports it to be a " Linux kernel ARM boot
executable zImage (little-endian)".)

Correct, just rename vmlinuz as zimage and dump into your /TFTP/ directory..

zImage (u-boot name) = vmlinuz (kernel output name)

Also copy the (.dtb's) int /TFTP/dtbs/*

Unlike the normal so-called "uname" boot, the nfsboot setup does not appear
to use an initrd ramfs image.

initrd ramfs's just don't work with nfs root... It'll hard lock on
bootup.. Thus it's disabled..

What function does this file system provide?
What impact, if any, does the lack of an initrd image have on a system
booted via TFTP and running a rootfs via NFS?

In this new setup, what are the best practices for updating the kernel?
Given this setup, is it advisable to arrange to mount a user home directory
via another NFS share to support development? Or would it be better to
include the user home directory as part of a single (rootfs) NFS mount to
keep things simple?

Install like you normally do... Just make sure on the server side to
update the new vmlinuz -> zImage and *.dtb's into the /TFTP/
directory. (Since the tftp directory could be anywhere, i really
didn't write a script for the bbb to handle it..)

Regards,

3

Brian, Yeah I’ve been busy ( always am during the summer ), and have barely even thought about my blog site. uImage which I used in my blog post I suppose for this purpose is depreciated now, but without major tweaks was the only way to load the kernel over tftp back then ( as far as I know ).

Now days, I’ve lost interest in loading the kernel via tftp, and only use NFS for my rootfs, so do not expect an update for that article. Lately, I’ve been concentrating on setting up g_ether statically in the kernel to load the kernel over usbnet. But as I said above . . . I’m busy and “round-to-it’s” are few and far inbetween

. Maybe this late fall / winter . …

4

Lately, I’ve been concentrating on setting up g_ether statically in the kernel to load the kernel over usbnet.

Meant, load the root file system over usbnet.

5

@Robert - Thanks for confirming!

@William - Interesting. Can you shed some additional light on this approach? usbnet is Ethernet over USB, correct? Is this just an alternative to using the wired Ethernet jack…and thus one still is mounting the rootfs via NFS from a NAS or something like that? Or is this a different approach? How does the BBB access/mount the rootfs? Would this approach work for supporting cross development on a Linux laptop with a filesystem shared with the BBB?

I was able to download a kernel via TFTP, boot and successfully mount an NFS rootfs…sorta.

I had originally setup a rootfs on the NFS share as follows:

Setup a uSD card:

  • Created using https://rcn-ee.net/deb/testing/2014-08-05/lxde/bone-debian-7.6-lxde-armhf-2014-08-05-4gb.img.xz

  • Configured and setup on a BBB Rev C.

  • Successfully booted, run, and updated numerous times on the BBB.
    Shutdown the BBB, move the uSD card to my MAC. Then, from Linux Mint running in Virtual Box on the MAC:

  • Mount the uSD card.

  • Create a tarball from the mounted uSD card whilst sudo.

  • Untar the tarball into the exported NFS share location on Linux Mint whilst sudo.
    My assumption was that I should be able to move/copy the rootfs from the uSD card using a separate system. Note that the uSD card was NOT running a live system when I made the copies.

Unfortunately, when the BBB subsequently tries to boot using the copied rootfs, the following happens:

[…] Checking root file system…fsck from util-linux 2.20.1
fsck.ext4: No such device or address while trying to open /run/rootdev
Possibly non-existent or swap device?
fsck died with exit status 8

So, I resorted to creating a fresh rootfs using https://rcn-ee.net/deb/testing/2014-08-05/lxde/debian-7.6-lxde-armhf-2014-08-05.tar.xz.

All went fine and the system booted and I could ssh to it.

Unfortunately, this system seems to have some broken network setup. A bit of digging reveals that /etc/resolv.conf is not setup correctly:

domain localdomain
search localdomain
nameserver 192.168.1.1

Whilst /etc/resolv.conf from the rootfs on the original uSD card that I’ve been using:

domain hsd1.wa.comcast.net.
search hsd1.wa.comcast.net.
nameserver 10.0.1.1

Also, /etc/network/interfaces does not have the stanzas for the USB gadget, and /etc/udhcpd.conf is not setup properly. There may be other issues, but this is as far as I’ve looked.

Obviously something is either missing or went awry.

Thoughts?

Cheers,

ba

6

If you want my opinion, ditch Linux mint NOW. Personally I will not use anything other than Debian for a support system to the BBB, and would NEVER use X for this purpose. Especially in a VM . . .

Yeah yeah, Linux mint is based on Ubuntu and Debian( testing ) ( depending on version ), but thats part of the problem.

7

@Robert - Thanks for confirming!

@William - Interesting. Can you shed some additional light on this
approach? usbnet is Ethernet over USB, correct? Is this just an
alternative to using the wired Ethernet jack...and thus one still is
mounting the rootfs via NFS from a NAS or something like that? Or is this a
different approach? How does the BBB access/mount the rootfs? Would this
approach work for supporting cross development on a Linux laptop with a
filesystem shared with the BBB?

I was able to download a kernel via TFTP, boot and successfully mount an NFS
rootfs...sorta.

I had originally setup a rootfs on the NFS share as follows:

Setup a uSD card:

Created using
https://rcn-ee.net/deb/testing/2014-08-05/lxde/bone-debian-7.6-lxde-armhf-2014-08-05-4gb.img.xz
Configured and setup on a BBB Rev C.
Successfully booted, run, and updated numerous times on the BBB.

Shutdown the BBB, move the uSD card to my MAC. Then, from Linux Mint
running in Virtual Box on the MAC:

Mount the uSD card.
Create a tarball from the mounted uSD card whilst sudo.
Untar the tarball into the exported NFS share location on Linux Mint whilst
sudo.

My assumption was that I should be able to move/copy the rootfs from the uSD
card using a separate system. Note that the uSD card was NOT running a live
system when I made the copies.

Unfortunately, when the BBB subsequently tries to boot using the copied
rootfs, the following happens:

[....] Checking root file system...fsck from util-linux 2.20.1
fsck.ext4: No such device or address while trying to open /run/rootdev
Possibly non-existent or swap device?
fsck died with exit status 8

When you copied the rootfs, the root device is listed in /etc/fstab,
along with the fsck call. Just minimze it:

voodoo@hades:$ cat /opt/wheezy/etc/fstab
debugfs /sys/kernel/debug debugfs defaults 0 0

So, I resorted to creating a fresh rootfs using
https://rcn-ee.net/deb/testing/2014-08-05/lxde/debian-7.6-lxde-armhf-2014-08-05.tar.xz.

All went fine and the system booted and I could ssh to it.

Unfortunately, this system seems to have some broken network setup. A bit
of digging reveals that /etc/resolv.conf is not setup correctly:

domain localdomain
search localdomain
nameserver 192.168.1.1

Whilst /etc/resolv.conf from the rootfs on the original uSD card that I've
been using:

domain hsd1.wa.comcast.net.
search hsd1.wa.comcast.net.
nameserver 10.0.1.1

/etc/resolv.conf is autogenerated on bootup...

Also, /etc/network/interfaces does not have the stanzas for the USB gadget,
and /etc/udhcpd.conf is not setup properly. There may be other issues, but
this is as far as I've looked.

odd..

Btw, here's how i setup the nfs rootfs.. I first boot the bbb, with
the microSD, then run this:

Just change all occurances of:

192.168.0.10 & /opt/wheezy/

and it should work for you.

I should make it more generic, for people other the me to use..

Regards,

8

When you copied the rootfs, the root device is listed in /etc/fstab,
along with the fsck call. Just minimze it:

voodoo@hades:$ cat /opt/wheezy/etc/fstab
debugfs /sys/kernel/debug debugfs defaults 0 0

Yes, on the failing rootfs I see the / mount (/dev/mmcblk0p2 / ext4 noatime,errors=remount-ro 0 1)

I’ll eliminate that and give it another try.

Btw, here’s how i setup the nfs rootfs… I first boot the bbb, with
the microSD, then run this:

https://github.com/RobertCNelson/boot-scripts/blob/master/tools/developers/nfs-rsync.sh

Just change all occurances of:

192.168.0.10 & /opt/wheezy/

and it should work for you.

Thanks! I’ll have a look.

Maybe the auto generation isn’t working when I boot from a virgin rootfs created from the tarball?
Is your expectation that one should be able to use the rootfs tarball as the basis for a root file system mounted on NFS, mounted on flash?
Is it possible that the initial bootup is assuming something that it shouldn’t be?

Thanks again!

Cheers,

ba

9

@William - Interesting. Can you shed some additional light on this approach? usbnet is Ethernet over USB, correct? Is this just an alternative to using the wired Ethernet jack…and thus one still is mounting the rootfs via NFS from a NAS or something like that? Or is this a different approach? How does the BBB access/mount the rootfs? Would this approach work for supporting cross development on a Linux laptop with a filesystem shared with the BBB?

Yes, usbnet is the description modprobe gives the cdc_ether( g_ether?) module on the host side I believe. I can think of two reasons why this is useful.

First, if you’re on a complex network for the host( university, work etc ), this could alleviate some potential headaches. Secondly g_ether networking is supposed to be nearly twice as fast when using Linux on both ends. This means faster boots.

So far the initial testing I’ve done shows that it is faster by around 20Mbit / s using iperf as the test. This is from Windows 7 x64 as the host, and a minimal Debian wheezy image on the BBB side( ~191M total ). Then testing to my Debian support system, which is in a virtualbox VM on yet another Windows 7x64 machine . . . is incredibly slow. Somewhere around 15Mbit / s. Which is terrible.

However, I was recently given an eeepc with a broken screen that I was able to put back together ( came to me in pieces ), and get working again. I have installed Debian wheezy on this machine, and will do some testing between Linux<->Linux when i get the chance. Also I am likely to use one of Robert’s premade demo images, to show others how to do the same( in the context of converting to NFS ). I am pretty sure this will be pretty easy, but this is one reason why I want to try. TO make sure.

10

If you want my opinion, ditch Linux mint NOW. Personally I will not use anything other than Debian for a support system to the BBB, and would NEVER use X for this purpose. Especially in a VM . . .

Yeah yeah, Linux mint is based on Ubuntu and Debian( testing ) ( depending on version ), but thats part of the problem.

Hmmm, OK! Would you like to enumerate why you wouldn’t use Mint? I was under the impression the Mint-17 is based upon Ubuntu 14.04LTS, and thus fairly stable. Personally, I can’t stand Unity…but YMMV. What distro would you suggest?

Well, at the moment, all I have is my MBP laptop to support this effort. So, either I setup NFS on the MAC and hope for the best, or use a VM running some Linux. I thought I’d give the VM approach a try as a first step in order to not introduce native MAC NFS vagaries into the mix. Probably could try that option now that I have things limping along.

When you say NEVER use X, I’m assuming you mean running X windows on a dev env (Linux Mint)? I’m not running X on the BBB (well, I do often use X forwarding to the MAC/XQuartz for stuff like (gasp) emacs, xterm, …). My thought was to do dev on the MAC (straight away or via a VM) using a shared file system between the MAC and BBB so I didn’t have to copy files around, nor risk loosing everything if the BBB goes toes in the air or the uSD craps out.

I’m all ears on suggestions for a good dev setup though!

Cheers,

ba

11

Part of my reasoning is probably personal preference, but the majority of my reasoning has to do with package conflicts. Debian stable(wheezy at the moment ) should always be pretty much straight forward in this regard ( no surprises ).

Where as LMDE is based on Debian testing, a branch which is subject to change, and not guaranteed to be 100% functional. To clarify, this means it will probably boot, and most packages are very likely to work as well, but not guaranteed. However, the bigger problem lies in various tool versions. As I’m sure the Linux Mint team does at least test their builds some. So if you’re trying to build something for your BBB that requires a certain version of x.y.z tool, and your support system uses a different version . . . You’re likely going to run into errors, or worse yet; Silent errors leaving you very frustrated for many moons.

In short, and in this context - Debian just works. Now, if you’re a Linux / troubleshooting guru. This would probably be less of a problem for you.

12

Where as LMDE is based on Debian testing

Regular Linux Mint is based on Ubuntu, and in this case is probably worse than LMDE.

13

Brian, anyhow if you’ve got a huge time investment into your current support system, just use what you have now. But do keep what I say in mind. My comments are based on experience, and I have been down the same road you’re on now. I did this all last year, and for a while it was a very frustrating experience for me. Things could have changed since though ( but I’m thinking unlikely ).

The first 3 times I tried following Roberts instructions last year I failed. Part of the problem for me was that the instructions were less clear ( to me ) than they are now, and I had a couple years of Debian “rust” to shake off. I also tried Linux Mint ( DE ), and kept running into package conflicts. I even ran into a problem where Debian caused an error( DASH versus BASH ), which was relatively easy to fix.

Anyhow, I’m not really trying to tell you what you should / should not use. All I am trying to do is give you some advice based on my own experiences. If LM works for you then great.

14

William,

Thanks for the comments. Yes, I am in the same position as you…a year on. I’ve not touched Linux in any serious way for a long time. I have been involved in a number of commercial deployments that target CentOS/RHEL. My role was that of a Java server developer, so I didn’t get all that close to the metal. I am just recently coming off of 3 years of serious Android development work and hoping to transition to some embedded stuff (Android and/or embedded Linux) professionally over the next couple of years. So, I do appreciate your input.

FWIW, I am thinking that things might have changed a bit since you last looked. I’m using Mint-17 self described on their release page as:

Linux Mint 17 features Cinnamon 2.2, MDM 1.6, a Linux kernel 3.13 and an Ubuntu 14.04 package base.

Apparently this is not LMDE…that is a different distribution that the Mint folks offer (and yes, that is still Debian testing, assuming jessie). I remember reading up on Mint changing directions a bit in the not too far distant past, concentrating on improving UI on top of a core Ubuntu distro. So far, so good. Nice UI, performant enough. No problems with package conflicts.

Regarding the usbnet stuff, I read your response to be that usbnet offers the potential for a higher bandwidth connection than available over the Ethernet jack on the BBB. Correct? All other things being the same? That is, NFS rootfs mount, TFTP, …

Cheers,

ba

15

I do not think TFTP kernel load will be possible over usb. Although there was a GSoC project last year for that. Last I heard semi recently, the developer was still having some issues. His vision was to boot a BBB over NFS from an Android device, like a tablet or phone. Pretty cool project I think, but not really my thing.

But yes, I’ve read some performance tests where g_ether can get up to as high as 18MB/s( Mega bytes ). I do not know if this will be possible on the BBB or not ( different usb implementation etc ), but that is why I’m going to be figuring this out for myself.

16

Brian, as far as Ubuntu goes . . . Yeah let us just say I dont have very much good / nice to say about it. I’ve done tons of testing on Ubuntu, and it has always let me down. Not to mention it’s upgrade path used to literally be a nightmare.

17

If you want my opinion, ditch Linux mint NOW. Personally I will not use anything other than Debian for a support system to the BBB, and would NEVER use X for this purpose. Especially in a VM . . .

Yeah yeah, Linux mint is based on Ubuntu and Debian( testing ) ( depending on version ), but thats part of the problem.

Hmmm, OK! Would you like to enumerate why you wouldn’t use Mint? I was under the impression the Mint-17 is based upon Ubuntu 14.04LTS, and thus fairly stable. Personally, I can’t stand Unity…but YMMV. What distro would you suggest?

Well, at the moment, all I have is my MBP laptop to support this effort. So, either I setup NFS on the MAC and hope for the best, or use a VM running some Linux. I thought I’d give the VM approach a try as a first step in order to not introduce native MAC NFS vagaries into the mix. Probably could try that option now that I have things limping along.

When you say NEVER use X, I’m assuming you mean running X windows on a dev env (Linux Mint)? I’m not running X on the BBB (well, I do often use X forwarding to the MAC/XQuartz for stuff like (gasp) emacs, xterm, …). My thought was to do dev on the MAC (straight away or via a VM) using a shared file system between the MAC and BBB so I didn’t have to copy files around, nor risk loosing everything if the BBB goes toes in the air or the uSD craps out.

I have a MBP which I love, but I wouldn’t use it for development for the same reasons I wouldn’t use Windows for development and that is because neither support case sensitive file system. Also, OSX tools are quite old and sometime incompatible with their GNU equivalents (options are different more often than not compared to GNU versions), so you have to use MacPort, HomeBrew, Fink, etc. Regarding Mint, Ubuntu, Debian, etc, there isn’t really much between them other than personal preferences. There are both benefits and downsides to each, so choose one and stay with it. Truly speaking, each one needs some work to get it stable and working the way you want. Debian might be perceived as more stable, but it uses old version of almost every package and the core repository is way smaller than Ubuntu so you have to hunt around for other repos to find the packages you need and then Debian becomes less stable. Ubuntu was a bit flaky for a while, but 14.04 is much better and the distro I use daily.

Regards,
John

18

Debian might be perceived as more stable, but it uses old version of almost every package and the core repository is way smaller than Ubuntu so you have to hunt around for other repos to find the packages you need and then Debian becomes less stable.

Hunt around for what packages ? In the context of the current discussion I’ve never had to “hunt” for anything. I’ve had to compile my own stuff from sources when I wanted something custom . . . Now if you want cutting edge stuff, you’re almost certainly going to run into trouble no matter what distro you use. But that is not what we’re talking about. We’re talking about running a distro in a VM for the sole purpose of supporting the Beaglebone black.

19

Brian I see where your source of confusion is coming from. Right now after spending a few hours reading through various configuration files and making adjustment as needed . . . I’m very aggravated.

Mostly due to the fact that I spent a great deal of time figuring all this out last year, and now it’s different. To the point where I’m seriously considering ditching the latest images, and stick with the slightly older images I have working perfectly already.

20

login as: william
william@sanitized’s password:
Linux arm 3.8.13-bone47 #1 SMP Mon Apr 14 04:38:52 MST 2014 armv7l

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Fri Aug 15 02:07:19 2014 from 192.168.0.2
$ sudo iperf -c 192.168.0.2
[sudo] password for william:
------------------------------------------------------------
Client connecting to 192.168.0.2, TCP port 5001
TCP window size: 21.0 KByte (default)
------------------------------------------------------------
[ 3] local 192.168.0.1 port 58475 connected with 192.168.0.2 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 162 MBytes 136 Mbits/sec
$ sudo iperf -s
[sudo] password for william:
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[ 4] local 192.168.0.1 port 5001 connected with 192.168.0.2 port 35283
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.2 sec 44.1 MBytes 36.4 Mbits/sec

So 17MB/s reads, and 4.55MB/s writes. Write speed is probably this low because the rootfs sits on a fast ethernet NFS share network. Which “coincidentally” is roughly half the NFS share’s network speed. Reads from a Windows 7 x64 based iperf server is just under 15MB/s, but with no real way to serve up a Linux type filesystem.