I am Tobin Davis, QA Tech for Ubuntu Arm. I have been testing Ubuntu
on Arm platforms since March, 2009, and starting with our 10.04
release, on beagle (C4). Recently, we made the switch to arm hard
float for our entire repository. This change, along with distro wide
changes to enable multi-arch support has opened a large can of worms
for me testing wise. While I have a good handle on testing packages,
most of my testing is done on the pandas, as they are faster (dual
core), and finding threading bugs on SMP is 10x more likely than on
UP. With this in mind, I am having to focus less on image testing on
beagle/beagleXM. Also, I just don't have enough beagles to test with,
nor the infrastructure to automate them properly.
What I am asking is for the community to step up and help with smoke
testing the daily precise omap images, to ensure that they boot
properly and support the base hardware fully. One issue I am
currently hitting is that I have no audio and very spotty networking
in my beagleXM (Rev B). Since I have only the one board, I have no
way to verify if this is a kernel bug or maybe a bad board (networking
works if I physically unplug the network cable, wait a few seconds,
then plug it in - otherwise I don't get link lights).
I am not asking for someone to download and test every image every day
(although that would really be nice), but for volunteers to test at
least weekly, and also help with the milestone testing (Beta 2 is
coming with testing opening on March 26). We are also nearing kernel
freeze, followed by release freeze and final release the last week of
April.
If anyone has questions on what to test, or how to use our images,
please ping me (GrueMaster) on #ubuntu-arm or #beagle on irc.freenode.net, or email reply to this thread.
I can definitively help out. Do you have a wiki/online-spreadsheet
link that users of the community can easily add quick testing
details.. date/image, boards revision, check boxes for basic
functionality, etc.. I know linaro has been doing things like that on
their site.
I am working with a Beagleboard together with a Beagleboard Trainer und Angstrom Linux. Is it possible for user software to autmatically find out, wether a Trainer Board is conntected to the Beagleboard or not?
I’m Carlos Hernandez and I’m part of TI’s SW validation team.
I finally had some time to play w/ the ubuntu images and I think I can help with the ubuntu validation effort on beagleXM.
I would like to run a sanity test plan, similar to the one we run at TI, to validate basic functionality of most drivers.
We have tools to automate nightly tests however we first need to address following 2 issues:
Ubuntu installation menu:
We will need to write a script to respond to the ubuntu installation menu. Let me know if you already have a script to take handle the ubuntu installation menu. If you don’t have it one, it is not a big deal.
Missing test tools
We use various test tools (iperf, lmbench, rt-tests, ltp-ddt, etc. ) during the execution of sanity test plan. These tools are not available in the ubuntu filesystem. Is it possible to include them in the fs image for test purposes?
Thank you all to the call for testing. Ubuntu has a tracker for testing images at http://iso.qa.ubuntu.com/qatracker. You can browse the testcases and results without an account, but to really be active, you will need a Launchpad account to enter your test result data.
The arm platforms on the tracker are:
omap - Beagle, BeagleXM, and theoretically BeagleBone (I have not tested this but have heard it just works).
omap4 - Panda, Panda ES.
mx53 - Freescale Quickstart board.
ac100 - Toshiba AC100 netbook.
armadaxp - Arm server platform (not publicly available yet).
Each test case has documentation on what to test. The scope is very limited, so repetitive testing with different options is very helpful (autologin vs login prompt vs encrypted home, keyboard detection, etc).
Due to recent job changes, I will no longer be able to work on testing these images as often as I had in the past (weekly if not more frequent). It is now up to the community to ensure that these get tested on a regular basis to ensure high quality standards.
For this level of automation testing, I was gearing up to use preseeding on the preinstalled images (recently added with 12.04 Beta 2). Otherwise, I recommend using the netboot image to do a fully automated installation to a USB drive (use the SD for the fat boot data). This can be preseeded with packages, partitioning, and a lot of other automated functionality. This was what I used to automate kernel SRU testing, with Jenkins-CI as a controller/dashboard. Once the systems are reimaged, tests can either be launched via jenkins, or just remote ssh. I have tests written in bash & python that do post-image package installation, configuration, and test execution.
Another method that I was going to try to ramp up was to use the usb-boot method (at least on Panda) to boot a netboot image or to reflash the SD with a preinstalled image for automated installation. The u-boot-spl from Linaro can now handle this.