Video capture on the BeagleBoard XM

Hi Andrey, thanks for your help on this.

The Point Grey Chameleon is a USB2.0 camera. It works fine on the BeagleBoard using the capture functions from http://sourceforge.net/projects/libdc1394/!

You guessed right about overwriting the images. It turns out that when I was queueing the images up for saving, I was queuing up pointers to the images rather than copies of the images, which explains why the video seemed to be jumping all over the place. It’s fixed now.

Now the problem is that I can only get ~4fps when capturing from both cameras simultaneously. And even at that frame rate, the color image is sometimes split down the middle (see attached image). My SD card is the Sandisk Extreme Pro which supposedly has the fastest write speeds on the market. I also have external USB storage plugged in. I tried saving the color images to the sd card and the infrared images to the USB storage, and I tried saving all the images to the sd card, but it doesn’t seem to make a difference.

Do you think it’s even possible to get ~15 fps on the BeagleBoard, with 1280x960 color image and 640x480 infrared? Or am I wasting my time here? The reason I need such a high frame rate is because I want to do tracking. And the reason I need 1280x960 resolution is because I’m doing people detection from aerial imagery, so the camera would be quite a distance away from the people. I will gladly post my code if you think I can improve the frame rate…

By the way, I ran top in parallel with my code and there is nothing else using up any significant amount of memory.

Hi Elizabeth,

The Point Grey Chameleon is a USB2.0 camera. It works fine on the
BeagleBoard using the capture functions from
http://sourceforge.net/projects/libdc1394/!

Oh, I was not aware that libdc1394 also supports USB cameras. Thanks
for the hint.

Now the problem is that I can only get ~4fps when capturing from both
cameras simultaneously.

While experimenting with two UVC USB cameras:
http://veter-project.blogspot.com/2010/12/stereo-video-stream-from-on-board.html
we were able to get something around 15fps with two 320x240 YUV
frames. Later, with two analog cameras connected over USB frame
grabbers (http://veterobot.org) we were unable to capture at all in
parallel (USB driver were reporting a lot of bandwidth and other
errors). So I conclude that you results will heavily depend on quality
of drivers for your cameras.

And even at that frame rate, the color image is
sometimes split down the middle (see attached image).

Are you capturing with libdc1394 or using V4L API? It looks like you
are somehow mixing the data from different frames. With V4L API there
is typically select()-based capturing loop. How is yours organized?
How do you detect that the next frame is available? How do you assure
that you read the complete frame when it is available? Are you sure
that you are not overwriting again some existing data? To me it looks
like a bug in capturing loop.

Do you think it's even possible to get ~15 fps on the BeagleBoard, with
1280x960 color image and 640x480 infrared? Or am I wasting my time here?

I do not think it is possible. But do not trust my words and try to
estimate it by calculating the required bandwidth target_fps *
(color_frame_size_in_bytes + ir_frame_size_in_bytes) and compare it
with the bandwidth offered by USB 2.0 (do not know exact value, but it
should be easy to find it in Internet). Then subtract some amount
(let's say 10%) to take in account whatever overhead there might be
and you will see what you can expect (theoretically) from USB
subsystem. But again, I doubt, that your requirements are feasible
with this hardware.

While trying to overcome USB throughput we were thinking about
connecting the second video-camera over hacked OTG port (converted to
host) and supplying separate power to the grabber, but did not manage
to make it work either.

Regards,
Andrey.

OK, forgive me for asking a stupid question, but if limited USB bandwidth is the problem, then why can I get 15 fps from both cameras on my laptop. They’re connected via USB on my laptop also. Are there different types of USB2.0?!

OK, forgive me for asking a stupid question, but if limited USB bandwidth is
the problem, then why can I get 15 fps from both cameras on my laptop.
They're connected via USB on my laptop also. Are there different types of
USB2.0?!

Do not know exactly. Maybe there are two separate hubs on your laptop
in contrast to one (with 4 ports) on BB?

Oh I see. On my macbook I have 2 USB2.0 ports which have up to 480Mbps each…

And woud using a USB mouse and a USB flash drive eat into that bandwidth significantly?

+ the smsc95xx 10/100 ethernet adapter.. It's all shared on one and
only one 480Mbps port...

Regards,

Now the problem is that I can only get ~4fps when capturing from both cameras simultaneously. And even at that frame rate, the color image is sometimes split down the middle (see attached image).

You will get the same results with any other camera using beagleboard. I’ve been using it for two years and I had to move to other platform just because there is no response when mentioning this USB issue in this group.
Reasons? Maybe that quartz clock between the CPU and the USB hub in the board… or maybe just a problem with the USB driver. By the way I hope some day it will be solved.

My SD card is the Sandisk Extreme Pro which supposedly has the fastest write speeds on the market.

Your SD Card is modern and backwards compatible but the problem resides in that beagleboard SD card interface cannot run faster than a class 10 interface. There are arm boards in the market (and also cheaper) that have sata ports that will allow you to read and write at greater speeds.

I also have external USB storage plugged in. I tried saving the color images to the sd card and the infrared images to the USB storage, and I tried saving all the images to the sd card, but it doesn’t seem to make a difference.

Do you think it’s even possible to get ~15 fps on the BeagleBoard, with 1280x960 color image and 640x480 infrared?

I think that with beagleboard you can only encode 1 video source to 720p and that can only be done using the DSP, that by the way cannot be controled with a library of the style of OpenMax, I mean it cannot be used in a simple way. The beagle can only write at 10Mb/s to the SD so without encoding you will only be able to write 3 or 4 images per second at the resolutions you mentioned.

Or am I wasting my time here?

The reason I need such a high frame rate is because I want to do tracking. And the reason I need 1280x960 resolution is because I’m doing people detection from aerial imagery, so the camera would be quite a distance away from the people. I will gladly post my code if you think I can improve the frame rate…

In my opinion BeagleBoard is a good platform to start with and learn, and then to get in contact with Texas to develop your own board, or to move to other platform with all you learned. In my opinion beagleboard isn’t fast enough, and the USB issue continues without being solved.