[beagleboard-gsoc] [GSoC 2010] USB sniffer

Google Summer of Code
1

Hi all,

First, my USB sniffer project got accepted for GSoC 2010, thank you!

I have a few "infrastructure" questions:
  - For the BeagleBoard project registration, should I take over the usbscope project [1] (if it is possible, and if Michael is OK with that), or create a new one?
  - For the source code, I expect my work to be mostly modifications to the kernel (limited to a few files I believe), do you think I should clone the complete Linux kernel, or just maintain a set of patches over the omap tree [2] (clearly mentioning which revision it is based on)? Cloning the omap tree takes forever (as it gets all the history from kernel 2.6.13 or so), so I was wondering if it is possible to have some kind of partial git tree... Otherwise I would simply put patch files in my repository. Any idea?
  - Can I create a wiki page on elinux.org about the project? (e.g. http://elinux.org/BeagleBoard/GSoC/USBscope) I think it would be nice to keep all the information at a single location. That would not replace the RSS feed you're asking for, but complement it.

All for now, I'm starting to read about USB, and look at OE toolchain and the MUSB driver...

Best regards,

Nicolas

[1] http://beagleboard.org/project/usbscope/
[2] git://git.kernel.org/pub/scm/linux/kernel/git/tmlind/linux-omap-2.6.git

2

It would be best if you base the work on the upcoming validation kernel: http://gitorious.org/angstrom/angstrom-linux/commits/beagleboardXM

Gitorious has a 'clone' option to make it easier to have a private tree.

regards,

Koen

3

My previous mail is a bit unclear, sorry for that. What I meant to say is:

You should target "mainline" (e.g. linus' tree) so you work can go upstream more easily, but it would be nice if your project also worked with the kernel that's going to ship with rev C5 and xM.

regards,

Koen

4

Hi Michael,

Sure, take over the usbscope project.

Well, I'm not sure if it's possible to do it actually, I don't see any option to modify another person's project. I guess I'll just create a new one.

For the source code you should read up on linux kernel development
(Google should find some guidelines). Basically you should take a git
clone of the appropriate omap repo and do your development. You can do a
git clone that only copies down the latest version and no history
(sorry, don't remember off hand how to do this, but Google should
provide answers).

Thanks I didn't know about the "--depth=1" option!

> [...]

You should be able to edit pages on elinux, but you might need someone
else to actually create a stub page for you... Look here for details of
the mailing list to ask someone to create the page
(ELinuxWiki:Mailing List - eLinux.org). You should be able to
create an account yourself by clicking on Login (top right of screen).

Looks like I can create pages:
http://elinux.org/BeagleBoard/GSoC/USBSniffer

Good luck with the MUSB drivers... ;]

Well, thanks ,-)

Best regards,

Nicolas

5

I guess I'll use angstrom-linux/beagleboardXM as a base, to get the latest improvements in the MUSB driver. As long as that tree is not too far off the mainline kernel, porting my patches from one branch to another should not be too difficult.

Regards,

Nicolas

6

Hi Nicholas,

Your project seems quite interesting, but I'm curious as to how you're
going to go about doing it. I would imagine that you've considered the
potentially negative effects that any latency could have for, e.g.
isochronous transfers.

In any event, A USB packet sniffer based on the beagleboard would be
_very_ useful for a few projects I'm currently working on, and most
certainly for _lots_ of future projects by anyone interested in
hacking usb devices.

Have you set up a blog so I can follow your progress?

7

Hi Christopher,

Thanks, great to hear about your interest in this project.

Of course, latency can be an issue, and that's one of the reasons why we are thinking about a implementation that lies completely in the kernel (instead of having some user-space program to do the forwarding), but of course we won't be able to know exactly how it affects the transfers until I start implementing the driver.

I have a blog there: http://beagleboard-usbsniffer.blogspot.com/, not much yet, but more to come ,-)

Best regards,

Nicolas

8

Hi Nicolas,

Of course, latency can be an issue, and that's one of the reasons why we are
thinking about a implementation that lies completely in the kernel (instead
of having some user-space program to do the forwarding), but of course we
won't be able to know exactly how it affects the transfers until I start
implementing the driver.

It definitely needs to be in the kernel... but I wonder if that will
be enough for essentially being a practically zero-latency
pass-through device. It might be useful to look into using DMA for
forwarding URBS back and forth between interfaces, while a separate
kernel thread (task) performs the longer operations of adding them to
a FIFO object (i.e. dumping them out). Also, you might need to
consider that your module should essentially create zero overhead when
packets are not being monitored, although packet forwarding would
still need to be done - that sounds like a job for DMA if you ask me.

I'm not 100% sure, but if there is hardware linking the DSP to various
IO pads, then you might be able to use the DSP as a logic analyzer
(assuming that the sampling rate is high enough), and then use the
main CPU to 'parse' data.

I have a blog there: http://beagleboard-usbsniffer.blogspot.com/, not much
yet, but more to come ,-)

I'll definitely be watching your progress, and I might even check out
/ adjust your code to assist me in a couple of current projects :slight_smile:

By the way, were you informed when the BBs are being shipped? I have
an older rev B5 board with usb host capabilities noticeably lacking,
and am definitely looking forward to a rev Cx board.

C

9

Christopher,

The boards should all be shipped by now. Please contact Cathy for the tracking info. She is very busy, so be patient.

Regarding usb-sniffer, personally, I'd somewhat like to see a virtual hub implementation that would essentially create a bit of a firewall for implementing new gadget drivers with full in-system monitoring.

I think many of the issues for usb-sniffer have been discussed and look forward to posts from Nicolas describing the architecture planned to be implemented and the guidance from his mentors.

-Jason

10

Dear All. - all boards were shipped for student GSoC, I found that shipping company mis-directed 2 boards by placing them with another students board. I will have to reship those 2 . As soon as I am near my PC I'll be able to get tracking info and resend them.

Christopher, sounds like one of them was yours. I apologize for the problem and we'll get a new board out to you asap on Monday.

Did you need any cables for it?
Cathy

11

Hi Christopher,

Hi Nicolas,

Of course, latency can be an issue, and that's one of the reasons why we are
thinking about a implementation that lies completely in the kernel (instead
of having some user-space program to do the forwarding), but of course we
won't be able to know exactly how it affects the transfers until I start
implementing the driver.

It definitely needs to be in the kernel... but I wonder if that will
be enough for essentially being a practically zero-latency
pass-through device. It might be useful to look into using DMA for
forwarding URBS back and forth between interfaces, while a separate
kernel thread (task) performs the longer operations of adding them to
a FIFO object (i.e. dumping them out). Also, you might need to
consider that your module should essentially create zero overhead when
packets are not being monitored, although packet forwarding would
still need to be done - that sounds like a job for DMA if you ask me.

Sure, DMA is an option if performance constraints are not met. Yet, before going that way, that is, a platform-specific solution, I would prefer a platform-independant solution that integrate nicely in the kernel, with better chances of being pushed in the mainline kernel.

I'm not 100% sure, but if there is hardware linking the DSP to various
IO pads, then you might be able to use the DSP as a logic analyzer
(assuming that the sampling rate is high enough), and then use the
main CPU to 'parse' data.

Somehow, I doubt USB pins (that is, the ones before the ULPI transceiver) are available to the DSP... Even if available, the required sampling rate would probably be too high for the BeagleBoard.

Best regards,

Nicolas

12

Jason,

Christopher,

The boards should all be shipped by now. Please contact Cathy for the tracking info. She is very busy, so be patient.

Regarding usb-sniffer, personally, I'd somewhat like to see a virtual hub implementation that would essentially create a bit of a firewall for implementing new gadget drivers with full in-system monitoring.

For that purpose (i.e., testing new gadget drivers), isn't it easier to use the virtual gadget controller provided in the kernel (dummy-hcd)? That virtual controller allows your gadget to appear as a device on your PC. From there, I suppose that usbmon is available if you want further logging.

With my project, I suppose you could chain 2 BeagleBoards, the first one running the tested gadget driver, the second sniffing the traffic between the host PC and the first BeagleBoard: that way you could test how the gadget driver behaves on the real controller (instead of a dummy one).

Best regards,

Nicolas

13

Hi all,

I wrote an entry about device vs hub emulation on the blog (http://beagleboard-usbsniffer.blogspot.com/2010/05/device-vs-hub-emulation.html). I reproduced the content below, it would be great if someone could check if my understanding of USB hub is correct.
Then I'll start researching/writing something about how I think device emulation would work.

Best regards,

Nicolas

14

One other minor issue to keep in mind is the hub path has other side effects
on the sniffing -
1. There is a limit to the number of devices you can chain.A virtual hub would
cut that down by one layer. Minor issue.

2. As a hub, the power behavior changes. It can hide power problems. If you
emulate a powerd hub, devices that was connected up to a 100mA only port
would suddenly start working. On the other hand, if you emulated a non
powered hub, bus powered devices can stop working.

15

2. As a hub, the power behavior changes. It can hide power problems. If you
emulate a powerd hub, devices that was connected up to a 100mA only port
would suddenly start working. On the other hand, if you emulated a non
powered hub, bus powered devices can stop working.

I'm not sure if it would be possible to control the power of the virtual hub.
Then again this is pretty minor.
As a partial way around one could compare the results with the sniffer
with the results when using a regular hub

Hi all,

I wrote an entry about device vs hub emulation on the blog
(http://beagleboard-usbsniffer.blogspot.com/2010/05/device-vs-hub-emulation
.html). I reproduced the content below, it would be great if someone could
check if my understanding of USB hub is correct.
Then I'll start researching/writing something about how I think device
emulation would work.

wrt the blog text: not sure which port you mean with MUSB port. I
would expect the device-to-be-sniffed to be connected to the regular
(EHCI) port and the OTG port to the host.
The OTG port will definitely be able to deal with address 0 as it can
be used to act as a slave device.

Some more remarks:
- the device to be emulated can only be a HS device as other speeds
are not supported by the EHCI port.
- you might want to consider the case where a physical device actually
represents multiple devices. E.g. a simple voip handset might present
itself as an audio-in, audio-out, input (kbd) and output (lcd) device.
Ideally a sniffer should be able to sniff all traffic (so also if
there is a physical hub downstream).
Having said that, I think you should probably focus at a single device
first, but keep this in mind, so you do not close doors or make
decisions that disallow that (or if you have to then at least it was a
conscious decision).

And, as mentioned before, synchronous transfers may cause additional
pain (and be a reason to do most, if not all, in the kernel). Note
that synchronous does not mean delay-less (a hub will also cause some
delay), but it may well mean that the latency (if any) is constant.

Wrt the acquisition and display of the data: I would suggest
investigating if it is possible to use a wireshark plugin.
Then again, probably the initial focus should be to have the data
passing through the beagle (probably starting with a single device).
Maybe that is a good mid-term goal.

Frans

16

One other minor issue to keep in mind is the hub path has other side effects
on the sniffing -
1. There is a limit to the number of devices you can chain.A virtual hub would
cut that down by one layer. Minor issue.

2. As a hub, the power behavior changes. It can hide power problems. If you
emulate a powerd hub, devices that was connected up to a 100mA only port
would suddenly start working. On the other hand, if you emulated a non
powered hub, bus powered devices can stop working.

Sure. But I guess that whatever we do (hub or device emulation): it will be hard to debug any power problems: If the BeagleBoard uses external power, it would not draw any power from USB, if not, it would draw the power for the board + the USB device.

Unless you are talking about the power requirements advertised in the USB descriptors. In that case, yes, hub emulation would change the behaviour.

17

[...]

Hi all,

I wrote an entry about device vs hub emulation on the blog
(http://beagleboard-usbsniffer.blogspot.com/2010/05/device-vs-hub-emulation
.html). I reproduced the content below, it would be great if someone could
check if my understanding of USB hub is correct.
Then I'll start researching/writing something about how I think device
emulation would work.

wrt the blog text: not sure which port you mean with MUSB port. I
would expect the device-to-be-sniffed to be connected to the regular
(EHCI) port and the OTG port to the host.

Yes. By MUSB I mean the slave (gadget, OTG) controller (the EHCI controller does not seem to be part of the MUSB IP, from what I see in the datasheet).

The OTG port will definitely be able to deal with address 0 as it can
be used to act as a slave device.

My concern is that the OTG port may only be able to deal with _one_ address at a time. It will answer to address 0 after a reset, but as soon as it is assigned an address by the host, it will answer to that address, and not any other.

Some more remarks:
- the device to be emulated can only be a HS device as other speeds
are not supported by the EHCI port.

Sure, for LS/FS, we need a hub (a real physical USB hub ,-)), I'm aware of that.

- you might want to consider the case where a physical device actually
represents multiple devices. E.g. a simple voip handset might present
itself as an audio-in, audio-out, input (kbd) and output (lcd) device.

As long as all these devices answer to the same USB address, it should not be a problem.

Ideally a sniffer should be able to sniff all traffic (so also if
there is a physical hub downstream).

That, I don't think would be feasible. If there is a hub downstream, there will be multiple addresses.
What we could do, if there is a hub, is to "pick" one device, and forward the traffic to it (that would be useful for the LS/FS case, where a hub is needed).

Having said that, I think you should probably focus at a single device
first, but keep this in mind, so you do not close doors or make
decisions that disallow that (or if you have to then at least it was a
conscious decision).

Yes of course. That's why I prefered a hub solution in the first place (more flexibility). But if that can't be done...

And, as mentioned before, synchronous transfers may cause additional
pain (and be a reason to do most, if not all, in the kernel). Note
that synchronous does not mean delay-less (a hub will also cause some
delay), but it may well mean that the latency (if any) is constant.

Sure.

Wrt the acquisition and display of the data: I would suggest
investigating if it is possible to use a wireshark plugin.

There is a wireshark plugin for USB traffic dumped from usbmon (USB). If we keep the usbmon interface, that comes for free.

Then again, probably the initial focus should be to have the data
passing through the beagle (probably starting with a single device).
Maybe that is a good mid-term goal.

Indeed. I plan to start with simple devices, with limited number of endpoints, and slowly start to support different endpoint types (control/bulk, then interrupt, and finally isochronous).

Thanks.

Best regards,

Nicolas

18

Hi everyone,

Another blog entry: http://beagleboard-usbsniffer.blogspot.com/2010/05/device-emulation.html

I hope it makes sense, comments are welcome.

Best regards,

Nicolas

19

Actually MUSB *does* support LS, FS and HS, it's the EHCI on the omap that doesn't :slight_smile:

20

Actually MUSB *does* support LS, FS and HS, it's the EHCI on the omap that doesn't :slight_smile:

Not as a slave/gadget device, at least according to the Technical Reference Manual (http://www.ti.com/litv/pdf/spruf98f), p.3147:
"* USB 2.0 peripheral (function controller) in high/full speed (480/12 Mbps, respectively)"

The MUSB in _master_ mode does support all speeds, though:
"* USB 2.0 host in high/full/low speed (480/12/1.5 Mbps respectively), [...]"

Regards,

Nicolas