GSoC 2010 - USB sniffer

GoogleGroups
1

(sorry if this is a duplicate, I tried sending to
discussion@beagleboard.org earlier today, but it hasn't appeared in
the Google group yet)

Hi all,

I am planning on applying for the Google Summer of Code 2010 program,
and I am very interested in the USB sniffer/scope project [1]. I know
that hardware USB sniffers are expensive hardware, so it would be
useful to have a cheap alternative using the Beagle board, even if you
may lose some low-level debugging features.

It seems like someone started a project a year ago [2], does anyone
know if this went further than a concept, or if some development was
actually started?

My idea would be to start off with a userspace USB gadget + device
drivers (both seem to be feasible in userspace), and maybe move some
code in kernel space if this cannot meet the performance requirements,
or if some features are missing/unsupported: For example, it seems
like the Gadget Filesystem interface (USB_GADGETFS) only supports
"single-configuration USB device" [3], which may be a problem for some
device types.

Any ideas/comments on this?

Thank you,

Best regards,

Nicolas

[1] http://elinux.org/BeagleBoard/GSoC/Ideas
[2] http://beagleboard.org/project/usbscope/
[3] http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob_plain;f=drivers/usb/gadget/Kconfig;hb=HEAD

2

Hi Nicolas,

Sounds like a nice project.

I have wrote drivers in the past for the OTG subsystem, so I'm familiar with it. I would say that the performance in user space could be an issue, but at the same time it would be a lot easier to implement rather than kernel space. I think in long term kernel space is better.

My first suggestion would be to write your own usb gadget class, where you will have to forward all the identification requests to the device attached on the host port (I'm not that familiar with the host API from the consumer perspective), but I think it shouldn't be that hard. Then I would suggest that you generate the logs of the transactions using something like netlink sockets, so you can have an userspace application that can easily obtain the debugging data.

Regards,

Diego.

3

Nicolas,

i did some early testing for this type of configuration with the nail board:

http://www.elinux.org/Nail_Board

my experimentation was limited to sniffing single function devices and had some very basic success, but was limited by the processing power of the s3c2410(running at 200MHz). with the omap3 you should be able to have much better results.

Dave

4

Basically you'd end up with two kernel drivers and a user space app that connects the two. very similar to the way you use a uart driver and connect it to an input driver:

http://www.kernel.org/pub/linux/kernel/people/dtor/inputattach.c

just add some logging inside the code.

Dave

5

Basically you'd end up with two kernel drivers and a user space app that
connects the two. very similar to the way you use a uart driver and connect
it to an input driver:

http://www.kernel.org/pub/linux/kernel/people/dtor/inputattach.c

just add some logging inside the code.

There is already some support for logging, see CONFIG_USB_MON and
Documentation/usb/usbmon.txt. The last I looked at this, it was host centric.
So some work would be needed to expand its functionality to include logging
the gadget side (if it isn't there already). That could also be the basis of
the data shuffling between the host and the gadget parts.

CONFIG_USB_MON would grow on an existing API instead of inventing yet another
one.

6

Hi everyone,

Thanks for your comments and idea. I also got an answer from Michael
Evans off-list (who started the usbscope project [2]). One of the
problems he encountered is that the OTG port is used for sniffing, and
therefore is not available for communication with the host PC. His
idea was then to create a gadget USB hub, to which you could connect
various devices (i.e. a virtual Ethernet controller, the emulation of
the device to sniff, etc.). As a temporary fix, we could also connect
a (real) hub to the USB host port, and have an USB->Ethernet adapter
on it, as well as the device we want to sniff, but having a all-in-one
solution would be surely nicer.

So, if I summarize and comment on your points:
1. It is probably easier to implement a user-space gadget driver (even
though GadgetFS is still marked as experimental), but this may cause
some performance problems, particularly if one day we want to make it
usable on less powerful platforms.
One of the advantages I also see of staying in user-space is that it
may be easier to do replay, or transfer USB data over a network. But
it should be possible to add some replay facilities in the kernel (or
simple use usbfs), and for the network part, we could integrate it
with usbip [4].
2. Making use of CONFIG_USB_MON is a good idea. Indeed, it works only
in host mode, but we can still use it, without modification. What we
would then need is a gadget which forwards USB data to a host port,
and CONFIG_USB_MON would be able to log these forwarded transfers.

David, what kind of approach did you take in your experimentation on
the Nail Board? Did you develop something in user-space, kernel-space?
Is the code available somewhere?

Thanks,

Best regards,

Nicolas

[4] http://usbip.sourceforge.net/

7

Nicolas,

Starting off from user space might be a good start, but if you want to
be able to do the best possible bandwidth and the least delay probably
you'll need to move to kernel/driver space eventually.

Also, I'm not sure if it is possible to have a really transparent box.
It would be nice if it that is possible, but I can imagine it isn't.
If not you might need to let the world think you are actually a hub.

With respect to the problem there are actually a few issues:

First one is acquiring and passing the data. In order to get the max
performance, you need to minimize the overhead (read: minimize the
number of context switches).
I did something similar for IP in the past (actually bridging from
ethernet to another network).
The approach I used there was to have the interrupt handler that
receives the data write into a circular buffer and a thread that would
output the data (blocking as long as there was no input data in the
circular buffer). In your case as there is traffic the other way too,
you might need to setup two such chains.

Second one is how to actually store the data (including timestamping
it). Depending on the time you want to monitor and the data rate you
might want to use some low level filtering or compression (as your
memory is also limited).

And lastly there is the issue of presentation.
You already mentioned usb snoop, and you could adopt that. However,
you might also want to have a look at ethereal/wireshark. Actually it
might even be possible to somehow add a plugin to wireshark that can
display usb data (haven't really delved into it).

BTW and if you need more info on USB: the boook "USB complete" seems a
good start.

Frans

8

(I sent this earlier, but it didn't seem to get through, again...)

Hi everyone,

Thanks for your comments and idea. I also got an answer from Michael
Evans (who started the usbscope project [2]), which I copied below.
One of the problems he encountered is that the OTG port is used for
sniffing, and therefore is not available for communication with the
host PC. His idea was then to create a gadget USB hub, to which you
could connect various devices (i.e. a virtual Ethernet controller, the
emulation of the device to sniff, etc.). As a temporary fix, we could
also connect a (real) hub to the USB host port, and have an USB-

Ethernet adapter on it, as well as the device we want to sniff, but

having a all-in-one solution would be nicer.

So, if I summarize and comment on your points:
1. It is probably easier to implement a user-space gadget driver (even
though GadgetFS is still marked as experimental), but this may cause
some performance problems, particularly if one day we want to make it
usable on less powerful platforms.
One of the advantages I also see of staying in user-space is that it
may be easier to do replay, or transfer USB data over a network. But
it should be possible to add some replay facilities in the kernel, and
for the network part, we could integrate it with usbip [4].
2. Making use of CONFIG_USB_MON is a good idea. Indeed, it works only
in host mode, but we can still use it, without modification. What we
would then need is a gadget which forwards USB data to a host port,
and CONFIG_USB_MON would be able to log these forwarded transfers.

David, what kind of approach did you take in your experimentation on
the Nail Board? Did you develop something in user-space, kernel-space?
Is the code available somewhere?

Thanks,

Best regards,

Nicolas

[4] http://usbip.sourceforge.net/

9

Your first message did get through :slight_smile:

Wrt hardware:

An alternative is to use the serial console.
That way you can control the process.
You then could consider storing the log data on SD and make them
available for analysis afterwards.

Another alternative would be to use the upcoming Beagleboard XM board.
This has a network interface. Not sure though how it interfaces to the
rest of the system (if it uses an onboard usb-to-ethernet interface
obviously this won't help). Guess Jason or Gerald can tell you about
the XM.

Yet another option is to use a zippy board:
http://www.tincantools.com/product.php?productid=16147

Have fun,

Frans

10

Frans Meulenbroeks wrote:

Your first message did get through :slight_smile:

Wrt hardware:

An alternative is to use the serial console.

yes, use serial console so that EHCI and OTG are free for the sniffer.
I would worry about how to extract the sniffer data later, let's make
it first pass traffic from one usb port to the other cleanly...

That way you can control the process.
You then could consider storing the log data on SD and make them
available for analysis afterwards.

Another alternative would be to use the upcoming Beagleboard XM board.
This has a network interface. Not sure though how it interfaces to the
rest of the system (if it uses an onboard usb-to-ethernet interface
obviously this won't help). Guess Jason or Gerald can tell you about
the XM.

XM uses an onboard 4 port hub with ethernet (so 5 ports in total)

11

The xM does use USB-to-Ethernet. Obviously this takes USB bandwidth, but that doesn't mean it can't be used. Also, you can capture in bursts where you pass data as quick as possible and halt other downstream data, then go into a slower pass-through mode at other times.

12

Nicolas,

my experiment consisted of using a usb->rs232 adapter, the simplest item i could find to start with. i used the serial gadget driver as a base on the slave side, and user the matching kernel driver on the host side(ftdi_sio). in between i used a variation of the inputattach.c application. basically it worked like this:

usb-rs-232 device -> host port -> host driver -> usbattach.c(user space

usbattach.c(user space) -> usb gadget driver -> slave port -> host pc

i never posted the code and left it at my former place of employer's office when i left last year.

thanks
Dave

13

All FYI. Here’s my reply to Nicolas. The intention of usbscope was not to “emulate” anything on the OTG port as this limits what you can connect and analyse. While you may only need to emulate the USB class (e.g. HID, NIC etc) this becomes more of a problem when dealing with complex USB devices… hence emulating a USB hub is a much more generic solution.

If you only want to capture USB traffic, then there are plenty of software tools available for Linux, OSX and Windows that already do this. If you want to get a bit lower down the stack, analyse more than messaging, tinker with the communications, go outside normal operation or test error conditions, then you need to get down and dirty with the EHCI driver…

14

Agree with that. I was more concerned about making the development process as easy as possible (for things like transfering files), and an Ethernet connection makes it very convenient, but, yes, for small files, we can also use the serial port.

Thanks.

Nicolas

15

Nicolas,

my experiment consisted of using a usb->rs232 adapter, the simplest item i could find to start with. i used the serial gadget driver as a base on the slave side, and user the matching kernel driver on the host side(ftdi_sio). in between i used a variation of the inputattach.c application. basically it worked like this:

usb-rs-232 device -> host port -> host driver -> usbattach.c(user space

usbattach.c(user space) -> usb gadget driver -> slave port -> host pc

Ok that was more or less my original idea, that is with user-space drivers.

i never posted the code and left it at my former place of employer's office when i left last year.

Ok no problem, thanks anyway.

Best regards,

Nicolas

16

Michael,

All FYI. Here's my reply to Nicolas. The intention of usbscope was not
to "emulate" anything on the OTG port as this limits what you can
connect and analyse. While you may only need to emulate the USB class
(e.g. HID, NIC etc) this becomes more of a problem when dealing with
complex USB devices... hence emulating a USB hub is a much more generic
solution.

I think I misunderstood your idea earlier. Indeed, emulating a USB hub is a great idea: the job of the hub is to forward the data anyway. Now I'm wondering if it is possible to create a hub gadget (I know a bit about usb devices, but I should research on how hubs work).

Without a hub, to emulate a device, I would follow the following procedure:
  - Read the configuration from the device, and "display" that configuration on the OTG side.
  - For all endpoints of the device (according to the configuration), listen on the same endpoint on the OTG side, and forward the data (that's for data going from PC->device, the same can be done for the other direction).

Now, I am wonder how low-level it is possible to be with the OMAP device controller. I see that "Many Texas Instruments OMAP processors have flexible full speed USB device controllers, with support for up to 30 endpoints (plus endpoint zero)." (USB_GADGET_OMAP, [1]). So I assume that endpoints you want to work with need to be somehow "registered" on the controller, and that you cannot just say that you want to receive anything that comes (but I may be wrong).

If you can emulate a hub, then you may be able to get away with the endpoints, and get something much more generic and flexible.

If you only want to capture USB traffic, then there are plenty of
software tools available for Linux, OSX and Windows that already do
this.

I still see potential applications, even if you stay at a high level, for example:
  - If you want to be transparent to the software
  - If you use other host OSes, or to debug the USB stack you just developed for your new OS. It could also be useful to debug bootloaders for example.
  - If your host system and device are both proprietary.

If you want to get a bit lower down the stack, analyse more than
messaging, tinker with the communications, go outside normal operation
or test error conditions, then you need to get down and dirty with the
EHCI driver...

That would be very interesting too. The question is what kind of low-level control do the EHCI and OTG controllers give you.

Thanks,

Best regards,

Nicolas

[1] http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob_plain;f=drivers/usb/gadget/Kconfig;hb=HEAD

17

Note that a hub will cause more impact on the system to-be-sniffed.
E.g. if you decide you're doing a 2.0 hub and I am sniffing a 1.1
device the some things will show up differently at the host.
Preferred solution is to be as little intrusive as possible.

Frans

18

> Michael,
>
>> All FYI. Here's my reply to Nicolas. The intention of usbscope was not
>> to "emulate" anything on the OTG port as this limits what you can
>> connect and analyse. While you may only need to emulate the USB class
>> (e.g. HID, NIC etc) this becomes more of a problem when dealing with
>> complex USB devices... hence emulating a USB hub is a much more generic
>> solution.
>
> I think I misunderstood your idea earlier. Indeed, emulating a USB hub is
> a great idea: the job of the hub is to forward the data anyway. Now I'm
> wondering if it is possible to create a hub gadget (I know a bit about
> usb devices, but I should research on how hubs work).

Note that a hub will cause more impact on the system to-be-sniffed.
E.g. if you decide you're doing a 2.0 hub and I am sniffing a 1.1
device the some things will show up differently at the host.
Preferred solution is to be as little intrusive as possible.

In this scheme, sniffing a USB 1.1 device, a physical hub is unavoidable. The
EHCI port is purely a USB 2.0 HS only interface; a hub is needed for it to
even see the FS/LS devices. So chaining a virtual hub in there may not be as
bad as it may seem.

Emulating/handling other details like power control might prove to be more
difficult/troublesome.

19

I believe that hub emulation with sniffing is by far the most reusable approach. I'd love to see it used to connect a series of virtual gadget drivers, instead of attempting composite drivers that aren't supported on all systems. That would even give Windows users gadget serial and gadget Ethernet at the same time to their BeagleBoard.

The emulated hub gadget would give people code to make real hubs and to easily test new gadget drivers. The sniffing ability could be used in the debug of both gadget drivers and real devices.

20

The only other “gotcha” that springs to mind is that if you aren’t emulating the device then you’ll need to stop linux’s normal USB device processing (i.e. device attached, wait for it to settle, scan it, load drivers etc). There are a couple of options there, quick and dirty would be to hack the existing kernel drivers for the EHCI controller. A better option would be to add something into linux’s generic USB code to say ignore that piece of hardware and instead have it talk to this kernel driver / userspace program.

There’s no reason that I could find why you couldn’t build a USB hub class gadget driver and then flesh out the gadget code in the linux kernel to provide plugability of multiple drivers. That piece really should involve the relevent maintainer and the larger linux kernel community… (Least of all they might do the work for us :slight_smile: If they aren’t as keen then there’s nothing stopping us for doing the code and then submitting it either…!

And yes, there is potential for all kinds of applications if that works. (I already have a number of projects where the USB OTG port has to flick between NIC and storage devices… its a real pain in the neck and messy…!)