usb (usb-serial) problems on BBB

I have problems getting a usb-serial adapter (FTDI-based) to work on my BBB.

When it is connected the BBB dies after some time with a kernel oops: screenshot

This problems exist both on stock Angstrom kernel (i.e. running 3.8.13-r23a.43 right now) and also RNelson’s Ubuntu kernel (bug report here).

Any ideas or suggestions how to fix this?