Wednesday, 9 June 2010

Bidirectional communication with an Arduino

I now have transparent communication with an Arduino (programming the device work, as well as serial communication with the loaded program). You can find the proxy driver code here, or fetch the whole branch here.

Basically, the Arduino appears as a USB->serial converter to the host, with 2 bulk endpoints, with directions IN and OUT (for both directions of the serial transfers):

Bus 007 Device 120: ID 0403:6001 Future Technology Devices International, Ltd FT232 USB-Serial (UART) IC
Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 2.00
bDeviceClass 0 (Defined at Interface level)
bDeviceSubClass 0
bDeviceProtocol 0
bMaxPacketSize0 8
idVendor 0x0403 Future Technology Devices International, Ltd
idProduct 0x6001 FT232 USB-Serial (UART) IC
bcdDevice 6.00
iManufacturer 1 FTDI
iProduct 2 FT232R USB UART
iSerial 3 A800evGn
bNumConfigurations 1
Configuration Descriptor:
bLength 9
bDescriptorType 2
wTotalLength 32
bNumInterfaces 1
bConfigurationValue 1
iConfiguration 0
bmAttributes 0xa0
(Bus Powered)
Remote Wakeup
MaxPower 90mA
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 0
bNumEndpoints 2
bInterfaceClass 255 Vendor Specific Class
bInterfaceSubClass 255 Vendor Specific Subclass
bInterfaceProtocol 255 Vendor Specific Protocol
iInterface 2 FT232R USB UART
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0040 1x 64 bytes
bInterval 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x02 EP 2 OUT
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0040 1x 64 bytes
bInterval 0
Device Status: 0x0000
(Bus Powered)

The first endpoint (EP 1 IN), for communication from the Arduino to the host, worked almost immediately: apart from the transfer type, this is the same as for HID devices.

Getting the second endpoint (EP 2 OUT) to be configured properly on the gadget side was a little tricky. In fact, the infrastucture provided in drivers/usb/gadget/epautoconf.c, namely the function usb_ep_autoconfig, does not allow to choose an endpoint address: This function looks at the list of endpoints provided by the gadget controller driver, and simply selects the first endpoint that supports the request transfer type, direction and maximum packet size (here, Bulk OUT, 64 bytes). In our case, the function selects EP 1 OUT, while the host still thinks transfers should be done on EP 2 OUT.

To get it to work, I had to rewrite usb_ep_autoconfig (see find_gadget_endpoint), and I was a little horrified to see the way gadget controllers endpoints are advertised: Each endpoint structure has a char* name field, which tells the endpoint address, direction, and possibly the supported transfer types. To find EP 2 OUT, the function has to go through all the endpoints, until it finds one whose name is ep2out. A bit kludgy, but well, it works, at least for the MUSB controller...

Data transfers are handled differently, whether they happens on a IN, or OUT, endpoint. For IN endpoints, we follow the following procedure:
1. Initialization (see bridge_endpoint): We submit an URB (USB request block) to the device, asking for data.
2. Device callback (see device_epin_irq): We got some data. Copy it to a request (the gadget equivalent of an URB), and submit it.
3. Gadget callback (see gadget_epin_complete): The request was submitted correctly. Resubmit the URB, so we can receive the next packet.

OUT endpoints are handled similarly: Upon initialization, a request is submitted (bridge_endpoint), the gadget callback (gadget_epout_complete) submits an URB. When the URB completes (device_epout_irq), the request is resubmitted.

This method, that is, with only one URB/request "flying" at the time, seems to work well for low throughput applications, but buffering may be required for higher throughput.

Sunday, 6 June 2010

Weekly report - Week 2

Status:
  • Modified the proxy driver to be more generic: descriptors requests are forwarded to the device, and the descriptors are parsed (and modified if required) on the fly: but still, only interrupt transfers from the device to the host on endpoint 1 are supported.
  • Ported my set of patches to the kernel 2.6.35-rc1 (i.e., the latest vanilla tree): a few functions got renamed, but nothing major. I compiled and ran that kernel on the BeagleBoard. I had to tweak some files to get the MUSB block to accept being run in peripheral-only mode (commit), but the MUSB block seems buggy (even with the Ethernet gadget), so I gave up on that. From a discussion on #beagle, it seems like it would be better to use the linux-omap git tree (but I don't know which branch), and apply "some patches" from Angstrom.
  • Started working with a new device: the FTDI USB->Serial converter found, for example, on Arduino boards. Device->PC communication uses endpoint 1, in bulk mode, and worked with little modifications of the existing code.
Plans:
  • Get PC->Device communication to work with the Arduino, which uses endpoint 2 in bulk mode (to get at least one OUT endpoint working).
  • From there, dynamically connect all endpoints advertised in the configuration.
Risks/problems:
  • So far I've been targeting kernel 2.6.32 (from angstrom-linux/beagleboardXM), but at some point I would like to target the latest kernel, so I would need to get the MUSB block to work properly with a recent kernel.
  • I have encountered strange problems with a keyboard: pressing num/caps lock only takes effect at the next key stroke, I need to investigate this.

Wednesday, 2 June 2010

Towards a more general solution

I spent the last few days generalizing my proxy driver, in such a way that most HID devices (mice, keyboards), should now be working. Actually, any device that uses only endpoint 1 in interrupt mode and IN direction should be supported by the current implementation.

There are no static descriptors hard-coded in the driver anymore, so GET_DESCRIPTOR requests are intercepted, and the configuration descriptors are parsed and cached (using functions found in the kernel). Also, the descriptors are modified on the fly, to compensate for the fact that the communication between the PC and the BeagleBoard happens in Full Speed, while the BeagleBoard-device communication uses Low Speed.

Two fields are dynamically modified:
  • bMaxPacketSize0 in the device descriptor. It seems like the MUSB driver only support 64 bytes as maximum packet size (which is what is mandated by the High Speed mode), while the mouse uses 8 bytes (the only possible value for Low Speed devices). In theory, Full Speed devices can advertise a maximum packet size between 8 and 64 bytes, but the MUSB driver does not seem to support that.
  • bInterval in endpoints descriptors. As mentionned before, the meaning of that field varies with the device speed.
To get the proxy driver to work, you need to follow these steps:
  • Clone my kernel git tree. Use the backup-20100602 branch.
  • Compile the kernel with the default beagleboard configuration (see here). You just need to add CONFIG_USB_G_PROXY=m. I also disabled MUSB in host and OTG mode, as well as USB suspend, but this may not be necessary.
  • Install the new kernel on the board.
  • Clone the helper scripts git tree, and copy the content of the arm directory to the BeagleBoard (you need to modify load/setup scripts if you do not have have a copy of musb_hdrc.ko and g_proxy.ko in the same directory).
  • Run ./setup on the BeagleBoard, this will unload the g_ether gadget driver.
  • Plug your USB hub + a HID mouse/keyboard to the BeagleBoard (this can be done earlier).
  • Plug your PC to the BeagleBoard USB slave port (this can be done earlier as well).
  • Run ./unbind: This will unbind the device from the normal Linux driver.
  • Run ./load: this will (re)load the g_proxy driver.
  • Use the mouse/keyboard, it should work.
The code is still full of TODO/FIXME, and is rather likely to cause kernel panics.

Monday, 31 May 2010

Weekly report - Week 1

Status:
  • Managed to build Angstrom's console-image, as well as the kernel (outside OpenEmbedded), both work well on the BeagleBoard.
  • Wrote a prototype kernel proxy driver, and managed to get USB data to be forwarded properly between a USB mouse and a Linux PC, with the BeagleBoard in between (git tree here, but no really usable yet).
  • Modified the MUSB controller driver to disable high-speed USB if the gadget asks for full-speed (commit here).
Plans:
  • Get the proxy driver more generic, and get completely rid of the static descriptor copy.
  • Dynamically connect endpoints, according to the endpoint descriptor.
Risks:
  • Differences between low-speed, full-speed, high-speed, for example in the meaning of some descriptor fields (e.g., bInterval for endpoints): since the BeagleBoard MUSB cannot operate in low-speed, this would require some "rewriting" of the descriptors to get the device to work.
  • Also, my Linux PC asks for the Device_Qualifier descriptor (a high-speed only descriptor) when connected to the BB, but not when connected directly to the mouse, even though the device is advertised and detected as full-speed.

Thursday, 27 May 2010

A first device works: USB mouse

Today I managed to get the following to work: I connect a USB mouse to the host port of my BeagleBoard (via a hub, since the BeagleBoard does not support low-speed USB), and the slave USB port of my BeagleBoard to my PC. Then, I can use that mouse to control the cursor of my PC.

For that, I have a "proxy" kernel driver on the BeagleBoard, that is registered both as a USB device driver (on the EHCI side), and as a USB gadget driver (on the MUSB/OTG side).

Most USB drivers register at the interface level (that is, as a struct usb_driver), so control only a part of a device (one device could have multiple interfaces, for multiple functions). That is not what we want, so, instead, the proxy driver registers as a struct usb_device_driver, that claims the device at a lower level. Normally, there is only one of such low-level drivers, found in drivers/usb/core/generic.c. This generic driver takes care of all USB devices attached to the system, and looks for interface-level drivers for each of the interfaces of these devices.

Since the generic driver claims all new devices (and never releases them), the trick is to unbind the device from the generic driver, so that it can be claimed by our proxy driver. This is done using a command like:
echo 1-2 > /sys/bus/usb/drivers/usb/unbind
Once "unbound", our driver is able to claim it.

On the gadget side, there is nothing special: the driver is registered as a struct usb_gadget_driver. Then, we do everything we can to make the PC believe that it is connected directly to a USB mouse. For that purpose, we reply to GET_DESCRIPTOR control requests from the PC with the following device descriptor:
Bus 001 Device 005: ID 046d:c00e Logitech, Inc. M-BJ58/M-BJ69 Optical Wheel Mouse
Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 2.00
bDeviceClass 0 (Defined at Interface level)
bDeviceSubClass 0
bDeviceProtocol 0
bMaxPacketSize0 8
idVendor 0x046d Logitech, Inc.
idProduct 0xc00e M-BJ58/M-BJ69 Optical Wheel Mouse
bcdDevice 11.10
iManufacturer 1 Logitech
iProduct 2 USB-PS/2 Optical Mouse
iSerial 0
bNumConfigurations 1
Configuration Descriptor:
bLength 9
bDescriptorType 2
wTotalLength 34
bNumInterfaces 1
bConfigurationValue 1
iConfiguration 0
bmAttributes 0xa0
(Bus Powered)
Remote Wakeup
MaxPower 98mA
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 0
bNumEndpoints 1
bInterfaceClass 3 Human Interface Device
bInterfaceSubClass 1 Boot Interface Subclass
bInterfaceProtocol 2 Mouse
iInterface 0
HID Device Descriptor:
bLength 9
bDescriptorType 33
bcdHID 1.10
bCountryCode 0 Not supported
bNumDescriptors 1
bDescriptorType 34 Report
wDescriptorLength 52
Report Descriptors:
** UNAVAILABLE **
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 3
Transfer Type Interrupt
Synch Type None
Usage Type Data
wMaxPacketSize 0x0004 1x 4 bytes
bInterval 10
Device Status: 0x0000
(Bus Powered)
Currently, I have a copy of the descriptor statically defined in the driver, but of course, in the future, this descriptor will have to be read from the device itself.

Then, whenever the drivers receives a control request it does not know how to answer (for example, a class-specific HID control request), it forwards the request to the device, waits for the reply, then writes back the reply to the PC.

With that, the mouse is fully recognized by the PC, but nothing happens when you move it, because mouse events use interrupt transfers on endpoint 1. That can be fixed by listening for interrupt transfers from the device, and forwarding those to the PC.

At that point, the cursor moved, but with some huge latency: clicking was fast, but there was a delay of a few seconds when moving the mouse around... This was due to the following: the mouse is a low-speed USB device, and its endpoint descriptor asks the host to poll it for events every 10 ms (bInterval = 10). However, the MUSB device controller on the BeagleBoard appears as a high-speed device (there is probably a way to force the controller in full-speed mode, but the current driver does not seem to support that), and the definition of the polling interval, for a high-speed device, is 0.125 ms * 2^(bInterval-1). By setting bInterval to 10, the BeagleBoard device controller only got polled every 64 ms, so, when I moved the mouse, this created a lot of events, which got buffered somewhere in the BeagleBoard, until they could be released to the PC.

I fixed the problem by modifying the descriptor, and putting bInterval = 7, that is a poll every 8 ms.

I can write up some instructions about how to get this setup to work, but I guess this is not so useful if you do not have the same mouse as I have, or a similar enough mouse at least...

All the kernel modifications have been pushed in my git repository. The proxy driver code, in particular, can be found here, but the code is still far from clean and full of TODO/FIXME tags.

Tuesday, 25 May 2010

Received my BeagleBoard!

This morning I received a package from Digikey (thanks to Cathy!), with a BeagleBoard, a 5V power adapter, and a IDC10-DB9 cable.

I connected the BeagleBoard to my PC, using the ICD10-DB9 cable, followed by a USB->RS232 adapter. Then I powered it on, some leds blinked, and, nothing on the terminal... I then tried connecting it to another PC, with a serial port, nothing again.

I started wondering if I needed a null-modem cable, so I checked the pinout on the ICD10-DB9 to check if it wasn't one already. And I realised that the pinout was completely non-sensical (ground connected to RX, ring signal connected to TX)... Since the colors on the RS232 wires are not standardized, it looks like the people who built the cable did not realize that the colors changed with their latest supply of cables...

It seems like the problem affect quite a few people who ordered the cable recently, so here is how to fix it:
  • Take out all the wires from the ICD10 connector (easy to do with a needle).
  • The red and green wires should be together on one pin. Cut the green one.
  • Then, connect the red wire (TX, pin 3) on pin 2 of the BeagleBoard (RX).
  • The brown one (RX, pin 2), on pin 3 of the BeagleBoard (TX).
  • The yellow one (GND, pin 5), on pin 5 of the BeagleBoard.
  • Leave the other wires dangling, you don't need them (beware of shorts though, I taped them to avoid problems).
A picture of what it looks like in the end:


Note that this is only for this batch of cables, and it's always better to check the wiring with a ohmmeter, if you have one (the RS232/DB9 pinout can be found here).

Monday, 24 May 2010

Weekly report: Week 0

First, I should receive my BeagleBoard tomorrow, so I can get up to speed on the project.

Status:
  • Tried to build Angstrom (the latest git version), but ran into disk space problems (20+ gigs do not seem to be enough). Since I only need to do kernel development, I can also use demo images, and pre-built toolchains.
  • Downloaded the pre-built toolchain (angstrom-2010.4-test-20100421-x86_64-linux-armv7a-linux-gnueabi-toolchain-qte-4.6.2.tar.bz2).
  • Built a Beagleboard kernel based on angstrom-linux/beagleboardXM, using that toolchain.
  • Formatted a 1GiB SD-card according to the guide, and copied the content from the demo image to it. Since my SD-card is only 1GiB, and the image is 2GiB, I "dd"-ed the first FAT partition (to make sure that files are in the right order), but copied all the files on the ext3 partition.
Plans:
  • Try to finish building Angstrom (I freed some disk space, and maybe I messed up somewhere).
  • Test the SD-card I formatted.
  • Replace the kernel on the SD-card with the one I built.
  • Use the newest root filesystem (if I manage to finish building it).
  • Test the USB gadget and EHCI controllers with a few different drivers/devices.
  • Start working on the proxy driver (i.e., the project itself).
Risks:
  • Unable to build Angstrom: nevermind, I can use demo images (at least for now).
  • ?