Web enabling legacy devices
Recently, I’ve become quite fond of WebUSB and WebBluetooth and the possibilities it brings — and not only for the obvious geek factor.
UPDATE 2017–09: WebUSB has changed some since the posting of this article — including being released in Chrome M61. A followup post is on the way and will be linked here!
What was before?
To give a genuine idea of the struggles normally involved in getting USB communication devices working across different platforms, let me take you back to 2013, when some friends and I worked hard on delivering a cheap science platform for developing countries in form of the empiriKit project.
The task we gave ourselves was to find the easiest (for the user) way to connect low-cost sensor hardware to a web application running science experiments. The goal was two-fold:
- to teach science to school and high-school students, especially targeting rural areas in developing countries
- to encourage the students to get excited about writing (JavaScript) code
In our opinion, these two combined could give the students the necessary knowledge to start building something with local businesses — before the smartest students would leave for university and potentially flee the country.
…a catalyst for bright young minds to innovate with modern technology and create solutions with local businesses
After some iterations, we found an affordable Freescale (now NXP) @ARMmbed enabled board with some sensors onboard and decided to go with USB CDC (the formal class name of USB Serial devices, that e.g. on Windows will mount as a COMxx device) for our communication needs. In order to support schools with nothing but a corner TV, we first decided to use a RaspberryPi as the host computer. However, the complexity for the users and relatively high cost of all components made us transition to chrome.serial (accessible to so-called “Chrome Apps”), which — at the time — would run on most existing computers in the target regions. This is when all the “fun” started:
- On Linux, apparently most devices exposed as USB CDC devices are hijacked for and being probed for around 16 seconds by the ModemManager before being released again— because for some reason, I can’t understand, this is seen as the best approach. The solution is to get your device’s VID/PID blacklisted in the official ModemManager’s udev rules and waiting for the different distros to be kind enough to pick it up. ChromeOS was actually quite fast (about 3–5 months until landing in stable), while Ubuntu didn’t upgrade ModemManager until approx 2.5 years later (!!!!). This mechanism really have to change.
Ubuntu didn’t upgrade ModemManager until approx 2.5 years later
- On Windows, usbser.sys, a standard USB Serial driver is shipped (since Windows 8.1 and available for Windows 7) as part of the platform. Unfortunately, before Windows 10, even with perfect USB CDC descriptors in the device, this still required a signed “driver”, which is actually just a signed INF text file binding the VID/PID to usbser.sys. I imagine the initial discussion at Microsoft has been something like: “Yeah — average users will probably not accept it for memory sticks, keyboards and mice, but let’s make it really hard for manufacturers not paying an annual certificate fee to get their CDC devices hooked up…muhahaha”/s. — at least it was a huge pain for us. The person in Microsoft deciding to do the right thing for Windows 10 should get an award!
- On OSX, things seemed to run perfectly as (at least so it appeared) they solved the “is this a modem” discussion (see for Linux above) by providing two /dev/ttyXXX paths — one clearly marked as a modem.
- ChromeOS did not work for us initially — but because of a super swift inclusion of the VID/PID blacklisting, this became our primary demo platform when we went out on workshops or presentations.
the VID/PID we initially obtained was a kind help from Freescale, that could allow projects under certain conditions and very limited volume to temporarily “sublet” in their VID space.
After the worst issues were fixed for each platform, Haswell based laptops started to spread — and with them a new technology, USB3, which did not work with the USB CDC driver layer in @ARMmbed initially and it took us a long time figuring out that USB3 was the issue. Only after Apple responded to our bug report and tried out the board and firmware for themselves (Apple was a fast mover on Haswell and USB3, so we thought it was an OSX issue) — they helped us narrow down the problem. . o O (So much joy with USB ;))
Some time passed, where we didn’t find any suitable solution — but to our luck, a cool guy, Mike, found a fix and we could finally get back to focusing on the Web science experiments connecting to the device.
Shipping hardware, without proper funding in the project, was an impossible task
Unfortunately, we hadn’t really thought through how to make a viable business out of the solution — we just created it with altruistic motives in our spare time — and shipping hardware, without proper funding in the project, was an impossible task — especially when the target users are poor people in remote regions that not many other care about. — we learned our lesson… I hope.
The quality of the sensor readings of the mobile sensors varied too much for real science projects
We had to attend to our respective money generating projects for a while, but did manage to make a revival based on a mobile web app that went to a few pilots. During the time we had worked on the initial version, Android (and to some extent iOS) was available all over in the regions, we were aiming for. So naturally, we though: “Hey! Let’s make a pure web solution — no additional hardware needed to ship and no Windows drivers issues to stop people from using it”. However, the quality of the sensor readings of the mobile sensors varied too much for real science projects (fine for fun and exploration though).
What now?
WebBluetooth arrived, making it much more attractive to use the mobile solution to connect to higher quality sensors when needed —we will look into that next and I’ll promise to share the code and a howto article about it. But first, let’s see if something can be done with the USB hardware we already made…
WebUSB that can really become a game changer for USB connected devices. However, there are a few concerns that needs to be addressed correctly before traditional (legacy) industry hardware manufacturers can see a clear benefit:
- For security reasons, WebUSB requires trusted origins to be listed in the USB firmware and sent over on initial handshake with the browser. This makes it hard for hardware manufactures working with OEMs or other partners to use it, as they will have no easy way of knowing who will use and host software to communicate with their hardware. Same applies to anyone using remote portals, partners or other to connect. At least some best practice guidelines for this scenario needs to be in place.
- Introducing WebUSB in a device requires the definition of a separate WebUSB enabled interface in the USB descriptors. As many vendors just use off-the-shelf STM or FTDI chips for communication — after all, they are not in the USB business themselves — it may be hard for them to figure out how to implement it. Again, guidelines and easy solutions need to be produced. I know that an online database is planned to cover the gap, but I see that as a very last resort (offline cases, who owns the database, generic VID/PIDs, etc.).
WebUSB enabling existing hardware
The sensor hardware we used for the initial iteration of empiriKit, the onboard FRDM-KL25Z sensors, only communicated using USB CDC, but for WebUSB to work, we had to change that. We still wanted to be able to use USB CDC for the non WebUSB case, so the solution would be to modify the USB descriptors to make the device a composite CDC+WebUSB device.
A composite USB device is a device that, when inserted in a host machine, will provide what appears like two or more separate devices.
Initial experiments were made with the reference WebUSB platform, the Arduino 101, that Kenneth Christiansen used for his cool demo:
At first we tried the pure Arduino code but quickly turned to Zephyr that seemed closer to our needs.
Because of the nature of the Arduino 101, WebUSB will not be available instantly over the USB (unless you already have external power OR wait for the boot and DFU mode to pass), so I really wanted to see if we could WebUSB enable the FRDM-KL25Z ASAP.
Now — I was no USB descriptor expert a few weeks back, so initially, I tried to merge some code from the Arduino & Zephyr WebUSB examples and hoped for the best. When that didn’t work, I went searching through the mbed.com site and to my luck, Devan Lai made some initial WebUSB DFU code to get me started (I was very excited at that point! :)).
Before going into some of the more boring trial and error details, I’ll jump right to the fun stuff:
As you can see, the composite CDC+WebUSB device works fine on Linux but not before making sure my user was in the ‘plugdev’ group and adding the following line to a udev rule file in /etc/udev.rules.d:
SUBSYSTEM==”usb”, ATTRS{idVendor}==”1209", MODE:=”0666", GROUP=”plugdev”
Now, some may claim that that is acceptable for average Linux users to go through — but USB is ‘Plug and Play’… and that’s no ‘Play’ ;).
On Windows, even though they fixed auto mounting of CDC interfaces (appears as a COM port, which Windows calls it’s serial port interfaces), generic USB interfaces (not HID, CDC, Mass Storage, etc.) still requires either a signed driver (again — just an INF file pointing to winusb.sys) or some slightly obscure Microsoft specific descriptors delivered from the USB hardware. Their own documentation on this is quite bad and fragmented, so instead, I’ll point you to this excellent tutorial by Pete Batard:
However, even with this in hand (and a few other similar sources), we still were not able to get first time use of our WebUSB enabled device to mount and function correctly in Chrome on Windows 10. Either the obscure MS descriptors are lacking a byte somewhere or Chrome — using libusb, AFAIK — does not play well with the default winusb.sys driver that seems to be applied automatically on first connection. The Device Monitor seems happy but but libusb doesn’t appear to be when listing connections using the xusb command line tool. I’ll update this article when a solution is found.
For now, use Zadig to get the ball rolling on Windows
Don’t panic just yet though, as it’s possible to at least get WebUSB running after applying the WCID+WinUSB driver combo from Zadig on the WebUSB interface! NOTE: At the time of writing, the current stable release of Chrome on Windows (only) does not show the notification in the upper right corner, when WebUSB hardware is detected. You should, however, be able to see it in chrome://device-log or when requesting access in JavaScript.
The source code for the firmware and the accompanying mini Polymer 2 test application can be found here. Polymer ROCKS by the way!
Alternative solutions
It should be noted that we also experimented with USB Serial connections through the use of native proxies, accessible to the browser:
- go-webserial-websocket creates a proxy between available serial ports and the browser by using REST & WebSockets. Unfortunately, this requires a native installation on the host machine AND will not work in mixed content scenarios (a remote HTTPS web application will not be allowed to access a HTTP service — and it’s not possible to get a trusted certificate for localhost without some serious hacks ;))
- go-webserial-native-msg runs as a browser extension using nativeMessaging to talk to a similar proxy app and while still requiring a native installation, it does not run into the same problems as above regarding mixed content.
NOTE: We decided to use Go for the native parts to make it easy to run on multiple platforms….and the gopher is an excellent partner when coding it too!
Seeing WebUSB work out of the box feels like pure magic
It’s been an interesting journey through the years and it looks like the browser APIs are finally making the need for many types of native apps disappear, making everything much easier for developers and users.
Seeing WebUSB work out of the box feels like pure magic, and I am confident it will be a strong winner for USB to browser connections once the issues mentioned above are fixed and — hopefully — more browsers will adopt it.
/ Lars
