Vert.x the perfect reactor for IoT devices
When you build an IoT device, you need to verify that the main goal can be reached. Ours is analytics capability, no worthy service can be built without time-series analytics!
So far we’ve been working on a simple Raspberry Pi, using Warp10 and a first stage analysis with Warpscript. But in order to build a true product over itm we need to manage the BLE connection, expose a REST API, receive messages from Beddit sensor, decode and send them into Warp10’s ingress endpoint, serve a progressive webapp, authenticate the user, etc…
Even with a quad core processor Raspberry PIs, efficiency is key.
A classical engineering design would use a multi-threaded approach coordinated with locks. It is proven to work, but such approach has scalability problems, been there, done that…
This time we have opted for a Reactive approach based on Eclipse Vert.x. Vert.x is a toolkit for building applications on top of the JVM, another cool Open Source project of the Eclipse foundation.
Vert.x toolkit implements the reactor pattern as its core and is well adapted for the C10K like problems. You could think, “cool but where is the relationship between the C10K problem and IoT?”
Well, certainly we won’t handle 10K clients at the same time on the Rapsberry PI, but solving the C10K implies to be resource efficient which is one of the properties IoT devices must have.
Be specialised for a task, Verticles as “role model”
Verticles are chunks of code that get deployed and run by Vert.x. Our project is divided in 4 verticle each having a different role:
- The Main verticle the first launched, manages the deployment and undeployment of the other Verticles
- The Bluetooth verticle manages the BLE aspects such scanning, pairing and device communications
- The StreamProcessing verticle talks with the Beddit sensor protocol and transcode dataframes into Warp10 input format, stacks them in a heap and every second flushes it intoWarp10
- The WebEntrypoint verticle, exposes a web service API and resources
But, all this “happy crowd” needs to communicate in a “smart way”, and for IoT devices, a smart communication is an asynchronous communication.
Event bus the “asynchronous” timing belt
Vert.x integrates a light-weight messaging system, an event-bus that allows different parts of the application to communicate in a loosely coupled way. We uses the event-bus as an “asynchronous” timing belt, allowing the different verticles to exchange messages.
For example, the Bluetooth verticle receives datagrams (BLE notifications ) from the sensor during the night. These datagrams arrive with a frequency of some 150 messages per second, so the Bluetooth verticle must deal with the current message fast enough to be ready for the next one. In this situation each received message is published via the event bus to a decoder that will decode it without blocking data acquisition, as the data acquisition thread has been freed from the decoding task.
This mechanism allow to spread the work load on the different cores of the PI CPU in a very efficient manner. If one software component needs more resources during a short time, its impact on other components is very limited.
And for manage a BLE device, is there a magic library for that?
Yes… and no! Managing BLE device is effortless on Raspbian, since all can be done with command line toots likes hcitool and gatttool. Many resources in the Maker community talk about Bluetooth, for example this one from Adafruit is very comprehensive. But how can we spawn, kill and read and write data streams from Vert.x without blocking anything?
I love it when a plan comes together
Once again, the Vert.x team has a solution for that. This GitHub repository caught our attention. It is just a complete extension for manage child processes form Vert.x written by Julien Viet, Vert.x core developer and team leader. No wonder we were charmed by the Vert.x community once again!
After a couple few bugs corrected (located between chair and keyboard), this well-oiled machine receives, decodes and records around 150 datapoints per second with a very small memory and CPU footprint.
Definitively, Vert.x is the best IoT toolkit!
Furthermore, if you prefer to read the source, just go on our Github repository!
