A proof of concept to generate a colormap of urban traffic condition and vehicle diagnostic
Urban life sometimes seems to be governed by chaos, and maybe one of its most representative manifestations is the traffic.
In this Medium article we will show you our proof of concept to generate an interactive map showing the real-time traffic condition as well as diagnostic information of the vehicles.
It’ll not certainly provide a complete solution, but it would give us knowledge about the phenomena, and developing a simple proof of concept it’s always the best way to start, isn’t it?
Ingredients:
- OBD connector
- An Android App
- Apache Edgent
- Cloudant DB
- Elastic & Kibana
OBD
The backbone of our idea was an OBD connector, a small device that when attached to your vehicle acts as a bridge and expose the vehicle’s information using a Bluetooth interface.
These type of connectors are cheap and can be found on Amazon (et simila) for around 10–11$.
The information exposed may vary from car to car and can depend on the version of the OBD connector used too.
Considering that the OBD connector is supposed to be attached to a car to work correctly this creates some problem when developing our mobile application, I think that everyone agrees that it’s better to develop things at your desk than in your car with your notebook.
For that problem there are three solutions: build and hardware simulator, buy a hardware simulator or use a software emulator.
We chose to go for the third option an decided to use OBDSim, an old emulator with a spartan UI which emulates a small subset of sensors of a vehicle.
At this point, we had everything we needed to develop our mobile application
The Android application
Our idea required the data to be sent to a remote server in order to analyze them and produce meaning infographics.
For this task, we developed a simple android application which read data from the Bluetooth interface of the OBD connector and sends them using a lightweight connectivity protocol to our server.
The app was also supposed to handle the setup of the first connection with the OBD connector, the connections errors that could occur and the GPS location which will be used, later on, alongside the data coming from the vehicle.
The app also provides a painful spartan UI to see at glance the information that is getting read from the OBD connector.
The Android app at the end was written using Kotlin and made extensive use of two libraries:
- OBD-Java-API: a community-developed set of APIs to write queries for the OBD2/ELM327 module
- Car-monitoring-edgent library: our library to pack data and create a stream to a remote service using Apache Edgent
The car-monitoring-edgent Library was developed specifically for this project and got used to continuously send data in the background to an IBM Watson instance using the MQTT protocol, it also takes care of aggregating the data coming from the sensors of the car and the GPS of the smartphone.
Now we have the components to read and send data to the internet but what are we going to do behind the scenes?
IBM Watson IOT Platform & Cloudant Database
These two technologies are going to allow us to receive data without worrying about device-specific connection issues and directly store that in JSON format. The former is a message hub server that continuously listens for connection by potentially thousands of IOT devices, the latter is an easy configurable way to store all data incoming to Watson.
The first step is to create an IBM Cloud Services account and download the IBM CLI, then from Bluemix dashboard we create an instance of Watson IOT platform and an instance of Cloudant.
Now we have our resources, but how can we redirect the traffic from Watson to our DB?
First of all, because IOT platform didn’t migrate yet to the new IBM Cloud infrastructure we need to create an alias of our Cloudant instance in our Cloud Foundry project space through the CLI with this command: https://cloud.ibm.com/docs/cli?topic=cloud-cli-ibmcloud_commands_resource#ibmcloud_resource_service_alias_create
Since we have our alias now, we can actually bind our database to our instance of Watson IOT Platform. To do that, we simply navigate to the “Extensions” tab in Watson dashboard and do the binding by creating a new Cloudant extension, selecting a time bucket (day, week, month), a name for the table and the Cloudant instance.
The last step is to set up an IBM Function (in our case written in Python) in order to mirror every insertion in Cloudant DB to our instance of Elastic DB.
To do that we wrote a simple bash script that relies on IBM CLI commands.
First of all, our script creates a trigger linked to our Cloudant DB that is fired on each insertion, then it’s brought up a sequence of actions executed on trigger firing and finally that sequence is mapped to our trigger, by creating a “rule”.
The first action reads the inserted data and passes it to our custom Python action, that submits a POST request to the Elastic DB.
Now all data stored in Cloudant is also stored in Elastic DB. Nice, don’t you think? But what are we gonna do with all that almost meaningless data?
Elastic & Kibana
In order to take advantage of the data received we are going to use two technologies Elasticsearch and Kibana.
Elastic is a powerful and Open Source search engine. It allows submitting plaintext queries as well as more structured ones over collections of JSON documents. Kibana is built on top of Elastic and provides a console to interact directly with the search engine for example for asking queries or manipulating data. Moreover it offers a wide choice of tools for visualizing data.
At a conceptual level documents are organized in Indices, in a one to many relations (an index has multiple documents and each document belongs to one index).
The concept of an index in elastic is similar to the one of tables in SQL databases. When we create a new index we must specify the name and the type of the fields, as well as in SQL when we create a new table we define its schema.
To create the index enter the Kibana console, navigate on the Dev Tools and send the following request.
From now on, to add documents to the index it’s sufficient to send the JSON data inside the body of a PUT HTTP request, specifying in the URL the name of the index and the id of the new document.
Once we have the data we can create the interactive map simply adding a new geopoint visualization to the dashboard, selecting “speed” as aggregation field.
As we can see, speed ranges are automatically updated according to the minimum and maximum speed of the vehicles in the visualized area.
If we hover on a given bucket we can read the average speed in that area, that could be an entire city, or a district as well as the speed of a single vehicle depending on the current zoom on the map.
We also created a bar chart counting how many vehicles have the engine temperature in a given range. To do that it’s sufficient to add a “vertical bar” visualization to the dashboard specifying the aggregator (in our case the engine temperature) and the ranges we’re interested into, Kibana will do the rest.
Improvements
With this simple demo, we showed how to collect information from the physical devices, how to filter data, how to aggregate information and how to present them with a visual representation.
There is room for improvement on the data extraction part in order to prevent synchronization issues (between GPS and OBD) and the UI has to be reworked.
The next step would be to expose that information through APIs, that could be used for example by GPS navigation devices to always suggest the best route according to the real-time traffic condition, therefore reducing the global amount of traffic. Other users could be car producers, which could use that information to make statistics about the status of their vehicles.
Source code
The source of this mini project can be found on GitHub.
Team
This group-project was developed as part of the IoT course taught by Ioannis Chatzigiannakis at the Sapienza University of Rome.
The group consisted of:
