Azure IoT Hub + Azure function + Azure Cosmos DB — Walkthrough

Avirup Basu
May 27, 2018 · 10 min read

Note: I am assuming that the reader has a basic understanding on node.js, mqtt, basic concepts of programming and moderate problem solving capability.

Introduction

We all know what is IoT atleast some if not all. For starters, this is not aanother IoT article where we are going to do home automation. This is a simple walkthrough about adding a serverless backend to your existing IoT system. Here we’ll see that how Azure IoT Hub will be used as an event hub or a messaging system for “Things” and how, these incoming messages can be routed to the related database using Azure functions.

We won’t be using any device out here and simulate our PC as a device and use node.js to connect to IoT hub and stream some kind of data. Then, we’ll see how the streamed data to the IoT hub can be routed to the db after some logic using azure functions and we’ll use C# for that. Finally, we will see how the integration happens where we connect every component to everything.

Architecture

Fig 1: Architecture basic

The above figurre depicts the basic architecture or flow of the system. It is also to be noted that we’ll be using cosmos db’s mongo db API to access the databse. The whole idea behind this is to depict the flow of data from a device to the databse using a serverless architecture.

Azure IoT hub

Azure IoT hub is essentially an IoT platform and using this we are going to send messages from our simulated device. Technically, the device section of any IoT system is referred to as an edge device. So from our simulated edge device, we are going to send sone random data to the azure IoT hub using MQTT protocol. But, let’s find how to set the Azure IoT Hub first. Head on to https://portal.azure.com and create a new IoT hub.

Fig 2: Azure IoT Hub — Screenshot

Once created, we will add a new device and name it something relevant. Here in my case, the name of the IoT hub is “spektro” and the name of the device is “simulatedDevice”. To create a device, initially click on the the IoT hub you just created and head on to the “IoT devices”, under the tab “Explorers”.

Fig 4: Screenshot — IoT Devices

So, we are done with the creation of device, the next step is to write a code, that will simulate the device and send some exciting raw data. For the code, we are gonna stick to node.js and there exists a certain package which eases the task for us. Use the npm package manager for installing the module.

After a successful installation, we can use the above dependency. Another important parameter we need to note is the connection string. This will act as a key for our simulated device. For this, click on the device id, you just created now. In this case it was “simulatedDevice”. Once, you click, you will land up in device details.

Fig 5: Screenshot — Device details

From this page, you need to copy the “Connection string — primary key”. This will allow the device to communicate. Copy the string and paste it somewhere locally.


Simulated Device — Node.js

It was already mentioned earlier, about the dependency, that needs to be installed. Here we’ll see how the dependency is used to stream some kind of data. Our data in this case will be a json string containing the following parameters.

DeviceID and Data

“Device id” is the one which was created and “Data ”is just some random value between 1 to 100.

In the above code, we have used the function azcall() for streaming the messages after every 1.5 seconds. It’s a simple implementation of the azure iot hub SDK which uses MQTT protocol internally. In the function pubData(), we are publishing the data to the Azure IoT hub which essentially is a Json string.

The code is based on the sample provided by the Microsoft team.

Before executing the code, we have one small step involved, where we can check out the incoming messages to the Azure IoT Hub.

There exists an utility by Microsoft known as Device explorer twin, which allows us to monitor the messages. It’s an Windows C# app and can be downloaded from the below Github link.

https://github.com/Azure/azure-iot-sdk-csharp/releases/download/2018-3-13/SetupDeviceExplorer.msi

Please install the above tool and in order to use we need the connection string of the IoT Hub not the Device. So head back to the azure portal, and click on Shared access policies.

Fig 6: Shared access policies.

Click on shared access policies to open the policies panel. Here we are interested in “iothubowner”. Click on it to open the policy details.

Fig 7: IoT Hub — Connection String primary key

Copy the “Connection string — primary key”.

Now, fire up your device explorer tool and paste the connection string in the IoT Hub Connection string input panel.

Fig 8: IoT Hub device utility

Click on update. and after successful update, click on Data.

Fig 9: Monitoring messages

Here, in this section, your device that you created from the portal before should appear on the Device ID dropdown. Select your device and click on monitor.

Now, head back to the folder, where your node.js code was written and execute the code using any IDE or in my case, I used Windows Powershell.

Fig 10: Powershell window

At the same time maximise the device explorer tool and then we will see the incoming messages as shown in Fig 11.

Fig 11: Incoming message stream to the IoT Hub

If everything is done correctly, the messages should appear here which means that messages are being received by the IoT Hub. Now that we can send messages to the IoT Hub, we can now write a serverless API for routing these messages to the Cosmos DB using Mongo DB API. Initially we’ll setup the cosmos DB such that we know where exactly we need to route the data.

Azure Cosmos DB

I’m not going to give you a detailed info about Cosmos DB. You can have a look here.

Create a cosmos db from the azure portal as a new reqource and select MongoDB under API dropdown.

Fig 12: Cosmos DB

After your deplyment is completed, next head on to the db you created. In our case for reference, the name of the db is “spektrodb”. This cosmos db is based on documents and it’s an unstructured db. Therfore, we can create new documents on the runtime. We just need to create a database for now under the name of “database_spektro”.

Fig 13: MongoDB database create.

For now, we’ll leave it in this way and just copy the connection string as it will be required in our Azure function. Head on to the db you just created and click on quickstart.

Fig 14: Cosmos DB

Keep the above data primarily the connection string under .NET section somewhere saved locally. You may also want to have a look at the code as it’s going to be used in the Azure functions.


Azure functions

It’s like a micro-service that gets triggered whenever there is any triggers or any requests. Here we are interested to trigger an azure function whenever there is an incomiing message in the Azure IoT Hub. But before that, lets create an azure function first. Head towards the azure portal and search for azure function. In this case the name of the function is “spektrofunc”.

Fig 15: Azure function

Click on new function

Fig 16: Azure funtion -1

Next, click on custom function and navigate to “IoT Hub (Event Hub)”.

Fig 17: Azure function -2

After that, a configuration window will open where we will configure our iot hub named “spektro”. You can name your function any thing. I named it as “spectroIotTrigger”. Under event hub connection, click on new.

Fig 18: Azure function -3

Here click on IoT hub and the name of the IoT hub should appear. In my case, it was spektro.

Fig 19: Azure function -4

Click on select and finally on create.

Now, your function is ready for you to write code in C#.

Fig 20: Azure function -5

Before writing the code, let’s find out how the already provided sample will work. In order to test the sample code provided, we need to click on test, and on logs which is a place where the messages or errors are displayed. Click on run and the test message should be displayed.

Fig 21: Azure function -6

We can also test with our device simulator in node.js. Just execute the code in the backend, and the messages will be showed in the log window.

Fig 22: Azure function -7

Now we can see the incoming messages in the azure function. Next task is to handle these incoming messages which will be used to publish data to cosmos db.


Cosmos DB integration in Azure function

Before jumping off to integrate cosmos, we must include the dependencies. Now, adding dependencies in azure functions is a bit tricky. Click on “View files” just above the “Test” button. Then we need to add one file named “Project.json”. In the file, we need to manually declare the dependencies. For cosmos db with mongo db API, we need MongoDB driver and for handling json, we need Newtonsoft.json. You can also declare the .net version.

Let’s decalre them using the following json.

Now, click on save and then browse to run.csx file.

Include all the necessary “using statements”.

Now save and run. If there isn’t any compilation error, then dependencies have been successfully installed. Now let’s write the code for decoding and pushing data. So our target is to create a new document named after the device ID and then the document will contain the required parameter ID and parameter values. In this case, its going to be simply,

{“Data”:”12"}

where Data is the parmeter ID and 12 is the value. The code is shown below.

The above code is the azure function written in C#. It initially decodes the JSON received in the string message from the azure IoT hub. A class was created to handle the mongoDB message push. We pass the parameters through parameterised constructor and by an async call, we push the data to the CosmosDB using MongoDB API. There are two options for testing the code. You can create any name for the DB, in this case it was “Database Spektro”.

Recall the “Test” area where there was some sample message.

Fig 23: Azure function screenshot for testing

In the test area, you can replace it with a JSON and click on run. This should execute the code with the sample input. Test for any errors.

Now, lets check this with the simulated device code, we have written earlier in node.js. Head on to the local folder and start the node.js code. Once it starts executing, go to your CosmosDB and click on “Data Explorer”.

Fig 24: Cosmos DB — Data explorer

Click on the database that was created and then on the table name which is basically the device ID. Our device ID was simulated device as in the node.js code and after we click on documents, the json files appear. Click on any one and you can check out the data.


Conclusion

Thus, we have seen how we can simulate an IoT device using node.js and send messages to the IoT hub. Finally, we route the messages into respective tables in CosmosDB based on the deviceID and stream some random data. This has a lot of usecase(s). Let’s take an example of telematics data from a car or any machine to be sent to the DB after a certain time interval or some kind of remote weather station or any IoT usecase where data is involved. Now, that you know the flow of data, why not comment some usecase(s) which perfectly fits in this flow. Would love to see new cool stuffs. The part that I didn’t cover is the use of rule engines which can be programmed in the azure function. That leaves the scope for another technical article which I’ll do in the upcoming days. If you have any issue with the above article or face any difficulties, drop a comment below. :)

Avirup Basu

Written by

A tech geek, a problem solver and a head full of crazy stuffs. Senior IoT Developer, Prescriber360 | Intel Software Innovator

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade