Implementing a secure & scalable IoT solution — AWS IoT Core
I’ve been experimenting with various AWS IoT services for many months and I recently implemented it for one of our customers, looking for a fully managed IoT solution. So our client manufactures and distributes IoT based water purifiers that converts contaminated tap water into pure drinking water.
Initially their devices were publishing all their sensor data to ThingsBoard which is an opensource IoT platform (they configured it on an EC2 instance). During the initial phase it was a suitable solution but as their devices scaled in number, it was unable to fulfill their operational requirements hence creating a need for a fault tolerant, scalable and managed IoT platform. Here is a cropped image of their hardware module from first production batch, which looked pretty cool to me (sorry I can’t upload the full circuit board with components due to some privacy concerns).
We’re going to refrain from talking much about the hardware module as we didn't play any role in it’s testing and production but with whatever is apparent from the picture above, it consists of a GSM and WiFi module for data transfer while PIC24F Micro-controller was the brain behind board. Besides that there were various sensors used to measure different device attributes like TDS, Water Pressure and Filter Health etc.
So after evaluating various pros and cons of different public cloud IoT platforms (perhaps a topic for another day), our technology team finally decided to use AWS IoT Core.
In this Article, I would like to focus more on how we implemented AWS IoT core with their devices along with other backend services that were used in the process.
Architecture
Let’s get a 10,000 foot over view of the overall architecture before I get into the details of how each resource was provisioned and used in the production environment
As we can see, the on field devices connect with IoT core after establishing TLS connection and communicates over MQTT protocol. There is one IoT Rule created in the rules engine which selects all payload data on a previously defined reserved topic.
The IoT rule further triggers five simultaneous actions. The first two actions are pretty simple, they insert all payload data into dynamodb tables. The third action forwards all data into a Lambda function which performs some custom calculations and stores the net results into a separate dynamodb table.
Another action was used to republish device acknowledgement on a subscription topic whenever a payload was received. Finally a separate Lambda function was provisioned to simply forward the telemetry data on a virtual machine which we were using to manage customer centric data.
IoT Core, Rules Engine Challenge!
As mentioned above, I used a Lambda function to perform some calculations and store data into dynamodb table. Initially I assumed that the response of the lambda function would be sent back to the invoker (i.e: device). However that was not the case. Neither did any of the Rules Engine actions enabled me to achieve this automatically. So I had to republish the response data back to the device on individual subscribed topics using aws-sdk.
To further clarify the above challenge, let’s assume that a simple Lambda function which makes an api call to dynamodb, gets triggered by an IoT core rules engine. The response of this function is returned and stored in a data attribute as shown below.
The response data will not be sent automatically to the client side which invoked this Lambda function through the Rules Engine and will have to be further republished on the subscribed topic using aws-sdk for iot core as shown in the code below.
Hence a similar custom solution was adopted to fulfill our client’s custom requirements. The solution can be summarized into the following steps of construction:
- IoT Device publishes a payload on an IoT Rules engine topic which triggers a Lambda function.
- The Lambda function gets data from dynamodb and republishes the response back to subscribed topic using aws-sdk.
Caution: Please ensure that the lambda function is not republishing on the IoT Rule origin topic else it will create an infinite loop. The best practice would be to republish on device specific topics.
Again this is based on my personal knowledge and understanding revolving around the said technology at the time of implementation. As always, I am open to suggestions for any possible alternatives.
Firmware Management & Device API Gateway
An S3 bucket was used to maintain our device firmware versions and also the bucket endpoints were configured for OTA firmware updates. An Api Gateway was also created and hooked up with relevant lambda functions that would pull device data from dynamodb and make it available for usage at an api level. The api documentation was further published on Swagger for reference.
Quick Implementation
Okay so now that we’ve had a detailed discussion on the over all architecture, core services and operational flow. Lets quickly look into how we can create an IoT Thing and initiate testing so that we can start receiving some data.
Provision an IoT Thing and attach policy to SSL certificates
The first step is to create an IoT Thing which is hardly a two step process. Simply log in to aws console and click IoT core. Then in the left panel click Manage and select Things. Go to the top right corner and select ‘Create’ as shown below.
Create a single thing and give it a name and type. Generate security credentials and download certificate and keys, you will be needing these later in the connection string.
Activate the certificate by attaching an IAM policy which would give it relevant access to IoT core.
Testing AWS IoT Core Broker with Mqtt.fx
Download Mqtt.fx by clicking here after selecting the OS of your choice (Linux, MAC, Windows). Once the download is complete, launch the app and click the connection tab to initiate connection profile.
Inside the connection profile, insert Broker Address, mqtt port and a random clientID. Select SSL/TLS tab and insert relevant file paths to previously downloaded certificates. Don’t forget to mark checkbox for PEM formatted.
Finally go back to the main screen and press Connect button. Once the red led turns green, the connection between Mqtt.fx and AWS IoT core is established.
Now lets publish some data on our “helloTest” Topic and see how it looks like at the receiving end. Go back to console and click Test panel under IoT Core. Paste “helloTest” under topic name and click on “subscribe to topic” (no need to change quality of service since QoS was also zero in mqtt.fx). IoT Core would begin to listen on the respective topic.
Go back to Mqtt.fx and click on Publish message. Now IoT core live terminal would display newly received payloads that were published by the client application.
So with this we’ve successfully received data on IoT Core from Mqtt.fx. This will hopefully make things much smoother once you start with your hardware integration.
Recap
Now just a little recap, we had a detailed analysis of a practical IoT core implementation and its various use cases. We also looked into a Rule’s Engine Challenge that was faced during development and how it was addressed. Finally we went through a quick hands on lab in which we created an IoT Thing and started publishing data from an Mqtt client application.
Next Steps?
So now the next steps from here would be to look into further details of IoT Core particularly the Rules Engine, Thing Management and it’s security. There are also other IoT Services like FreeRTOS, IoT Analytics and IoT Green Grass that I would highly recommend to look into when building IoT solutions on AWS platform. Please feel free to ask any questions or give your feedback in the comments. Take care!
