The Hitchhikers Guide to AWS IoT
A detailed introduction to AWS’ Internet of Things offerings.
There are three certainties in life: death, taxes and the impenetrability of AWS documentation.
For the budding IoT developer, life can seem even more confusing because, at a glance, there are eight independent products you seemingly need to master before you can get started.
As a developer adjacent to AWS’ IoT offering, it’s important for me to nurture a deep understanding of the products and how they work together. So every now and again I throw together a home-brew IoT project to brush the dust off and achieve the impossible of keeping up with all the latest features!
And this time, dear reader, I’m going to take you along for the ride.
Before we kick off the series, though, I’m going to take a wander through AWS’ Garden of IoT. Or IIoT. Or IoS. Or Connected Smart Devices. Or whatever the cool kids are calling it these days- there’s probably a “blockchain” in there somewhere…
So here are the offerings as listed in the AWS Management Console:
AWS IoT Core
Unsurprisingly the bulk of AWS’ IoT functionality is provided by Core. There’s a lot going on here, and it can seem a bit disjointed. When it comes down to it, however, AWS offer the following (in an opinionated order of immediate usefulness!):
- An MQTT (seemingly the winning IoT messaging protocol; flamewars pending,) broker to allow two-way device-device and device-cloud communication.
- Certificate provisioning and authentication for your devices onto the service. (n.b. AWS IoT also supports Cognito and IAM; but these are typically for use with mobile/web platforms working with the estate, rather that the connected device itself.)
- A proxy mechanism (termed the “Device Shadow”) that allows you to set, store and synchronise a state (JSON) object between each of your devices and their cloud representation.
- An automation system that allows you to forward SQL filtered telemetry from the MQTT broker to a handful of downstream AWS services. This includes saving to S3/Dynamo and invoking lambdas.
- A searchable attribute/group based taxonomy for your devices (distributed across “Thing Types” and “Thing Groups”). This can be used in tandem with provisioning to allow devices to pick-up/declare attribute values on connection.
- A job system (think “firmware upgrade” rather than Mages, Thieves and Knights) that allows you to define a process and then run and monitor its progress across a definable subset of devices in the estate.
IoT Core also encompasses a series of special ‘on device’ SDKs that allow your devices to easily interact with the IoT specific APIs (most notably the MQTT broker and the Device Shadow).
AWS IoT Device Management
Remember what I said about AWS Documentation? Despite how things might look like from the outside, IoT Device Management doesn’t actually exist. It is the name given to describe several aspects of the IoT Core offering (namely groups, jobs and provisioning).
All documentation and console links will lead you to AWS IoT Core.
AWS IoT Analytics
There’s little point in connecting all these things if we’re not going to attempt to do something intelligent with the data. Analytics provides some mechanisms for doing this in a way that feels something like a cross between the Rules functionality from IoT Core and Kinesis’ (AWS’ stream offering) own Data Analytics.
The feature allows you to create pipelines that listen to data from a set of the MQTT Broker topics (there’s a reason I said the broker was the most useful part of the set), manipulate it using a lambda, transform it using something similar to XLST for JSON (including enrichment from the device shadow and type/group) and then dump it in a data lake; which you can then query and visualise using a subset of SQL.
AWS Greengrass
A not uncommon pattern in IoT (especially with power/data constrained devices) is to have a collection of relatively dumb nodes that connect to a local ‘edge’ device. The edge device works as a ‘local cloud’ to the devices; providing a nearby hub where data can be processed, messages brokered and a gateway to the (actual) cloud.
Greengrass is AWS’ solution to this problem. In someways it’s like having your own mini-AWS in a box. Although in most, it isn’t. Greengrass provides free (closed source) software (unfortunately named “Core”), which you can install onto a linux-class device. That allows you to (in a once again opinionated order of immediate usefulness!):
- Create an MQTT broker and device discovery service to serve your local network of nodes. This uses the same certificate-based security as Core, except that it also requires you (for better or for worse) to specifically allow communication between devices/the cloud/topics on an route-by-route basis.
- Run ‘lambda’ functions on your edge device. I’m using quotes here because these ‘lambda’ functions are somewhat different to the cloud offering. While you can still use them to respond to MQTT messages, you can also use them as ‘long-lived’ applications that start with the device. There are some additional limitations (they must be written using a special SDK and only Node 6, Python 2.7 and Java are supported), and should target mega- rather than giga-bytes of RAM. However there’s some additional functionality too- they support binary events (for those devices that feel JSON is too verbose), and can access the Edge device’s hardware/file-system.
- Have a local version of the device shadow (state objects) that can be remotely synchronised back to the cloud.
- Handle deployment updates (configurations and new lambdas), as well as OTA updates for the underlying Greengrass ‘Core’ platform.
- Run trained machine learning models; although frankly this is just an extension of the ability to run local lambdas that have access to the underlying hardware.
Amazon FreeRTOS
And to finish with the easiest. This is an operating system built and maintained by Amazon for use within the AWS IoT ecosystem. FreeRTOS seems to be as close as anyone will get to a defacto operating system for microcontrollers. Amazon’s FreeRTOS offering adds two things to this open kernel- a set of maintained modular open-source libraries that allow interaction with the wider AWS IoT ecosystem (including OTA update support), coupled with the ‘certification’ of devices as being supported by AWS.
If your device is a Raspberry Pi (or other linux-class) device, then FreeRTOS won’t hold much interest for you. If your solution is power-constrained, however, chances are you’re looking at microcontrollers (I had the difference explained to me as “if the description of what it does includes ‘and’ more than twice, you need a Pi!”). At which point you might consider FreeRTOS to save you a lot of the boilerplate; just know that only 4 of the 40 platforms supported by FreeRTOS are AWS Qualified.
And that’s it! (If you’re wondering why there aren’t 8 things, it’s because the rest are made of one-click buttons…). For the rest of the series I’ll be taking a much deeper dive into each of these offerings, so watch this space!
