How we built the iotSky backend using AWS Lambda

In this post I will try and describe how we built the backend for iotsky almost entirely on AWS Lambda. But first, the mandatory shameless plug :)

What is iotSky?

iotSky is a platform aimed towards teaching students / hobbyists about the Internet of Things (IoT). It is a PaaS (Platform as a Service) which provides an abstraction for users so that they can easily connect their devices to the cloud and have them communicate / interact with each other. Bring your own devices and the software to to make it come alive while iotSky provides the underlying infrastructure for you to do this easily and securely.

The Backend

The iotsky backend is built almost entirely on AWS lambda. The primary reasons for this were:

1. Not having to do server maintenance / monitoring was a big win
2. Scales as per our need
3. Easily integrates into the other AWS services
4. Very cheap, at least to start off.

The basic architecture looks something like this:

iotSky Architecture

There is a VPC with public subnets and private subnets. Something like this. All the lambda functions are housed in the private subnets. It can connect to data stores like ElastiCache and RDS also within the private subnets.

Each of the lambda functions (written in python) perform a specific task. They are triggered via SNS messages. i.e. A SNS message activates lambda1. lambda1 does some processing based on the info contained in this message (e.g. create a iot ‘thing’, update a row in the db etc). It then publishes a new SNS message with a new payload. lambda2 is a downstream function which gets triggered as it is subscribed to this second SNS topic and proceeds to do it’s own thing.

In this way, each lambda function performs a specific task and then publishes a SNS topic for any downstream lambda function to act if necessary. The advantages of this are that each lambda function can be specific to it’s task (which makes unit testing it easier) and lambdas can thus be chained in this way to achieve tasks of arbitrary complexity.

Example

The following is an example of a chain of lambdas that creates an iot ‘things’. In iotsky, you can create a project and give your things names. Under the hood we use the aws api to actually create the ‘thing’, policy and attach a cert to it. We then provide you with a cert, public / private keys and a subject on which only you can publish / subscribe to. In under 30 seconds you are up and running ! Here is the schematic:

CreateThing Use Case

Here is sample lambda code that creates the thing:

And here is sample code that updates the db:

And that’s pretty much it. Hopefully this post has shown how you how to get started with lambda functions.

Suggestions for the AWS Team

1. Deploying lambda functions should be easier.
To solve this challenge we use apex.run. It automates the task of building the zip file, deploying to was etc
2. Integration testing of the chained lambdas are a challenge unless you simulate SNS somehow.
The way I got around this was to essentially feed the lambdas on localhost the same message that SNS would send it. So something like:

echo -n ‘{“Records”: [{“EventVersion”: “1.0”, “EventSource”: “aws:sns”, “EventSubscriptionArn”: “arn:aws:sns:EXAMPLE”, “Sns”: {“MessageId”: “94df01b5-ee98–5db9–9903–4c211d41eb6e”, “Signature”: “EXAMPLE”, “Type”: “Notification”, “TopicArn”: “arn:aws:sns:EXAMPLE”, “MessageAttributes”: {“Test”: {“Type”: “String”, “Value”: “TestString”}, “TestBinary”: {“Type”: “Binary”, “Value”: “TestBinary”}}, “SignatureVersion”: “1”, “Timestamp”: “1970–01–01T00:00:00.000Z”, “SigningCertUrl”: “EXAMPLE”, “Message”: “{…}”, “UnsubscribeUrl”: “EXAMPLE”, “Subject”: “TestInvoke”}}]}’ | apex invoke update_db — env prod — profile prod

NOTE: iotSky is still fairly new so not much to report about in terms of load / scale / performance characteristics of lambdas etc. Once we have more data in that regard, i’ll write a follow up post to this with our experiences of it.

Follow us on twitter @iotsky