Connecting an IoT device to Hut34 - the hard way

Last week I showed you how I connected a phone to the Hut using an existing service. I also connected a Raspberry Pi with temperature sensor to Hut34. I wanted to a more flexible result, so I chose a different method … it turned out to be a little more involved.

Hut34 has built a distributed network where IoT data can be exchanged and monetized. My goal is to show you how to connect. Read more about our project here.

After sketching the infrastructure out I realised that the process involved a few sub-tasks:

1. Send data from Pi to the cloud;
2. Store data in the cloud;
3. Send to Hut 34 when requested.

To do this I stitched a few services together. Note: Part I is from Calvin’s excellent tutorial on streaming Pi data to AWS IoT.

The data flow is as follows for Part 1.

1. A python script on the Raspberry Pi publishes data (timestamp, temp) to AWS IoT using the MQTT protocol;
2. An IoT rule on the platform forwards the data to the Elastic Search instance (ES);
3. Kibana is used to visualise the data from ES.

Elastic search isn’t the best too for this job, but I stayed true to the tutorial.Talk to us on Telegram

When Hut 34 makes a request to the endpoint supplied by API Gateway, the following occurs:

1. A javascriptLambda function is triggered by API Gateway;
2. The Lambda function retrieves the last known state of our temperature sensor (ie. the latest record from ES);
3. The Lambda function sends the data to Hut 34.

While the solution was more complex than I would like, it illustrates what can be done by connecting the existing services out there. This solution has a lot of flexibility through the whole stack. I could integrate more sensors quite easily, change the timing of data updates, or use MQTT to only return data when queried. The other advantage is that the whole stack will scale to support multiple devices.

The API Gateway / Lambda pairing allows for logic to be built into the API I use to serve Hut 34, and to respond to different requests in a manner of my choosing. Consider for example if I had groups of air quality sensors across the city of Sydney. A custom API would allow me to retrieve data from a single location, or just Carbon Monoxide data from all locations.

This is overkill for a single Raspberry Pi — but a lot more powerful as a custom solution and would allow me to expand my service to serve multiple data streams rather than just single data point. This is something that is interesting and is being explored by a few other services out there.

The only thing I would do differently next time is to use a Dynamo DB in place of ES. I think it would be better suited for the job and would still allow persistent data for historical trends.

Please follow Hut 34 to read about more things I’ve built and comment below with ideas of what you’d like to see next.

Follow us on twitter, talk to us on Telegram.

Note: AWS is not related to the Hut34 Project in any way. I just happened to use a lot of their services.