Efficient IoT with Confluent Kafka on Hivecell
EdgeIR reports that enterprise companies have a growing interest in IoT tech. IoT implementation allows fast, precise, and “smart” customer service, but at scale, it results in a flood of data from tracking devices, sensors, cameras, and appliances of all sorts. In order to process and secure this detailed data, companies would normally go to cloud storage — and pay the cost of moving, storing, and processing the data.
There must be a better way to manage business-relevant data within a cost-effective framework.
Pavlo Lobachov, software engineer at Hivecell, put together a demo of an edge computing infrastructure solution showing how IoT and cloud can be optimized using Hivecell and Confluent Kafka.
Here is a simple demonstration using a Raspberry Pi equipped with a temperature sensor, connected to a speaker and a single Hivecell. Hivecell runs Kafka MQTT proxy, acting as a data collection service. Confluent Kafka, which subscribes to the MQTT server, pulls the data and stores it locally on Hivecell.
The sensor is set to report it’s temperature once a second to the MQTT server, 3600 inputs per hour, resulting in 84 Mb of collected data locally stored on the Hivecell.
Our case study end-user is only interested in being notified when the temperature changes beyond set value. When there is a qualifying delta of temperature value, 0.01°C or more for our demo purposes, Hivecell commands the Raspberry Pi to send an audio alert to the speaker and pushes the result together with the registered time and temperature to Confluent cloud. The best part is that the business-relevant data, temperature changes, requires only 9Kb of cloud storage.
84 Mb of gathered data → processing at Hivecell → 9Kb of uploaded data.
To recap, the painless set-up of IoT tech + edge computing server + cloud in this demo shows:
- Fault-tolerant local storage
- Data processing in real-time
- Reducing the bandwidth and required cloud storage capacity 9.3 times
For real-world applications, this model is even more promising. It allows replications of large data partitions across multiple servers. Sending pre-processed data means that only the relevant data is moved to the cloud. Industries that could benefit from this model range from farming and manufacturing to retail services, oil and gas safety systems, and healthcare.
Hivecell makes this solution work on a large scale.
Hivecell is push-button easy to provision and connect to local IoT, with SaaS runtimes like Confluent Kafka managed fully remotely.
Locally deployed; remotely managed — Hivecell makes managing your edge infrastructure easy.
#edgecomputing #IoTedge #industrialIoT #kafka #hivecell
