Design a reliable IoT architecture
When designing IoT architectures, it’s crucial to thoroughly explore various factors and think more about them before implementing the system. This comprehensive approach helps us identify and implement the most effective solutions.
IoT involves handling massive amounts of data quickly and securely. Building a system to receive, process, and respond to this amount of information is challenging. There are many ways to approach this, but I want to share our approach at Fasterbase our latest startup project.
Connectivity
This is a critical component of our system. Devices send us messages, and our service must receive, process, distribute, and store them as quickly as possible.
MQTT
We chose RabbitMQ to handle incoming messages, but we needed a way for all type of devices to send data to our servers. Since our customers use different hardware, we decided on MQTT to support many IoT devices with different structures and OS.
Mosquitto is known as a lightweight, fast, and secure MQTT broker. However, we decided to use HiveMQ to handle this part of our system so we could focus on developing our core features.
We carefully configured Mosquitto with a QoS level of 2 to prioritize reliability over speed.
To strengthen security, we enabled TLS and added JWT token verification within the payload to identify devices for each message.
Choosing effective topic names can be challenging. The best approach depends on your specific needs. In our system, with multiple clients, customers, and devices, we use the /client_id/device_id/customer_id/event_id pattern to route MQTT messages efficiently.
RabbitMQ
While MQTT handles device-to-server communication, we needed a way to distribute messages between our internal services (processor, log, data provider) and client instances. To improve MQTT performance and distribute the load, we integrated RabbitMQ as a message broker.
Incoming MQTT messages are directly pushed into RabbitMQ, allowing for efficient message routing and processing.
Data Storage
We use a multi-storage approach using three different services. Our tenant-based strategy assigns each client an independent database, so each client's information is in a separate database with different types of databases.
Cold Storage
We set up Cassandra for direct log storage. Due to Cassandra’s query limitations, we index essential data for filtering while storing the remaining information in a non-indexed column. Cold storage is read-only, serving as an archive for historical data.
Warm Storage
Warm storage holds recently accessed data for rapid retrieval. For instance, consider a temperature sensor that triggers AC activation when the last 5 minutes’ average exceeds 25 degrees.
We store the past 5 minutes of temperature data in a time-series MongoDB database (our warm storage) for quick access. To further optimize performance, we cache calculated data in Redis.
Devices Configuration
We employ a tenant-based storage strategy, providing each client with an isolated database. For other data, we use MongoDB due to its flexibility in handling diverse device schemas defined by our customers.
Conclusion
There are numerous ways to design IoT architecture. While cloud giants like Google, Amazon, and Microsoft offer comprehensive IoT platforms, there are also low-code solutions like our Fasterbase that accelerate backend development. Ultimately, the best approach depends on specific project needs.
