From IoT Device to a Full-Featured System

The variety and ease of development using Arduino, Raspberry Pi and other development boards, offers any developer to get an IoT device very quickly. Within a few days or weeks, you have a very nice device, connected and response to several sensors or motors, WiFi and Bluetooth, but that’s about it.

What about having this device to be part of a real system? a system that can interact with a backend server, having a management system, communicate with other web services?

If you got into developing your IoT device on the board using some C or Python, you are more than capable to develop and deploy some other services to make the system quite comprehensive. The following architectures and technologies will make it simple to understand and take it forward.

Just one note before we start. There are lots of solutions, architectures and technologies to bring a system to work as a basic product, and surely as a mature product. Per my perspective, I wish to review here some basic fundamentals that are easy to understand and deploy, and can take a system quite quickly and easily into a working product, then migrate it using the same technologies, with some extra care and design, to a scalable system that can support many users to come.

Architecture Overview

High level IoT system architecture

The architecture above describes the main components flow considering the following:

  • Development board, sensors etc.
  • Persistent back-end and database.
  • Management user interface to manage users, admins and other system objects.
  • User Interface for non-managers.

Other aspects to consider:

  • Security between all system components.
  • Communicating with additional web services.

Development board & Sensors

I will not get into the different boards and how to choose the best board for a specific project, as this subject is big enough to be discussed separately, but since we discuss a full system capabilities, like communicating with servers, providing UI and probably do some more work, we are talking about stronger boards, like Raspberry Pi. I do believe that Arduino will do a nice job when talking about some simpler, distinctive tasks.

Back End & APIs

The board provides an infrastructure for handling the sensors and protocols that solely related to the IoT device functionality, but what if we need to maintain users database, keep persistent data or do some work that need more computing power?

A back-end server (that usually hosted in one of the major cloud providers) can provide a good and stable platform for these tasks. The backend will expose API calls (usually in REST format) that will be exposed on the web and be accessible by our IoT device.

An example of such request can be:

https://www.mydomain.com/users/1

This url is actually a REST request for “retrieve user # 1 information and post it to me”. The backend server will retrieve the information from a persistent database (e.g. MySQL, PostgreSQL, MongoDB etc.), and send the response in a common structure that the board will expect to get (usually JSON).

Backend solution that will provide API requests and responses involve some code development and cloud deployment. There are many technologies for developing backend software like Flask or Django (Python based) Ruby on Rails or Sinatra (Ruby based) Node.js (Javascript based) and more. For most development languages like Python, Javascript, Ruby etc. there are heavier frameworks that provide much more features (e.g. Django and Rails), and there are lighter frameworks that provide specific features, and may be easier to use for API and some UI only (e.g. Flask and Sinatra).

I found Python Flask as a very nice, light and easy framework to work with. Although Django (for python) and Rails (for Ruby) enables a much larger framework, I see these frameworks as much heavier to deploy and maintain, especially when we need lightweight API and some UI capabilities. There is one more advantage for using Python as the backend language, which is the fact that one can develop Raspberry Pi using python as well, and you keep using one development language for both.

As for database persistency, any database mentioned above will satisfy small system, and will enable growth and high-availability capabilities as the system expands. Most of them are open source, and any backend technology can integrate with the database.

As these APIs are now deployed and probably exposed to anyone, a proper security should be applied to the API, making sure that the api calls are authenticated, means that only the clients and boards that you allow to access the api (using api key or similar) are authenticated as valid api users

User Interface

So we have a backend server that communicates with the persistent data in the database, but what about managing the data? adding users? check for collected data? follow user activities?

Some UI (User Interface) should be added to enable this functionality.

All the frameworks mentioned above provide UI development capabilities (aka front-end), and with some code you can develop very nice web interface that communicates with the backend and provide all these nifty capabilities.

Bear in mind that some other aspects should be considered when developing the user interface, like who can access it (user management), what data should be exposed etc. , so before jumping into development, make sure you spend some time designing the data structure, roles and data flow between the database, backend and the different users of the UI. Remember — the UI is usually accessible from the web, and you are responsible for keep it safe.

Except the native UI capabilities of the frameworks, there are dedicated front-end frameworks like Angular.JS and React.JS , which are very popular among UI front-end developers. If one should decide to use these technologies, its very easy to integrate! as we are using an api-based backend as discussed earlier. A bunch of additional api REST calls provided by the backend will be used in front of the front-end application. Just keep in mind to have appropriate authentication, even separate access control to distinguish the board and device access from the UI access.

Security

When your APIs exposed on the web, there should be a strict control who can access these APIs, get and set information. A quick search for RESTful API authentication for your chosen backend technology will return some appropriate methods to secure your API. Basic authentication should do the work, as long as you keep in mind that with every API call, you send a hash key that will be identified by the backend, and authenticate the call.

Additional Aspects

Once the system is ready, there are other aspects that should be taken into consideration, most of them relates to the operational side of the system like deployment, logging, monitoring etc.

Every cloud provider (where you host your backend) will provide various solutions for logging and monitoring that are quite easy to implement.

If your backend application is packed and containerized using Docker, you will find serverless services as well, that allows even easier deployment, scalability and operation.

Make sure to give some thought once deciding which direction you choose, as easy, scalable or cost wise as it looks, depending on your planned growth and resources.

Summary

As you can see, with some basic architecture and some coding, you can create a quite nice functional system around your IoT board. Each component can be further replaced if new technology will be chosen in an further step of the project, as long as there is a solid understanding of each component requirements, functionality and the appropriate technology to satisfy all of those.