Rethinking IoT Architecture — the need for distributed systems architecture for the Internet of Things

The Internet of Things (IoT) refers to connected device and sensor technology enabled by secure network connectivity and cloud infrastructure, to reliably transform data into useful information for people, businesses, and institutions. At its core, the Internet of things means just an environment that gathers information from multiple devices (computers, vehicles, smartphones, traffic lights, and almost anything with a sensor) and applications (anything from a social media app like Twitter to an e-commerce platform, from a manufacturing system to a traffic control system. Yet data alone Is useless, it needs to be interpreted and turned into information and this information is often required to be exchanged with other devices or applications to generate business value. It is therefore necessary to generate trust in this huge amount of data being generated and exchanged between systems. Blockchain and its ability for Building trust in distributed environments without the need for authorities is a technological advance that has the potential to change many industries, the IoT among them.

The powerful capabilities of smart, connected products not only reshape competition within an industry, but they can expand the very definition of the industry itself. The competitive boundaries of an industry widen to encompass a set of related products that together meet a broader underlying need. The function of one product is optimized with other related products. For example, integrating smart, connected farm equipment — such as tractors, tillers, and planters — can enable better overall equipment performance. The basis of competition thus shifts from the functionality of a discrete product to the performance of the broader product system, in which the firm is just one actor. The manufacturer can now offer a package of connected equipment and related services that optimize overall results.

That said there are many challenges to adoption of IoT at scale. These are mainly because of the design of today’s internet and the intermittent connectivity of today’s networks. There are several aspects that apply to IoT systems that affect their architecture and implementation, as follows:

1. Scalability: Scale for IOT system applies in terms of the numbers of sensors and actuators connected to the system, in terms of the networks which connect them together, in terms of the amount of data associated with the system and its speed of movement and also in terms of the amount of processing power required.

2. Big Data: Many more advanced IoT systems depend on the analysis of vast quantities of data. There is a need, for example, to extract patterns from historical data that can be used to drive decisions about future actions. The extraction of useful information from complex data such as video is another example of analysis requiring large amounts of processing. The ability to mine existing data for new insights and the need to combine different datasets in novel ways are characteristics likely to be part of an IoT system. IoT systems are thus often classic examples of “Big Data” processing.

3. Cloud computing: IoT systems frequently involve the use of cloud computing platforms. Cloud computing platforms offer the potential to use large amounts of resources, both in terms of the storage of data and also in the ability to bring flexible and scalable processing resources to the analysis of data. IoT systems are likely to require the use of a variety of processing software — and the adaptability of cloud services is likely to be required in order to deal with new requirements, firmware or system updates and offer new capabilities over time.

4. Real time: IoT systems often function in real time; data flows in continually about events in progress and there can be a need to produce timely responses to that stream of events. This may involve stream processing; acting on the event data as it arrives, comparing it against previous events and also against static data in order to react in the most appropriate way. There is a parallel need to ensure that corrupted data is detected and not used — whether introduced by faulty sensors or malicious action — since the use of corrupted data could cause harm and damage to humans, equipment, and the environment.

5. Highly distributed: IoT systems can span whole buildings, span whole cities, and even span the globe. Wide distribution can also apply to data — which can be stored at the edge of the network or stored centrally. Distribution can also apply to processing — some processing takes place centrally (in cloud services), but processing can take place at the edge of the network, either in the IoT gateways or even within (more capable types of) sensors and actuators. Today there are officially more mobile devices than people in the world. Mobile devices and networks are one of the best known IoT devices and networks.

6. Heterogeneous systems: IoT systems are often built using a very heterogeneous set of. This applies to the sensors and actuators, but also applies to the types of networks involved and the variety of processing components. It is common for sensors to be low-power devices, and it is often the case that these devices use specialized local networks to communicate. To enable internet scale access to devices of this kind, an IoT gateway is used.

7. Security and Privacy: The question of the security and trustworthiness of distributed heterogeneous IoT systems is a hard problem whose solutions must scale and evolve with the systems. Data protection is necessary, including significant privacy concerns regarding data that relate to individuals. Gaining assurance that these systems are safe, secure, resilient and uphold their stakeholders’ expectations about privacy is especially challenging.

8. Compliance: Providing confidence about the operation of these IoT systems is necessary both due to the regulations of specific industries, sectors and verticals and also the norms and expectations of the stakeholders of the IoT systems.

9. Integration: IoT systems do not exist on their own but need to connect to existing operational technology systems like factory systems, building control systems, and other types of physical management systems as well as existing enterprise systems including enterprise applications and enterprise databases.

There are mainly 3 types of system architectures utilized by modern software applications which in turn determine the architecture of an application.

Centralized systems: Centralized systems are currently the most widespread model for software applications. Centralized systems directly control the operation of the individual units and flow of information from a single center. All individuals are directly dependent on the central power to send and receive information and to be commanded. The central network owner is a single point of contact for information sharing. The biggest issue with a centralized network is with a single central owner it also becomes a single point of failure. Further, with a single copy stored with the owner, every instance of access to the resource leads to an access issue with time.

Decentralized Systems: In a decentralized, non-distributed (or co-located) system, all the parts of the system are in the same physical location. With multiple central owners that have the copy of the resources. This eliminates the biggest problem of single point of failure with centralized network. With multiple owners, if a particular central node fails, the information can still be accessed from the other nodes. Further, with multiple owners the speed of access to the information is also increased. Decentralized systems form the basis of parallel and cluster computing where different processors on the network can execute the same instruction set on different parts of the same dataset (shards) or different dataset.

A decentralized system can thus be characterized as a collection of mostly autonomous processors (nodes) communicating over a communication network and having the following features:

• Individual goals: the nodes are working to achieve individual goals which may need to be coordinated to get the end result (eg map reduce)
• Loosely coupled: the nodes on the network should be loosely coupled
• Direct synchronization with other nodes on the network: the nodes on the network should be synchronized via a common clock
• Shared memory: the nodes on the network should use a shared memory space to store intermediate state and data
• Geographical distribution: the nodes on the network should be collocated, with minimum geographical distribution for data redundancy if required
• Homogeneity: the nodes on the network should ideally be autonomous and heterogeneous in nature

Distributed Systems: In a distributed network the nodes are not collocated but distributed geographically also It avoids the centralization completely. The main idea for the distributed network lies in the concept that everyone gets access, and everyone gets equal access. A node can be any active electronic device, including a computer, phone or even a printer, as long as it is connected to the internet and as such has an IP address. Nodes are often arranged in the structure of trees, known as binary trees. processing these transactions can require large amounts of computing and processing power, meaning that the average computer’s capabilities are inadequate. A node can either be a communication endpoint or a point of communication redistribution, linking to other nodes. Every node on the network is considered equal, however certain nodes have different roles in the manner in which they support the network.

A distributed system can thus be characterized as a collection of mostly autonomous processors (nodes) communicating over a communication network and having the following features:

• Common goal: the nodes are working to achieve a common goal that cannot be achieved using a single processor
• Loosely coupled: the nodes on the network should be loosely coupled and should be able to communicate via a messaging system using a standard protocol
• No direct synchronization with other nodes on the network: the nodes on the network should not be synchronized via a common clock
• Distributed memory: the nodes on the network should not have a shared memory space if possible, but shared distributed memory space for the entire system can be provided as an abstraction
• Geographical distribution: the nodes on the network should be geographically distributed and should avoid any centralization
• Autonomy and heterogeneity: the nodes on the network should be autonomous and heterogeneous in nature

Most current IoT architectures use cloud computing which is centralized. The formulation of a distributed approach has the attractive property that software can be run in parallel among the nodes and near to where the computing is actually needed thus reducing the transfer of data between the individual nodes. This approach expands the boundaries of “the cloud” beyond data centers to all computing resources from large data centers all the way to consumer devices on the edge of the network. This approach to computing in effect creates a cloud fabric that is potentially millions of times larger than the existing central cloud infrastructure. The result is an evolving continuum of swarm of services using computing resources as needed from edge nodes in clusters all the way up to large server nodes in data centers and in the process provides the security, privacy, integration and compliance required by IoT architectures.

That said distributed computing is not easy, like other areas in computer science, distributed computing spans a wide range of subjects from the applied to the very theoretical. On the theory side, distributed computing is a rich source of mathematically interesting problems in which an algorithm is pitted against an adversary representing the unpredictable elements of the system. Analysis of distributed algorithms often has a strong game-theoretic flavor, because executions involve a complex interaction between the algorithm’s behavior and the system’s responses. The simplest model of distributed computing is a synchronous system where all nodes operate in a lockstep fashion. This model is commonly known as the LOCAL model.

During each communication round, all nodes in parallel

(1) receive the latest messages from their neighbors

(2) perform arbitrary local computation, and

(3) electing a leader then

(4) sending new messages to their neighbors.

In such systems, a central complexity measure is the number of communication rounds required to complete the task. This execution of state machines in a distributed system such that each individual node executes an arbitrary local computation then communicates the computation to other nodes to arrange the results in the in the required order for the next round of computations formulates the practical problem of distributed computing as such a system cannot inherently be scalable or real time nor does it have the ability to provide real time analytics of data streams that may be required to trigger subsequent computations. There is also the need for historical data analysis that can be used to drive future decisions using artificial intelligence and this decision making needs to happen as close to the computing node as possible in order to reduce latency.

Modern IoT architectures therefore required a combination of decentralized and distributed systems where clusters of parallel computing resources deployed at the edge communicate with a mesh of heterogeneous devices deployed as network nodes to achieve distributed computation and storage over a wide area network. This distributed computation and storage can be enabled using technologies like distributed ledgers and smart contracts as defined by a blockchain.