The AnyLog gRPC service

A unified insight to distributed data without centralizing the data

Written by: Moshe Shadmon & Mark Davidson

gRPC is a high-performance Remote Procedure Call (RPC) framework that can run in any environment. It enables client and server applications to communicate using read-write message streams.

gRPC is designed for low latency and high throughput, making it suitable for distributed systems where performance and scalability are critical. For example, in IoT and edge ecosystems, devices and edge services often need to exchange data efficiently with backend systems, centralized systems and the cloud.

AnyLog is a software agent deployed on edge nodes, making each edge node a member of a P2P, decentralized and secure network that is optimized to manage IoT data and resources at the edge. Using virtualization, distributed edge data is serviced as if it is centralized. Users and

applications issue data queries to the “edge”, and these queries are satisfied as if the data is

centralized, whereas the physical data remains in place. The same platform allows to treat all edge resources as a single machine. Users interact with a virtual and single representation of the distributed edge (Single System Image — SSI).

The Value Proposition

Leveraging AnyLog as an added layer to a distributed system provides powerful functionalities:

• Data remains in place — no need for massive data transfers to centralized locations.
• Applications (at the edge or the cloud) interact with a virtual and unified view of the data (as if the data is centralized).
• Near real-time insight to the data.
• Data is managed at the edge using existing services (provided by the AnyLog instances), with a high degree of automation. These services provide local databases, services to create the schemas, rule engines, security protocols and more.
• The data is unified across all edge nodes, transparently, through a global shared metadata layer (using a blockchain).
• Users and applications interact with a Single System Image.

Integrating AnyLog to a gRPC server

AnyLog connects as a client to a gRPC server. The .proto files define the data exchange, and the client functionality is enabled on the AnyLog node using an AnyLog command (see details in the AnyLog gRPC documentation).

With this setup, the distributed edge data is available to the applications, near real-time, using SQL and AnyLog commands, without centralizing the data, without cloud contracts, and without the need to know which are the edge nodes that host the data. The same setup will manage and monitor the distributed edge resources as if the distributed resources are a single machine.

AnyLog gRPC diagram

AnyLog instances are deployed at the edge and configured to request data over HTTP/2 (each AnyLog instance can be configured to request data from one or more servers). Data received from the servers is processed and hosted using the AnyLog services at the edge. This data remains at the edge, on the distributed AnyLog instances. Using virtualization, the distributed edge data is serviced as a unified collection, as if the data is centralized.