The AI and IoT Revolution Is Here, But It’s Missing a Key Component
CES 2025 offered an exciting look into the future of technology, with AI and IoT innovations taking the spotlight. Yet, after conversations with over 100 IoT companies, a significant question looms: how can we efficiently manage distributed data?
Today’s data management solutions rely on centralizing data in the cloud to generate insights and make data-driven decisions. But this model is inherently paradoxical. Why is data sent away from the application at the Edge to the cloud only for the decisions to be returned to the Edge?
People move data from the Edge to the cloud not because it is the best solution but because there is no data management solution for the Edge. And let’s be honest: cloud providers aren’t going to develop such a solution anytime soon. Why would they? The current Edge-to-cloud model works entirely in their favor. By centralizing data in the cloud, providers position themselves as gatekeepers of your data, raking in massive profits through usage-based pricing. These pricing models aren’t about covering the cost of service — they’re designed to cash in on the value of your data. The more essential and frequently accessed your data is, the more you pay.
From a technological perspective, revolutionizing the Edge requires breaking free from the limitations of cloud dependency. The current Edge-to-cloud approach stifles efficiency and eliminates the possibility of true real-time capabilities. Real-time can only happen if data remains in place at the Edge. Instead of centralizing distributed data in the cloud, the future needs a data management platform for the Edge, a platform that handles physically distributed data directly at its source. The solution is AnyLog.
What is AnyLog?
AnyLog is a software company that redefines the current data paradigm: instead of moving the data to the cloud to execute queries, the queries are sent to where the data resides at the Edge.
In other words, AnyLog brings cloud data services to the Edge without centralizing the data.
The core ideas behind AnyLog were first published at the Conference of Innovative Data Systems Research (CIDR) in 2020 [1], and subsequent advancements and development results were published in CIDR 2024 [2]. Today, the AnyLog software stack is available commercially and as an open-source project distributed by the Linux Foundation called EdgeLake [3, 4].
So, how does AnyLog work?
To better understand how AnyLog works, let’s first look at how data is managed at the Edge today.
Data management today …
In industries like oil rigs, EV charging networks, or energy production farms, vast amounts of data are generated to monitor and manage physical infrastructure. Having a unified view of this data is crucial for operators to answer questions like: “What is the day rate of my oil rigs?”, “How much energy was generated in the last 10 minutes?” or “What is the current charging load?”
Answering these questions typically requires using programmable logic controllers (PLCs) to collect and send data from each physical device (e.g., chargers, oil rigs, solar panels) to Edge devices (e.g., servers, gateways). Since, at this point, the data is still distributed across multiple Edge devices, users choose to centralize the data to the cloud to create a unified view. Edge applications are then connected to the cloud to generate insights and control decisions.
The key question is whether we can make the same decisions and have the same data interface directly at the edge without moving data to the cloud. The short answer to this question is yes.
Data management with AnyLog …
Similarly to the traditional approach, data generated by each physical device is pulled by a PLC and sent to Edge devices. However, instead of transferring this data to the cloud, each Edge device installed with an AnyLog agent joins a peer-to-peer (P2P) data network that effectively creates a virtual data lake with the same interfaces as in the cloud. This new data layer enables applications to seamlessly access, extract, and aggregate data without modifying APIs or determining which Edge device holds the relevant data. Although the data layer is composed of physically distributed Edge devices, AnyLog makes data appear to applications as if it is stored in a single database.
Some of AnyLog’s current use cases
- Transforming the city of Sabetha, Kansas into a Smart City: Bachelor Controls, a solutions integrator, installed AnyLog’s software on the city’s existing infrastructure. The project took two weeks to go operational. After two weeks in production, Sabetha terminated their cloud contracts; the entire city infrastructure is now managed using AnyLog without incurring any cloud costs, all by utilizing existing infrastructure. Read more in the Case Study.
- IBM Integration: IBM integrated AnyLog into Open Horizon, and it can now be deployed and orchestrated on a cluster of nodes with a click of a button. More information. GitHub Link.
- Autonomous boats: Compagnie des Bateaux d’Annecy operates 15 tour and cruise ships, including restaurant vessels, across the Franco-Swiss lakes. To optimize operations and management of the fleet, AnotherPeak, a local system integrator, has deployed AnyLog nodes across the fleet to track engine and battery performance, manage alerts, and predict charge levels.
- Federated Learning Platform: Execute federated learning workloads with AnyLog as the facilitator. Deploy AnyLog on each Edge device. Connect your data. Define your models. Let AnyLog facilitate the entire process. More to come.
Interested in utilizing AnyLog or EdgeLake (open source) in your current solution? Contact us here or through email; we would be more than happy to discuss your use case.
[1] CIDR 2020 Paper
[2] CIDR 2024 Paper
[3] AnyLog Homepage
[4] LF Edge EdgeLake Project Link
