Vision for the Edge Cloud

Edge Cloud seems to be stuck in 1.0 that started in 1990s. The original use-case was just about the Content Delivery Network (CDN). With evolving, modern workload patterns, it’s time to rethink the Edge Cloud.

1. 1.0 = read-only: Deliver static content from an edge location. Let’s call this “read-only” use-case because all end consumers did was to read the content.
2. 2.0 = read + cloud essentials: Edge clouds added some essential functionality to the ‘read-only’ use-case. Some examples include DDoS protection, ‘light’ compute at edge, access logging, load balancing. You could argue that observability is a transformative change but shouldn’t everyone add observability wherever there is infrastructure? For that reason, I consider observability to be a “cloud essential” as well.
3. 3.0 = read + write + execute incl. AI inference at the edge: Modern use-cases are way beyond the original ‘read-only’ use-case. AI & IoT use-cases are evolving more rapidly. Some examples of ‘write’ use-case include capturing clickstream data from Apps, streaming data from IoT devices. These streams of data need to be collected asynchronously and with low-latency. Similarly, AI inference can be delivered from the edge — model training may happen in the central cloud but delivering inference is better done from the edge with low-latency.

Edge Clouds must evolve to address these modern use-cases. Central clouds should be limited to use-cases that handle transactions that require strong consistency. For all other use-cases, where async / eventual consistency is enough, it should either be ‘edge only’ or ‘edge first’. Given that several use-cases will require seamless handoff between edge and central cloud, my gut is that ‘edge first’ will be the new normal.

But there is one problem — it’s too expensive to deploy servers / containers / shards across hundreds of edge locations. You’ll end up running under-utilized capacity across so many edge nodes that it will become prohibitively expensive to maintain. The answer is — serverless — Edge Clouds have always run CDNs in serverless fashion and they need to bring that paradigm to modern use-cases as well.

Modern Edge Cloud presents a great opportunity to expand use-cases beyond traditional CDN use-case

Modern Edge Cloud = r-w-x (read-write-execute) including IoT & AI use-cases + Edge First + Serverless.

Background

I’ve been leading Product Management for cloud based products for more than 10 years — including my leadership roles at Engine Yard (PaaS) and CloudFix (cloud cost optimization). While I felt the need to run more workloads for these products on the Edge, I could live without it.

Recently, I started work on an AI startup, EC7 (eCommerce price optimization with AI), that relies on (a) streaming data collection (b) ML models and (c) using Generative AI. I realized that these are better run on the Edge but cannot really do any of these on the Edge cloud (not even on AWS Local Zones).

For our clickstream data, low-latency data capture is extremely important but once the data is captured (and client got 200/202), we really do not care if it takes few minutes or few hours to process the data — just asynchronous low-latency data collection is all we need.

For ML models, we have data pipelines, lakes in the central cloud but once the model is built, all we need to do is deploy that model on the edge for low-latency inferences — there is no need for central cloud when it comes to inference.

For Generative AI, wouldn’t it be great to cache LLM queries and responses on the edge with a vector database so that we don’t make API calls every time we need a response? The idea is to achieve low-latency while saving on API calls and tokens.

Unfortunately none of these are possible in today’s Edge cloud. Developer demand for these modern use-cases on the Edge will rapidly increase with AI & IoT explosion.

Demand driven by AI & IoT use-cases presents a great opportunity for the Edge Cloud to expand dramatically beyond the CDN-era.

So, what do I mean by ‘Edge’?

When I say ‘Edge Cloud’, I am not referring an Edge Cache, Point of Presence (PoP) — those are all the words that were defined for the CDN-era.

Visual representation of where modern Edge Cloud lives

Let’s think through first principles of what we expect from the modern Edge Cloud (1) Low-latency in read, write and execute use-cases (2) Fully Managed infrastructure and (3) Cost efficient. As long as we deliver on these promises, it’s less relevant how many POPs there are and how close the Edge Cloud lives to end consumer.

For those reasons, I believe that the modern Edge Cloud will live somewhere between (green shaded area in the diagram) how AWS defines Local Zones and the PoPs / Edge Caches of the traditional CDNs.

It’s perfectly alright to start with Local Zones level of infrastructure to begin with as long as we can deliver on those three promises.

Serverless, Distributed & Purpose-Built

CDNs have delivered the ‘magic’ very well for decades. The ‘magic’ lies in three important factors:

1. Serverless: Developer did not have to deploy a server, container or shard. Just configure the CDN and the rest of it was fully managed on behalf of the developer.
2. Distributed: The content and end-user connections are seamlessly distributed across edge nodes without the developer having to worry about how those requests are distributed to deliver low-latency.
3. Purpose-Built: Each service has to be purpose-built — we cannot just throw compute at the developer and let them figure it out. For example, if Vector Databases are offered in CDN-fashion, then you cannot just throw compute or the entire database at the developer — the service has to be well-defined and function out of the box so that the Vector DB can just be used to cache Gen AI embeddings.

When this magic is delivered well, (a) these edge services are easy to deploy = Developer Experience magic and (b) it costs the developer same $ irrespective of if 1M requests are served from the central cloud or from 200 edge locations. This is not pricing magic, it is the product architecture magic.

Runs in Your Cloud

I am generally not a big fan of multi-cloud — I tend to believe in investing one cloud (AWS is my favorite) and going deeper to leverage all the cloud-native functionality. However, when it comes to Edge Cloud, I believe that customers should be able to run their Edge Clouds within their own hyperscaler cloud— whether they choose AWS, GCP or Azure.

That is because (1) there are traffic / egress charges involved in many use-cases and (2) many use-cases will be edge-first — meaning that there will have to be integration between the edge and the central cloud. For example, if you are collecting streaming data — it may have to be stored in the central cloud because your ultimate goal is to get that data into the data lake.

Bonus: AWS Services that should run on the Edge Cloud

Running these on Local Zones to begin with may be a good idea.

1. Simple Queue Service (SQS)
2. Simple Notification Service (SNS)
3. Kinesis Data Streams
4. Managed Service for Apache Flink
5. DynamoDB

Note that this is not a comprehensive list. There are other services such as ElastiCache, Redis and OpenSearch but those are not currently offered in a true serverless fashion — as a result, it would be a bad idea to offer those on the Edge.

Hope you enjoyed reading the article. Please reach out to me if you have any comments or suggestions.