Mutable: Building the Low-Latency Cloud
Or: Bring Back The Mainframe!
We’re delighted to announce our investment in Mutable, a software infrastructure for aggregating CPU capacity, offering low-latency cloud services anywhere, on-demand.
It’s the year 2020, and our civilization fundamentally depends on CPU power. Mutable aims to radically change the way we allocate this computing power. Mutable’s low-latency cloud will allow any gamer to play on a 5000$ gaming rig using just a dongle and an internet connection. It will enable dramatic cost-reduction in IoT devices, allowing them to saturate our factories and cities, optimizing the way we manufacture, the way we live. Over this coming decade, Mutable’s localized edge cloud can unlock radical new opportunities around AR/VR, self-driving cars and AI, some of which we cannot yet fathom.
Ever since I learned about Mutable, I’ve begun spending an unhealthy chunk of my day angrily staring at my sleeping laptop. I paid more than $1000 for this metal box, and it’s just sitting there, unused, most of the day. Even when I use it, it is under-utilized! As I’m writing this, my computer’s CPU usage is hovering at around 20%. Overall, on average, I’m getting maybe a hundred-bucks-worth of my $1000 computer. I’m a ruthless optimizer, and I find this infuriating: if I’m going to pay $1000 for a machine that I’m only using at 10% capacity, it better well be a $10,000 machine!
After some soul-searching, I realized I’m just an old computer scientist: my true desire is a mainframe for the 21st century. One big room, serving everyone’s computation needs, and one dumb terminal sitting on my desk, like in the good old days.
Mutable is built to provide just that, on a global scale. Instead of having a strong machine on my desk that only I use, Mutable lets me get small time-shares in stronger machines, on-demand, for a lower price. It’s an “Uber for CPU time”.
The cloud was the first important step towards this vision, but the cloud is not good enough: cloud data centers are geographically far away from me, so any interaction with the cloud adds 50 milliseconds of latency or more, by design. The standard cloud is satisfactory if I want to run SaaS or to train large Machine Learning models. But what about tasks that require low latency or high bandwidth or real-time responsiveness, like gaming, video-editing, AR/VR, or predictive search? For those, current cloud services are not helpful. So even with the modern cloud at my disposal, I still need a strong machine on my desk to support these low-latency applications. I need to maintain this machine, update its hardware and software, service it, and pay for it. Wasteful!
Mutable fixes this: it aggregates supply of CPU time across many geographies, weaving local CPU services into a low-latency cloud. Mutable supplies access to strong machines that are closer to the end user, for example, machines already placed and maintained at my ISP, Telco service points or urban data center. This allows to outsource to the Mutable cloud many applications that currently have to be done on-device, most notably high-CPU gaming and high-CPU computer vision models (like those needed for self driving cars). As Mutable grows, my computer will outsource all CPU-heavy tasks allowing me to only leave a simple dumb terminal on my desktop, making my dream come true. In the future, after this transformation is complete, most people will have only a dumb terminal on their desk and in their hand, and low-latency CPU-heavy cloud computation will be as obvious and pervasively available as cloud infrastructure is today. Just like in the cloud revolution, once this transformation happens, a new wave of applications will be unlocked: applications that benefit from this new computational paradigm, and that we can’t even imagine today.
Realigning the Tech Market, Empowering Challengers.
Beyond personal computing, Mutable can revolutionize IoT, and specifically Industrial IoT. Manufacturers and users want to deploy a massive number of IoT devices, so they’d like them to be as small and cheap as possible. Most IoT devices should have a cheap weak CPU, and off-load all intensive computation to the cloud. But latency makes this vision difficult to realize. In industrial IoT, most edge devices are used to collect and process data from the environment and react to it in real time; but cloud servers are geographically too far away to enable a fast feedback loop. Over the last years, the Edge Computing industry has made major progress on software and hardware for these networks of IoT devices, but managing edge computing today is not much different from the pre-cloud-era use of IT resources: users have to buy and maintain their own hardware.
Mutable offers to fix this by disintermediating ownership and use, just like the Cloud did for SaaS. Mutable allows provisioning of computation resources as needed, on a pay-per-use model. Just as the electricity grid provisions kilowatt-hours of power, so will Mutable provision low-latency teraflops.
Over a longer time horizon, Mutable can reconfigure how we allocate CPU resources on a global scale, enabling Fog Computing. In such a future, computing devices will no longer be split to “servers” and “clients”, but rather constitute a latency-sensitive self-balancing “fog”. The entire network of CPUs will be thought of as points of surplus and demand, with Mutable aiming to balance those in real time, using dynamic pricing. Algorithms experts already use dynamic pricing to achieve central-planning-like efficiencies: such algorithms already enable efficient transportation, e-commerce and financial markets. These same type of algorithms will be used to clear the supply and demand of CPU cycles, achieving the best of both worlds: the optimization benefits of central planning, together with the economic efficiency of the free market.
Mutable democratizes infrastructure that is already available to the big corporations. Facebook, Google, Microsoft and others already operate many thousands of local data centers (called POPs) worldwide. These networks of POPs serve latency-sensitive requests, such as those from advertising marketplaces, predictive search, and new gaming platforms. Over the last year, these corporations have begun opening up these networks for external use; but still, in order to truly compete with FAMGA, companies today spend tens of millions of dollars to establish their own POPs. Mutable opens this field up to competition by offering a network of POPs available for anyone to use. This democratized infrastructure will open the playing field far and wide, just like cloud computing did in the 2000s or CDNs did in the 90s. Without the CDN revolution in the 90s, which democratized last-mile digital content delivery, we wouldn’t have had today’s high-bandwidth streaming internet: no YouTube, no Netflix. Mutable offers the next generation of CDNs — providing last-mile computation — and we hope it will have a similar dramatic impact, unlocking new business models.
So, how can Mutable go about building a low-latency cloud comprising of millions of servers in thousands of locations? One approach is to buy millions of servers and the rack-space for them. That’s what Amazon did to build AWS. But this doesn’t make all that much sense as a startup approach: it’s too expensive, and competes head-on with all the other players in the market. Instead, Mutable does not own any servers, and rather aggregates existing resources into an open network maintained via a powerful software infrastructure. Mutable then matches supply and demand on this network using a marketplace. In many ways, this is a blockchain-flavored solution, but without all the complexities that would arise from building a truly decentralized solution [1]. Mutable is thus a permissioned network, matching demand for low-latency CPU cycles with corresponding supply, and enjoying revenues from enabling and coordinating this market activity, just like in any other marketplace. To establish a large basis of supply, Mutable has made strong partners in the Telco and Cable operators: it took part in the CableLabs Accelerator that is run by the Cable companies and has built a strong value proposition for them. Mutable established a “long tail” of potential data centers and other providers to tap for extra CPU power around the world, as needed. All of this means that Mutable is able to offer a worldwide network of POPs, without owning any servers itself, just like Uber is a taxi company that owns no taxis. This approach of aggregating supply rather than owning supply is what gives Mutable the potential to become truly gargantuan: a single taxi company can only become so large, but a mobility marketplace can aggregate most of the available supply. Mutable may eventually aggregate most CPU resources on earth.
“NixOS Inside.”
Mutable’s mission to aggregate a massive amount of existing and nascent data center CPU capacity presents a slew of fascinating technical challenges. How do you establish a low-latency hosting layer over heterogeneous hardware, in heterogeneous geographies? And how do you aggregate and virtualize them to produce a seamless experience for both developers and end-users? If you’re in charge of maintaining a network of heterogeneous hardware, your devops nightmare scenario is for each machine to have its own quirks. On my home computer, that’s passable: I know which of my Lenovo’s drivers suck, and I restart the machine when things get out of hand. But a network of idiosyncratic hardware is entirely unsustainable. Mutable found a single important trick to addressing this: deterministic compilation and deterministic deployment. An Operating System which is built in a very predictable way is much easier to orchestrate.
Indeed, to provide a reliable and robust high quality service, Mutable opted for the most radical solution available: Mutable reinstalls its own tailor-made Operating System on every machine on the network. This OS ensures deterministic deployment and full control of the underlying machine and its network connectivity. Mutable’s OS builds on NixOS, perhaps the most obsessive operating system in existence. NixOS is an extremely stable and robust flavor of Linux. It uses a declarative, functional configuration model, which through meticulous setup eliminates surprises and ensures an OS that’s reliable, predictable, and easier to debug and troubleshoot. NixOS also comes with a strong existing toolset for deployment, virtualization, sandboxing and process orchestration. In short — the perfect tool for a control-freak devops. When onboarding a machine, Mutable installs its NixOS-based software image on it. This machine can then simultaneously serve whatever original purpose its actual owner requires, while also participating in Mutable’s marketplace.
Mutable’s CTO, edef, has been s notable contributor to the NixOS codebase for years and continues to support the community. Mutable’s origin in the NixOS ecosystem is one of its biggest strengths: it allows Mutable to build a unified virtualization layer on top of the eclectic hardware that will become the edge cloud.
But Mutable isn’t built on its virtualization alone: many other technical challenges exist when orchestrating such a huge network into a marketplace. Think of Uber for example: When clearing its marketplace, Uber needs to match riders with drivers within 10 seconds, in thousands of locations globally, at any given time. Mutable has the same challenges, just much harder: Mutable needs to handle millions of requests per second, while adding only miniscule latency. Dynamic pricing is another fascinating challenge: how do you adjust prices to balance supply with demand, in millions of locations worldwide, at 5ms resolution? As the network scales up, the fantastic team at Mutable is expanding to deal with these growing complexities. If you’re a great developer interested in solving these kinds of challenges, please get in touch with us or with Mutable! — they’ll be hiring very soon.
Overall, we at Lunar Ventures view Mutable as the next phase of Cloud infrastructure. In the early 2000s, cloud computing realigned the tech world, separating who uses the servers from who owns them. Cloud computing enabled economies of scale, and simultaneously allowed small companies to compete with market leaders. We believe the same pattern will repeat (or rather rhyme): low-latency edge cloud, which is currently a privilege of only the largest companies, will soon be available for challengers’ use, encouraging massive realignment and disruption. We believe that Mutable, with its great team and strategy, are the prime contenders to make this vision into reality. We are super excited to begin this partnership with Mutable, and can’t wait to help them create a new cloud allowing the next generation of applications to blossom.
Dr. Elad Verbin is a Berlin-based Computer Scientist. He is a founding partner at Lunar Ventures.
Lunar Ventures is a seed-stage deep-tech software VC fund, run solely by R&D experts, investing into globally ambitious startups across Europe. At Lunar, I work to revitalize the original purpose of risk capital — to grow cutting-edge algorithmic innovation into defensible tech-based unicorns.
Special thanks to Mitchell Shelowitz and the Shelowitz Law Group team for leading the legal work from the investors’ side.
Footnotes.
[1] I’m a big believer in blockchain-flavored startups that choose to implement their solutions without the blockchain part, in order to ship a mature product in a reasonable time frame. Blockchain technology carries great promise for the future, but it’s difficult and expensive to build mature and appealing products on it right now. A great design paradigm is “decentralizable software” — software that is technically built in centralized fashion where the startup has to be trusted, but that can be ported to a trustless platform relatively easily when/if needed. This allows delivering and iterating on the early stages of the project cheaply and efficiently, but keeping the potential to enjoy the benefits of decentralization when the project gains traction.
