AI/ML Infrastructure: Powering the Artificial Intelligence Revolution
It’s been nearly seven years since entrepreneur Andrew Ng, one of the foremost thought-leaders in artificial intelligence, stood in front of an audience at Stanford Graduate School of Business. “About 100 years ago, electricity transformed every major industry,” he declared. “AI has advanced to the point where it has the power to transform every major sector in the coming years.”
Ng’s analogy is now accepted as a universal truth in tech circles and, increasingly, in mainstream public consciousness. The overnight popularity of ChatGPT has only added fuel to the fire, igniting a public debate over whether artificial intelligence will spell the end of civilization as we know it or usher in a new era of prosperity and intellectual progress.
And yet, in many ways, the debate seems premature. Despite hundreds of billions of dollars pouring into machine learning and generative AI since 2020, uptake of artificial intelligence technology remains low and inconsistent across industries. While some functions such as risk and service operations have acted quickly to embrace machine learning, most industries are still laggards, with adoption rates in the single digits or low teens.
In addition, recent headlines on the struggle of well-funded Generative AI startups such as Jasper and Mutiny, or the slowdown in ChatGPT stats, cast doubt on what constitutes defensibility in the field of artificial intelligence. Ultimately, these data points raise the question whether we truly stand on the edge of an electricity-like revolution; and, if so, what form this revolution will ultimately take.
***
Building Blocks of a Revolution
A revolution — be it political, economic or technological — is seldom linear. (The etymology of the word itself, after all, refers to the continuous circular motion of celestial bodies). A revolution is a continuous push-and-pull exercise that forces us to slowly reimagine the world we inhabit; it is a process of discovery that requires us to move beyond isolated use cases and, instead, rewrite the rulebook in its entirety.
For instance, the mass adoption of electricity was far from instantaneous or straightforward. In Power and Prediction, economists Ajay Agrawal, Joshua Gans and Avi Goldfarb explain that, throughout the eighteenth and nineteenth century, electricity remained a scientific novelty, rather than an enabling tool of modern life. Even though the economic potential of electricity was well-known, both businesses and consumers were slow to adopt this new technology. Twenty years after Thomas Edison patented the first light bulb in 1880, only 3% of the US households had electricity, and it took another quarter of a century to reach half of the population. Similarly, while some industrial factories looked to electricity to lower the cost of power generation, it wasn’t until the 1920s that electricity supplanted steam in most factories nationwide.
The slow adoption of electricity was due to more than just simple inertia. It required a fundamental shift in the existing infrastructure. A new power grid had to be built out to support the transmission of electric power into factories, establishments and, ultimately, households. And as the power source became decoupled from machines and other means of production, both factories and homes had to be physically redesigned and reconfigured to this new reality. Ultimately it was these infrastructural changes that made electricity a transformational fixture in modern society.
If, then, artificial intelligence is anything like electricity, we must rethink and rebuild the infrastructure to power the AI/ML revolution — from how we collect, analyze and share data, to how we extract insights and make decisions.
***
AI/ML Infrastructure & GenAI Value Chain
Taking a step back, it’s worth defining AI/ML infrastructure within the context of the Generative AI value chain. Broadly speaking, the artificial intelligence value chain consists of six main components: (1) Hardware; (2) Cloud providers; (3) Data; (4) Foundation Models; (5) MLOps; and (6) Applications.
The two ends of the value chain — hardware and applications — have received a great deal of public attention over the past several months. Hardware players — companies such as Nvidia and Advanced Micro Devices — have seen their stock prices soar as demand for graphics processing units (GPUs) boomed. Similarly, AI-powered applications — from seemingly all-knowing chatbots to photo-editing apps — have captured the public’s imagination, ignited intense discourse on social media, and even flooded Instagram feeds (remember those Lensa AI avatars?).
But the middle layers — the “AI/ML infrastructure” which encompasses cloud providers, data, foundation models, and MLOps — remain woefully under-explored. With the exception of large language models developed by scaled startups like OpenAI and Anthropic, AI/ML infrastructure is still in its infancy.
Ultimately, our understanding of what artificial intelligence can accomplish is dependent on the development of robust infrastructure and tooling designed for this new era of machine learning. And, from an investment perspective, AI/ML infrastructure presents a compelling opportunity to realize superior returns, especially compared to applications and hardware. In fact, consider the recent history of venture investing from the early 2000s to today. Undoubtedly, massive IPOs of software applications such as Facebook, Uber or Coinbase will come to mind. However, the data paints a different narrative. Over the past two decades, a total of 401 VC-backed technology startups went public: 156 were infrastructure-focused, 150 were applications, and the remaining 95 were hardware. And although applications tended to have larger IPOs, infrastructure-focused investments outperformed both applications and hardware from a return-on-invested-capital standpoint. In our dataset, the median MOIC for infrastructure startups at IPO was 9x, higher than software applications (7.3x) and hardware (3.5x).
***
The Future of AI/ML Infrastructure
When it comes to AI/ML infrastructure, there are three investment areas that are especially compelling: data, model development and MLOps. The market map below showcases some early-stage stage startups operating across these three areas.
Data
For the past two decades, we have been told that we live in a “Big Data” world — a new technological era where the winners are able to harness the power of data to deliver superior products and services. But platitudes aside, the simple notion of “Big Data” belies a much more nuanced and complex reality. In fact, some industries — for instance, drug development — are hampered by data scarcity; other industries face limitations in their ability to use data that contains sensitive personal information. Similarly, enterprises sit on troves of proprietary data, but these datasets are often siloed in a multitude of systems that do not communicate with one another.
Today, most AI-applications are built upon foundation models that are trained on publicly available data. However, these models often result in generic responses and have raised concerns for potential biases and copyright infringement.
A number of exciting opportunities are emerging to bridge the gap between overly generic public data and impractical proprietary data. Startups focusing on better data aggregation and curation can deliver model-ready datasets in a low-code or no-code environment, while automatically detecting potential issues (e.g., outliers, mislabels, near-duplicates). In addition, synthetic data — algorithmically-generated datasets that are artificially manufactured rather than generated by real-world events — provides an effective solution in industries where real-world data collection is too expensive or time-consuming. In particular, we see opportunities for specialized synthetic data players focused on specific industries (for instance, healthcare, drug development or insurance) and use cases (e.g., computer vision).
Model Development
In addition to high-quality datasets, the power of artificial intelligence is highly dependent on the underlying foundation model. Unsurprisingly, the development of large language models or diffusion models is extremely expensive due to elevated cloud computing costs and infrastructure setup requirements. As a result, model development has remained the province of well-funded startups (OpenAI, Anthropic, Cohere) or technology giants (Google, Meta, Baidu).
However, innovation in model development has opened up new opportunities for smaller language models and even few-shot or zero-shot learning — a class of supervised learning models that can master a task using few training examples (or no training examples at all). Smaller language models present several benefits. First, they require far less computing power, which makes them a much more cost-effective and environmentally friendly solution. Second, they are particularly suitable for vertical- or industry-specific use cases. Verticalized models can provide support in niche industries or areas where foundation models like OpenAI fall short: from optimizing manufacturing processes to enabling more personalized content discovery. Ultimately, we believe that proprietary model development will offer companies a competitive advantage over startups that rely exclusively on existing foundation models; more importantly, it will enhance accuracy and customization, resulting in a better experience for the end-user.
Another lever to reduce the costs of model development lies in cloud computing. Because physical GPUs are expensive and hard to acquire, much of the work around machine learning occurs in the cloud. Certainly, incumbents such as AWS and Google Cloud have an advantage given vendor lock-in and high switching costs. However, specialized computing startups have an opportunity to gain market share by providing end-to-end cloud computing solutions that are decoupled from the need to set up one’s own infrastructure.
MLOps
The “software is eating the world” revolution of the past two decades has left us with a patchwork of tools and platforms to host, optimize, deploy and monitor applications and systems. Even for simple applications, developers have to juggle multiple tools to deploy an application in the cloud or manage feedback loops, and oftentimes they have to build out and manage their own infrastructure from scratch.
This approach is no longer sustainable in the age of artificial intelligence, especially given the high costs of training and running inferences for ML models. But, thankfully, a number of startups are tackling the issue of developer productivity by building comprehensive deployment solutions for AI/ML applications at scale. Similarly, low or no-code fine-tuning tools can help enterprises augment foundation models with proprietary first-party and zero-party data in real time. Other companies, instead, are developing optimization platforms to orchestrate GPU resources in order to better manage computing costs.
In addition, we are excited about industry-specific model hubs to help machine learning experts and data scientists share and collaborate on models. (One of Twelve Below’s portfolio companies, Health Universe, is building the ‘Github’ for Healthcare AI/ML, an open-source collaboration platform that hosts curated machine learning models for healthcare and clinical applications).
Finally, another MLOps area that is showing great promise is model evaluation and monitoring. Ensuring that a model is robust, compliant and ethical is a financial imperative for businesses, as data drifts, security leaks, or other malicious actions may trigger costly incidents and reputational risks. But more importantly, a culture of risk management will foster a deeper trust in artificial intelligence and further promote its adoption across companies and consumers.
***
Revisiting AI Applications
At first glance, the infrastructure layer may not seem as intriguing as the AI-powered applications that we have seen emerge over the past year and half — from AI-powered personal assistants, to website builders, to omniscient chatbots presents. That’s hardly a surprise: after all, most of us interact with applications on a daily basis, but we rarely think about the underlying infrastructure and computing platforms that power those applications.
However, from an investment perspective, we remain skeptical about the current wave of AI applications and their ability to realize venture-scale outcomes. Defensibility and distribution are the main challenges. Most AI-powered applications we see are built on foundation models such as GPT-3.5 or GPT-4, with little or no proprietary data; the lack of technological differentiation is an existential obstacle, as startups are immediately vulnerable to pricing pressures and product commoditization. In addition, incumbents have an unfair advantage when it comes to distribution. Both scaled startups and large enterprises have been quick to add AI features into their offerings: some have tapped into existing language models (for instance, Duolingo’s partnership with OpenAI), while others have developed their own models from scratch (e.g., Adobe’s Firefly). In either case, incumbents can easily layer AI on top of their core products and instantaneously reach millions of customers with minimal marketing expenses.
These challenges highlight the need for AI-powered applications to develop a unique go-to-market strategy, by focusing on under-penetrated verticals or underserved customer segments. Alternatively, winning applications must develop their own models or leverage proprietary datasets in order to maintain pricing power and rise above the competition. In other words, they have to start by looking at their own infrastructure.
That’s why, ultimately, we are confident that investing in AI/ML infrastructure will kick off a virtuous cycle: better infrastructure will beget better applications; and with better applications, we will further advance our understanding of the power of AI, resulting in stronger tooling and infrastructure.
***
If we believe that artificial intelligence is truly the new electricity, then we must do a better job of understanding the changes that are necessary to enable mass adoption of this technology. AI/ML Infrastructure is a great place to start this journey of discovery. Just like a new construction should not be built on an old foundation, this new age of artificial intelligence requires a whole new set of infrastructural solutions and tooling. Investing in the right infrastructure today will empower data scientists and machine learning experts to evolve their understanding of what artificial intelligence can accomplish. And — as we move from point solutions to system-wide solutions — businesses and consumers will be better equipped to harness the power of machine learning in their everyday lives.
If you have any feedback on this piece or you are building in the AI/ML infrastructure space, feel free to reach out via Twitter (@lorenzoligato) or email (lorenzo@twelvebelow.co). We would love to hear from you!
