(Foundation) Models as a Service: Big Tech’s Future AI Revenue Stream
The Generative AI Gold Rush for Big Tech and Startups
About the Author: Jason is an Investment Fellow at Thornapple. He brings both operating and investing experience through his tenure at several tech companies as a data scientist and across a variety of venture capital firms as an investor. Prior to this, he was an MBA VC Associate at Sweater Ventures, where he invested in early-stage consumer-facing startups. Jason earned his MBA/Ai from the Kellogg School of Business and McCormick School of Engineering at Northwestern University. Personally, Jason is a big fan of basketball and his NBA team is the Denver Nuggets.
Introduction
In 2017, Andrew Ng called AI the new electricity.
We’ve likely been hearing this more lately with the meteoric rise of OpenAI’s ChatGPT that propelled it to becoming the fastest-growing app of all time. All of this to say is that we will soon be living in an age fueled by artificial intelligence. Companies are trying to be a part of this watershed moment; from the startup side, we are seeing an increasing number of new entrants. In fact, NFX’s market map already has over 450 startups working in generative AI. Traditional Big Tech companies do not want to miss out either and have been building out their AI capabilities through massive investments, like Microsoft into OpenAI and Google into Anthropic, and continued development of foundation models like that of Meta’s LLaMA and Amazon’s chain-of-thought model.
While we are likely to see the mass adoption and proliferation of AI, much like the iPhone led to the explosion in the use of smartphones and mobile applications, many pundits are expecting foundation models, on which these AI programs are built, to become commoditized and that the true value will be in the application layer (much like mobile apps). Meta’s Chief AI Scientist, Yann LeCun, even goes as far as to say that the large language models are nothing new and have been around for years. He takes this a step further by saying, “It’s not only Google and Meta, but there are half a dozen [models] that basically have very similar technology to it.” As these models continue to advance, some experts believe that differences will continue to diminish and thus that these models will be interchangeable. Additionally, with the focus being on the development of these large language models, many companies are providing them open-sourced such as Hugging Face.
In this post, I will take a contrarian view and cover why I believe that these large language models will not become commodities but rather will become large new business units generating substantial revenue streams for current Big Tech companies and how startups can take advantage of this. I will specifically be examining the potential efficiencies that would be afforded to Big Tech by selling their already market-dominant cloud computing offerings and their proprietary large language models (LLM). Thus, to create value, startups should focus on:
- The application layer, built on top of Big Tech’s cloud and model layers, creates products and services tackling specific use cases.
- “CloudAIOps” to enhance the operational efficiency of the cloud and model layers.
What exactly is Generative Artificial Intelligence
Now you’re probably wondering what even generative artificial intelligence is. And while the field of artificial intelligence has been around for decades, only recently have large language models and transformer technology surfaced that allowed for generative AI to be born. Today what that technology entails is creating all kinds of new and exciting content that’s never been seen before solely based on user prompts! Essentially, large language models are fueled by huge amounts of text data, evaluated by upwards of billions of parameters from deep learning neural networks, and utilize the transformer architecture to generate that content. The combination of all these technologies is how users can generate brand-new images in DALL-E2 or get answers to any of their questions through ChatGPT.
AI + Cloud Computing: A Match Made in the Clouds
AI and cloud. Cloud and AI. These two technologies have been working closely together for years. This working relationship is now at a crossroads that will allow Big Tech to capitalize on the AI gold rush. Driving this forward are multiple catalysts that will lead to the joining of AI models and cloud offerings for these companies. One of these catalysts is the overarching macro tech industry beating (tech layoffs totaling over 192K in the US). Part of this tech downturn can be attributed to the slowdown in cloud services. In fact, the most recent earnings from Amazon, Microsoft, and Google indicate that cloud growth has slowed tremendously and will continue to do so.
The increased demand for AI, however, could actually signal a trend reversal and boom for cloud services. As more and more AI applications are created, more computing power will be needed, and these companies will turn to Big Tech for their cloud computing services. So, until this occurs, and until a path to profitability for generative AI is established, there would be a delay in realized profits. Therefore, as the tech industry continues to experience its cooldown and while the generative AI arms race heats up, Big Tech companies will explore other ways to create new sustainable value.
Introducing (Foundation) Models as a Service
Naturally, Big Tech will then want to capitalize on the gold rush that is generative AI. But rather than focusing on the application layer, Big Tech should utilize its scale and current offerings to dominate the cloud and model layers (as shown below in a16z’s AI Tech Stack). Few companies aside from Big Tech have the resources necessary for this infrastructure and so instead of diverting resources away to build products like chatbots, Big Tech could lean all into creating its (Foundation) Models as a Service (FMaaS) business unit.
FMaaS Big Tech
So why is Big Tech uniquely positioned to deliver FMaaS? I think the major factors to consider here are resources and costs, scale and data, and the ability to address AI issues. I’ll dive deeper into each of these in the following section.
Resources and Costs
To start off, a big factor is going back to resources and costs. Hundreds to thousands of GPUs are needed to train and power these LLMs. Currently, Nvidia’s A100 chip is the gold standard, and the cost shows that at $10K per chip (while we are no longer in a chip shortage, demand remains high and so chip prices have reflected that).
Now on top of those exorbitant chip prices, add in the training costs, GPU hours/computing costs, and inference costs. Here are a few examples:
- Training Costs: Meta’s LLaMA model used 2,047 A100 GPUs with an estimated cost of $2.4M over 21 days.
- Cloud Costs: Latitude CEO, Nick Walton stated that in 2021, his company was spending $200K per month for OpenAI’s software and AWS (OpenAI has lowered its price since this point).
- Inference Costs: Rowan Curran, a Forrester AI/ML analyst, estimates OpenAI pays $40M per month to process all ChatGPT.
So, to me, it doesn’t seem likely that startups would (or should) foot some of these bills to compete on models considering the resources and scale that Big Tech has.
Scale and Data
Now I mentioned the scale at which Big Tech operates but to put that into context: Google gets 80.2B visits per month and over 99K searches every single second, Amazon ships over 1.6M packages per day, Microsoft Office 365 holds over 45% of the office productivity market share, and Meta has nearly 65% of all US social media website traffic. These companies are processing a gold mine of data every single second of every single day. They know how we think, how we work, and how we interact.
Not only that, but Big Tech has access to vast amounts of capital (Google, Microsoft, Amazon, and Meta are holding $442B in cash stockpiles), the top engineering talent around the world (they’ve collectively employed more than 180K engineers), and can scale feedback from billions of users.
With all that scale and data, throw in some model fine-tuning and reinforcement learning through human feedback (RLFH), and Big Tech’s LLM will blow anything else out of the water.
Ability to Address AI Issues
It will then be up to Big Tech to ensure that their models are responsible as well as accurate. They also have the most to lose if their models are not up to snuff as seen in the past with Microsoft’s Tay going from playful to racist and more recently, Google losing $100B in market value after the demonstration of Bard giving an incorrect response about the first pictures of a planet outside of our solar system (the first images were actually captured by The Very Large Telescope in 2004).
In addition to bias, hate, and inaccuracies, we’ve also seen these LLMs “hallucinate” or make up erroneous facts altogether in response to prompts and being susceptible to jailbreaking through injection attacks by using prompts to override original instructions or filtering (as shown below to induce Bing Chat to ignore its mandates).
While these are major mishaps that have happened along the way, I do believe that given all the resources Big Tech has access to, and the reasons mentioned previously, they have the best shot to make these foundation models as error-free, bias-free, and hate-free as possible.
To recap, Big Tech has the resources, the scale, the data, and the know-how to ensure that their models are the best of the best. So where else can startups compete besides beyond the model layers?
Big Tech’s FMaaS Strategy
The genesis behind AWS was that Amazon was growing so rapidly that they needed additional computing power to scale their business. No one expected, not even AWS CEO Andy Jassy, that AWS would then become a source of competitive advantage and a powerhouse in the market. Since then, other Big Tech has followed AWS in building out their cloud computing arms but still lag Amazon’s first mover advantage market share of 34% (Azure is in second with 21% and Google follows at 11%).
Closed Ecosystems and Bundling
The genesis of AI revenue streams and departments at Big Tech will likely follow a similar path stemming from the need to grow and scale to keep pace with increased AI demand. Big Tech’s FMaaS strategy, however, will be primarily through closed ecosystems and bundling. Imagine Google pitching you a whole suite of products for all your generative AI AND productivity needs with Google Cloud Platform as your cloud layer, running on their Cloud Tensor Processing Units, building your applications on PaLM, and generative AI in the Google Workspace environment. In fact, on March 14, Google announced not just combining generative AI support and Google Cloud but also the launch of new products through Vertex AI, a generative AI app builder, and the incorporation of generative AI features into their Workspace. Google’s announcement was also on the heels of Microsoft’s on March 9 of Azure OpenAI Service that allows their customers to apply OpenAI models. And not to be upstaged, Microsoft released Power Platform with AI copilot for app building and its Office suite on March 16 during its Future of Work event.
While all of these are exciting announcements, they are tried-and-true strategies that have been implemented before. Big Tech has the ability to create these closed ecosystems and market increased operational efficiencies and cost savings to meet all their customers’ needs. In fact, in the mid-1990s, Microsoft established market dominance in search browsers by bundling internet explorer for free into their Windows operating system and causing the collapse of Netscape. Another example of this is how Microsoft has been eroding Slack’s market share by bundling Microsoft Teams with its Microsoft Office 365 offering.
While these tech companies continue to combine offerings in their existing products, it won’t be long before they dominate the market with their models, and from there can discontinue offering features for free or stop offering discounted prices. Once this happens, embedded staff and support will be needed and the creation of AI-specific departments will be born within these companies. Lastly, for tech companies such as Amazon or Meta which do not have productivity or cloud offerings respectively, their points of model differentiation can be for more nuanced proprietary data such as focusing more so on consumers or social media users.
Conclusion
How Startups Can Create Value in the Gen AI Gold Rush
We can see why these foundation models won’t be commoditized and why startups won’t find as much value in the infrastructure layers. That being said, there is still value for startups in this space. If you’re a founder and you want in on the generative AI gold rush, I have two recommendations for potential paths: focus on the application layer or focus on CloudAIOps to make the infrastructure layer more efficient to create value. If you’re an investor in generative AI, focus on startups solving these problems.
Application Layer
Starting with the application layer, if LLM as technology are synonymous with the iPhone, start building applications on that technology. As adoption and use become more and more mainstream like smartphones, using AI products and services will become second nature. For you, as a founder, you’ll have to build applications that solve specific problems, build a brand or community around your product, and make it so sticky and personable that it stands out above the rest.
CloudAIOps
If you want to focus more on the B2B side, I think there will be plenty of opportunity in what I refer to as CloudAIOps. This new market will be focused on making either the cloud or AI model layers more efficient or by making implementation and use easier for customers of those services. I think that this market will be larger, but also take a bit longer to flesh out. And that’s because it’ll take more time to really understand each layer involved, know and experience the pain points, and finally create products or services to tackle each of those pains.
Investor Viewpoint: What to Look For
If you’re an investor and evaluating generative AI companies, be wary of LLM or AI cloud startups. Any startups competing in this space will be outgunned by Big Tech’s scale, resources, capital, and vast proprietary data. In the end, those startups will be building from ground 0 and will have no exits in sight. Thus, they will continue to burn cash until they find a suitable pivot into the application layer or CloudAIOps.
Tactically, in terms of business model, I think that some combination of the SaaS model and/or the per-usage model will likely win out. Keep a close eye out for the startups who are building stickiness with consumers or embedding themselves in their customers’ tech stacks. These will be the startups that create the most value by solving a specific problem or making a product that’s 10x better than the rest. An example of such could be a generative AI learning platform that creates personalized lesson plans.
All in all, generative AI is here to stay and it’s a tide that rises all boats. Everyone from Big Tech to startups to everyday people will benefit from its proliferation and value creation. The team at Thornapple River Capital and I are excited to both witness and be involved in generative AI’s journey.
