Cloud Wars: The Attack of Snowflakes
Erik wrote a delightful post this week, combining the counterintuitive ideas that (a) the lowest cloud infrastructure layers are not commodity services, and (b) this means that the cloud providers could be happy ceding ground to others for higher level services, turning into pure play infrastructure platforms. Erik, please keep me honest if I got these wrong!
I’m in violent agreement with the first premise that the lowest cloud infra layers are not commodity services¹. But I think it’s unlikely that cloud providers would be happy ceding ground to others on higher level services. Cloud providers want to build higher level services though they can live with being a platform for others. Amazon loves being a retailer though it enthusiastically serves a thriving marketplace. In the 1990’s, while Microsoft built and owned the Windows platforms, that didn’t stop them from taking on office productivity.
Winter is coming…?
However, Erik argues that cloud providers are facing headwinds in going up the stack, pointing to increasing competition from startups, poor developer experience, and concerns of vendor lock-in.
Far from being a threat, more competition in startup land is good for cloud providers overall: when one Snowflake hits a home run, more investment flows into new services built in the cloud. Scoping down to just data warehousing, the market is big enough for Redshift to do reasonably well, independent of existing and new snowflakes. I don’t think this is a winner take all situation.
Developer experience on AWS can be sub-optimal, for sure. As a thought experiment, how different is stitching together tens of AWS services compared to stitching together tens of open source projects or commercial services built by independent startups. Do we expect all open source projects and startup services to all behave elegantly together? Narrowing down the scope, how does one AWS service API compare with a competitive service API? My bet is that it’s on par, if not better. I don’t think the AWS Management Console is the key point of comparison for most developers. Perhaps I’m getting this argument totally wrong… so please feel free to straighten me out!
“If you’re an ambitious person, do you go work at AWS?” I loved working at AWS not for the money or the scale. It was 99% about the people I worked with. Quietly ambitious and culturally self-selected to constantly improve. Prima donnas, who think they’re the best in the world on a subject, are rare in Amazon. A better question might be: If you’re building a new database, would you work at AWS? Maybe, maybe not². Perhaps a tangent, but what is it about databases that make them attractive as independent opportunities?
On vendor lock-in, Erik himself refutes these concerns in his post, and I won’t repeat them here.
Up the Stack: Vertical Integration
Contrary to perceived headwinds, Azure and AWS are increasingly moving up the stack with industry-specific solutions. Last month, Microsoft announced a re-org to shift the focus of its industry solutions org to be more Go-To-Market-oriented, seeking results, not just incubating new ideas. At AWS too, this industry focus has been in the making for years and isn’t due to Adam Selipsky’s new leadership as Ben Thompson comments while summarizing this year’s re:Invent keynote:
What this seemed to represent is AWS shifting from merely offering infrastructure primitives to embracing the idea of being a platform for platforms. This is different than Microsoft’s approach of offering both infrastructure-as-a-service via Azure, plus its all-encompassing Teams/Office-centric software-as-a-service, but it is a shift; according to that Bloomberg interview this very well may be due to the leadership change.
I happen to like Thompson’s phrasing of ‘platform for platforms’. For example, AWS would be a platform for Nasdaq, which is itself a trading platform. However, AWS’s platform for Nasdaq is very different from its platform for Dish. It’s also likely that more developers touch the Nasdaq or Dish platforms than the underlying AWS platform in the future. This reinforces Erik’s prediction that by 2030 ‘most developers don’t interact with cloud vendors’, although in a different way.
Evolving Consumption Models
Like Erik, I also expect the cloud consumption model to evolve, again in a very different way. Turning non-consumption into consumption is still the big opportunity for cloud providers today. In comparison, the shift from direct consumption into indirect consumption is perhaps one of several means to the end. As big as the cloud provider business appears, it’s only about 10% ($445.3 billion³) of the global enterprise technology spend, which is estimated to be $4.24 trillion in 2021⁴.
Moving enterprise workloads to the cloud will likely require a more vertically integrated approach, with both technical and business model innovation. Serving up a generic tech platform or building blocks won’t cut it. For example, if Nasdaq needs EC2 instances in its own data centers to comply with its fairness and latency requirements, they need Private Local Zones, which require a different economic model and a new set of technical capabilities compared to EC2 in public AWS Regions and Availability Zones.
Predictions
In the next decade, I expect there to be more versions of a financial services cloud, an industrial services cloud, a healthcare services cloud, a telecom services cloud, an automotive services cloud, and so on. Each cloud provider would be a ‘platform of platforms’ as Thompson calls it. The underlying cloud provider platform would be diverse in capabilities and complex in its dependencies. The industry specific platform is where most developers would engage with the cloud.
Many organizations already have a version of this internally. For example, while Stripe uses AWS extensively, it has an internal platform team that abstracts infrastructure services and focuses on improving internal developer productivity. The last mile challenge of molding cloud services to your own needs is real, even for the tech-forward ‘digital natives’.
Most enterprises, however, still have miles to go. Their journey will be paved jointly by cloud providers, by teams within the organization as well startups and others building industry-specific capabilities.
A vast majority of the cloud services opportunity is still in front of us. For cloud providers, moving up the stack is an imperative, not an option. If they have to incorporate third party offerings or build it themselves to fill in the gaps to move up the stack, they will. Fortunately, while we all love industry maps with clear boundaries, it is not a zero sum game and the rules of the game are yet to be written.
[1] Back in 2015, when I first joined EC2, I used to think that compute and storage are utilities, and that should make them relative commodity services. I was wrong. Choosing your compute, storage, and networking service provider is like choosing the country you want to live in. It determines the clock speed of your life. EC2 is one of the top innovation machines in AWS, if not in Amazon. S3 is the most reliable service I can think of. EBS solves some of the hardest problems and is a money spinner at this scale.
[2] Beyond Mongo, Elastic, and Snowflake, I’m thinking Planetscale, Fauna, RelationalAI…
[3] https://www.yahoo.com/now/global-cloud-computing-market-research-174500110.html
[4] https://www.statista.com/statistics/268938/global-it-spending-by-segment/
