The M-shaped software
Lessons I learned from AWS re:Invent
I still remember the days when AWS re:Invent conferences were small events for AWS (Amazon Web Services) customers. Little did I know. In the past decade, AWS re:Invent has grown into one of the world’s largest technology conferences. It is still about technologies developed and sold by AWS. But AWS is so dominant in enterprise computing today, much of the software innovation happens on AWS these days. That makes AWS re:Invent relevant for everyone in the software industry.
AWS re:Invent 2019 was held in Las Vegas from 12/2 to 12/6. There were 65,000 attendees and most paid $1,700 ticket price to be there, to hear what AWS has to sell, and how other AWS customers have bought / used AWS services — a true testament to AWS’s success! Las Vegas is no stranger to mega-events / conferences. But for a week, AWS re:Invent turned the Las Vegas Strip into a giant geek fest with speaking sessions spanning across 5 large conference centers. It could take an hour just to walk from one session to the next. I was one of the attendees who went there to learn about the innovations in the AWS ecosystem and how such innovations will impact the software industry in general.
The most important lessons I learned? Software is becoming both more abstract and more concrete at the same time. The middle layer that connects high-level users to native hardware is going to be more important.
Software is becoming more abstract
AWS is a computing infrastructure provider. It grew so big because infrastructure is such a pain point for most organizations. The cloud is supposed to be simple — providing computing power like electricity or tap water! However, as AWS is increasingly adopted by enterprises, it has become very complex. AWS now has over 100 different products.
To continue its success, it is natural for AWS to push for simpler and easier to use infrastructure products. In other word, software is becoming more abstract to hide away the details and complexities of the underlying infrastructure.
One of the hottest topics in re:Invent this year is serverless microservices. The idea is that developers should not be worried about servers, virtualization, containers, OSes, platforms, resources, and all the complexities that come with managing servers. Why can’t developers just write code, deploy it to the cloud, and make it automagically work everywhere under any load? After years of hype, the underlying infrastructure software for serverless is finally maturing with the advancement of Docker, K8s, and related technologies. In this re:Invent, there are multiple use case studies for serverless microservices.
During the keynote, AWS CEO Andy Jassy announced several AI products, such as the Amazon Fraud Detector, Amazon CodeGuru, Contact Lens for Amazon Connect, and Amazon Kendra. Those are essentially packaged AI microservices AWS itself offers to application developers.
One of the biggest announcements from re:Invent is the launch of SageMaker Studio. It is a web-based IDE for all your AI needs. Through the studio, users can directly interact with the AI microservices AWS built on the backend. AWS not only builds infrastructure services today, but also UI for business users to adopt such services.
The era of serverless microservice might finally be upon us. By serverless, software is becoming increasingly abstract from its underlying infrastructure.
Software is becoming more concrete
On the other end of the software infrastructure, software is also growing close to the native hardware. It is less abstract and more concrete.
Driving this evolution is that fact we are already in the post-Moore's Law era. The performance improvements demanded by the users cannot be simply met by faster CPUs anymore. AWS pioneers the approach to developing custom hardware that delivers high performance in specific applications, such as networking, parallel processing, and AI inference.
A great example is the AWS Nitro. It provides a suite of dedicated hardware I/O cards, controllers, and security chips. By utilizing a software infrastructure that takes advantage of the Nitro hardware, AWS was able to deliver new EC2 instance types that are both faster and cheaper. With Nitro, AWS shipped 3x more EC2 instances in 2018 than in the prior year.
Peter DeSantis demonstrated in his keynote that, through Nitro and a hardware optimized networking software stack, the EC2 c5n.x18large instance can now achieve almost linear performance improvements as more CPUs are added to the system. That is the same scaling characteristics as a Cray supercomputer. It allows AWS to offer “supercomputer as a service” to customers.
In re:Invent 2019, however, AWS went all out and announced new chips and hardware for accelerated AI and big data operations.
Graviton2 is an AWS designed CPU based on the ARM architecture. Compared with Intel and AMD’s x86 CPUs, Graviton2 delivers 40% more performance at 20% less costs. It also provides hardware acceleration for compression workloads.
The AWS Inferentia is a hardware processor designed for AI operations. For neural network inference workloads, Inferentia is 3x faster and 40% cheaper than generic CPUs.
AQUA is a new hardware-accelerated cache that improves query performance by 10x for data warehouse applications.
As hardware becomes more and more specialized, the demand also increases for software that can “natively” access hardware and take advantage of their special features. In that sense, software is becoming less abstract and more concrete. Instead of generic high-level applications, developers must again deal with quirks and special features of particular hardware.
Opportunities in the M-shaped world
The growing importance of both abstract and concrete software applications creates an M shape in the landscape where the importance of traditional middleware such as generic application servers and containers is probably going to be diminished.
However, that only gives rise to a new generation of middleware that enables easy microservices development and deployment on specialized hardware. From my own point of view, the emerging WebAssembly technology might just provide such a connection.
WebAssembly as middleware
WebAssembly (Wasm) started as a high-performance execution engine inside web browsers. It is language-agnostic (and supports popular new languages such as Rust), platform-agnostic, lightweight, fast, and provides native hardware access through a modular security model. Those features make Wasm a great execution engine for cloud-native microservices.
- Compared with Java and JavaScript virtual machines, Wasm supports many more programming languages, is much lighter, and provides native access to hardware features.
- Compared with containers like Docker, Wasm is much lighter and faster. Wasm apps run on any host system without change. Wasm also has a more refined modular security model for accessing native OS and hardware.
You can read more about the rationales and history for WebAssembly on the server-side.
A new hope
At Second State, we provide Wasm compilers, runtimes, and native hardware modules that are optimized for server-side applications. Without turning this article into a self-serving advertisement for my own company, I will just point you to two open-source products we recently announced in this area.
The developer preview of the Second State Virtual Machine
The developer preview of the Second State Wasm compiler
If you are a developer, I would love to hear your thoughts! Please get in touch via email michael AT secondstate.io.
What’s next
We believe that Wasm will have wide applications in blockchain, AI, and cloud-native microservices. We will see more of Wasm in future AWS re:Invent. Until then, let’s meet in Feb 2020 at the WebAssembly Summit conference on Google campus to discuss the future of WebAssembly on the server-side!
About the author: Dr. Michael Yuan is the author of 5 books on software engineering. His latest book Building Blockchain Apps was published by Addison-Wesley in Dec 2019. You can get started with your first decentralized web app in 5 minutes using the BUIDL online IDE (with tutorials from the book). He is also the CEO of Second State, a company that focuses on bringing WebAssembly and Rust technologies to cloud, blockchain, and AI applications. Prior to Second State, Dr. Yuan was a long time open source contributor at Red Hat, JBoss, and Mozilla. Outside of software, Dr. Yuan is a Principal Investigator at the National Institutes of Health, with multiple research awards on cancer and public health research. He holds a PhD in astrophysics from the University of Texas at Austin.
