Let serverless solve the technology problems you don’t have
This is a story about how serverless can help you do the things you already know you should be doing but aren’t. Before we get started let’s establish a shared understanding of a few things:
- Your organization doesn’t have technology problems. You might think it does, but the odds of it being true are not great. You’re probably not in the business of landing autonomous rockets on autonomous barges, for example. And my guess is that you’ve never created a machine that uses the convergence patterns of 200 individual radiation beams to target and kill cancerous cells in the human brain. Ditto for installing atomic clocks in datacenters around the world to get globally consistent SQL transactions.
- Your company still has important problems to solve. You might be operating in the media industry, where drastic changes in how content is produced and distributed have left you scrambling to figure out what viable revenue streams you can build a company around. You might be a retailer, competing to anticipate what consumers want and fighting to maintain brand loyalty when more and more things can be compared and purchased online. You could be a non-profit with a social mission, such as improving childhood educational outcomes or increasing access to basic services. Suffice it to say that your business has important, challenging problems to solve — and solving technology problems isn’t your business.
- Despite points 1 and 2 your organization is spending its time as though you were, in fact, in the business of solving technology problems. What percentage of time would you estimate your technology teams spend solving your business’ problems? Because everything outside of that — load balancing, capacity planning, deployments, five 9 availability engineering — is undifferentiated heavy lifting. Sure it’s critical to your business, but you aren’t solving any new problems that others haven’t, and the problems you are solving aren’t differentiators — they’re table stakes. Hard, messy, complicated table stakes.
Luckily for all of us a lot of really bright people have been hard at work figuring out how to spend more time solving our business' problems, and they’ve been sharing this information publicly for years. Entire conferences are dedicated to it. These practices and cultural changes are commonly referred to as DevOps. Practitioners and evangelists don’t always frame DevOps as a way to spend less time on technology problems and more time solving your business’ problems, but it’s all there. What is faster time to market if not the ability to try and solve your business problems more quickly than before? There’s so much quality material around what DevOps is and what you should be doing that I have nothing new to offer. If you’re looking for a terse primer of the concepts AWS has a good page for getting started.
The thing is you probably know all of this. Most of the people in your organization probably know all of this. So why aren’t you spending more of your time solving your business’ problems?
Chemical systems and reactions provide a useful analogy. All systems want to move from higher to lower energy states. A stable system indicates that the necessary amount of activation energy hasn’t been applied, or at least not for the required amount of time. When it comes to the physical world around us this built in change deterrent is fantastic and ensures that things stay relatively stable.
Companies similarly seek to move from higher to lower energy states. No sane organization would ever consciously decide to spend more time solving technology problems and less time solving their business’ problems. The reason they get stuck in a high technology-problem-solving state is that change is hard — it requires effort and time. And perhaps most perniciously, moving to a lower-energy technology state requires passing through an interim state where the organization is spending more of its time solving technology problems. It’s an investment, and the higher the cost and the longer it needs to be sustained the harder it is to make.
In chemical systems catalysts are powerful tools which change the equation — they allow reactions to occur with less effort and in a shorter amount of time. If you are trying to spend less time solving technology problems, finding a catalyst that will help make this process easier and smoother should be your top priority.
Serverless is the most compelling catalyst out there to help your organization make these changes. As with all buzzy words serverless can mean different things to different people. By serverless I mean ephemeral functions which encapsulate your business logic and expose organizational capabilities. These functions integrate with and consume managed utility services, and all of this is supported by tooling that structures your solutions and takes care of boilerplate, undifferentiated heavy lifting for you.
The functions and managed services that I find most compelling in this space are on AWS and Azure. AWS was the first big cloud provider in the functions space with the launch of Lambda in 2015, and Azure released their own Azure Functions offering late last year. For both platforms Serverless Framework is the secret sauce that brings all of this together and makes it accessible to everyone.
With that out of the way let’s look at how serverless helps get you out of an undifferentiated heavy-lifting hell and into a more pleasant space. These changes aren’t exhaustive and don’t cover the full gamut of what DevOps has to offer — these are just a few examples I’ve come across that have resonated with me.
Guard rails for the yellow brick road
If DevOps Oz is the destination, getting there entails following the yellow brick road. As many organizations have discovered, this road has sign posts but no guard rails. Tools like Docker have helped some companies along their journey, but the experience of many has been that doing the wrong thing is still easier than making the right choice. These kinds of tools and patterns work great and expedite the process for organizations that were going to make the change with or without the tool’s help, but do little for the organizations that need a tool to enable the change.
Serverless, and especially the Serverless Framework, make doing the right thing easier. Trying to put too much functionality into a single service? While nothing will technically stop you, you’ll feel the pain right away and get nudged into decomposing into multiple, smaller services. Having a hard time restructuring into embedded product teams? The Serverless Framework combines infrastructure and code into a single package, making it easier to deploy in complete, self-contained units.
There are still technology problems to solve with serverless — it’s a catalyst, not a panacea. The important thing is that even with these new problems you’re miles ahead of where you were before, and the self-correcting nature of the framework and tooling helps you stay on the right track for your entire journey.
Two pizza teams or bust
Large teams suffer from too many dependencies and an explosion of possible communication paths. Too many of the people on the team are too far removed from customer feedback to be able to identify gaps or problems, let alone course-correct and resolve them in a reasonable amount of time. Large teams also present an obstacle for creating embedded teams that own the full life cycle of a system or process — adding more people to an already large team slows things down even more and diffuses responsibility, when you should be focused on speeding things up and empowering people to make better decisions.
Serverless tackles these problems on both fronts. It makes your existing product teams smaller by helping you decompose your monoliths into microservices, and it reduces the amount of specialization needed to take an application from idea to production — thereby reducing the number of people needed to support this work. Cloud provider shared responsibility model for the win!
No more death stars
Have you ever been part of a project that was planned to last over a year? How well did that go?
These types of projects happen to organizations caught in a painfully reinforcing cycle of shortsightedness. Common ingredients include monolithic systems with overlapping capabilities, integration as an afterthought, and organizational silos within which decision makers only maximize against ultra-local consequences — tragedy of the commons be damned. In this type of environment a boiling-the-ocean approach appears to be the only option for solving Today’s Most Pressing Problem. The problem is a hard one to address because death stars breed more of the same — each ocean boiling leaves the enterprise less able to deal with Tomorrow’s Most Pressing Problem and the cycle is perpetuated.
By helping you decompose your teams and systems into smaller ones, serverless helps you escape this death spiral. API design is a first-class citizen of serverless architectures, and the tools make it dead simple to create clean, simple, and friendly APIs for others to consume. This clean decoupling helps you break down the monoliths and frees up your teams to operate and iterate more independently of each other. This type of decomposition has always been the right way to do things, and by making the right thing the easy thing serverless saves you from yourself.
Breaking the technology diversity & governance dichotomy
Running and maintaining your software is critical to the ongoing success of your company and it would be irresponsible to not have some form of governance in place to support it. At many organizations software governance takes the form of limiting developers’ choices by standardizing — becoming a Java shop, for example. Too often this results in a dry and barren technology landscape, even while the outside technology world grows in complexity and diversity. The key to falsifying the governance-diversity dichotomy is to decouple the cost of governance from the process of exploring new and innovative technologies.
Through built-in support for automation, extensibility, functional decomposition, and code-driven access control, serverless can help keep governance costs constant even while the overall system complexity grows. Code-driven governance is a paradigm shift for what is often the most conservative group within an organization, but it’s one that serverless makes easier.
Data-driven decision making & ongoing experimentation
Many times organizations have a two-tiered model of decision making, where complex analysis and the tools to support them are available for customer facing applications while backend systems languish in a decision making stone age, relying on HiPPO (highest paid person’s opinion) or anecdotes and stories about what people want. And some poor souls are working in a place where all decisions get made in these types of haphazard ways. These decision making processes could make sense in a simple or even complicated domain (see the cynefin framework), but they hold no muster for solving complex problems where patterns and causality are not clear and ultimately must be disentangled through probing and experimentation. If you don’t see the problems your teams face as complex that generally means you haven’t solved the simple or complicated ones yet, which itself can be a result of stagnating in a state of undifferentiated heavy lifting — stay there too long and you’ll find that your competitors have solved the complex problems for you.
By breaking systems down into small, decoupled services bounded by APIs, serverless builds a strong foundation for collecting usage information and meta data on a per-service or even per-function basis. By tying this information back to SLAs, KPIs, or other measures of business outcomes teams have the information they need to iterate and experiment — and the loose coupling of the underlying architecture gives them the space and freedom to do so.
Closing thoughts
As I mentioned at the start, none of the end states I’ve described above are new or novel — they are all well-documented best practices. You may not agree that all of these practices are relevant or valuable for you or your organization. That’s fine, I’m not trying to convince you of the validity of DevOps per se. My goal is to help you see an end state that lets your people spend more of their time solving your unique problems, and I believe that serverless is the most powerful catalyst out there for expediting that journey.
Look for future posts diving deeper into the topics above, as well as sussing out different adoption paths.
