Microservice Dreamin’

Now where do I put all that complexity ?….

This is a prequel to a series of articles and tutorials in which we share our experiences building bots and deploying them within serverless applications.

It’s not hard to understand why anybody would want to build a bot but the role of serverless micro-services is a much less visible but no less important part of what we do. We’re a small team of data scientists & developers that punch well above our weight running a lot of production code for our clients and for ourselves. We don’t have years of devops experience behind us so this just wouldn’t be possible without serverless technology.

Let’s start with the ‘Wardley’ Why

Advertising exec turned motivational speaker Simon Sinek never stops telling us we should “Start with the Why”. I’ve watched his TED talk several times but can’t say I walked away with useful insights that I could actually use.

By contrast strategist & self-styled “Thought Lord & Mapper” Simon Wardley, believes that we do things because of two very different kinds of “Why”- Purpose and Movement. I’ve found this particular view of why very useful:

Most days I drive to a really nice coffee shop a few miles from where I live. My (why of) purpose for going there is that I like the coffee in that shop, my (why of) movement is that I drive because it’s too far to walk.

Purpose seldom changes. I’ll always like coffee from that shop but my why of movement can change frequently and even unexpectedly - if someone offers me a lift I’ll take it. If they open a branch in the next street, I’ll walk.

Like most people I tend to favour the things I’m familiar with, mostly because I’ve sunk days, months or possibly years into learning them. That doesn’t mean they’re the best tool for the problem I’m trying to solve right now. Being able to articulate the difference between purpose and movement has helped me understand when it’s time to start thinking about letting some things go.

What’s our Purpose ?

We want to build bots because humans communicate through language and not clicks, drags or drops. Our bots have to be affordable, available when people want to use them and give users the best possible experience. These ‘whys’ don’t change.

How do we achieve our Purpose ?

Bot technology is still in its early days and likely to evolve rapidly for some years. It would be very naive of us to build a monolithic app that needs major rework every time some improved technology, component or idea comes along. Our application needs to evolve rapidly whilst keeping the codebase easy to maintain.

As many experienced developers will tell you, ‘Rapid evolution’ & ‘maintainable codebase’ in the same sentence is the kind of wishful thinking that’s more likely to accelerate your codebase down the road to Hell than prevent it from happening.

I can’t say they’re wrong but say that it doesn’t have to be that way. Some years ago I was lucky enough to work for a company that understood how hard it is to make code easy and how easy it is to make code hard. They were great developers & early pioneers of micro-services. All of their codebases were small and embarrassingly easier to work on than anything I’d experienced before. If a codebase got ugly they’d just throw it away and build a new one.

Since that time I’ve tried to emulate this way of doing things elsewhere but with limited success because I hadn’t really appreciated that micro-services don’t actually make things simpler, they just move the complexity out of the code and into the architecture. Sure I could make my code simpler by moving it into small services but joining those services together in a production ready way was complex and sometimes downright scary.

Despite all this, my experience all those years ago was so positive that I never gave up on my micro-service dream…

Microservices for the rest of us…

London 2019. I was lucky enough to attend the μCon micro-service convention along with a small army of fellow dreamers and one or two journeymen. I was very late and the meet & greet hall was empty (I hadn’t long moved to a town about 80 miles outside the M25 and was still grappling with how long it takes to get to Waterloo station, catch a tube then walk somewhere). A gathering of conference volunteers looked at me disapprovingly so I ducked into the nearest talk— one I wouldn’t naturally have chosen - Preparing for a Future Microservices Journey by serverless architect Susanne Kaiser.

I wouldn’t naturally have chosen a talk on serverless because I considered it to be an immature technology limited to building APIs (yes I hadn’t though about things like React). It was also because I’d spent the previous 8 months of my life learning Terraform, Kubernetes, Kafka & AMQP and already had what I thought was a plan to break up the monolith .

The speaker started by showing us a micro-service plan that, initially, looked suspiciously similar to many of my own:

I tend to think in circles not hexagons so typically my micro-service planning would evolve into something like this:

As you can see my architecture has a whopping six services. The example in the talk crystallised around just three but over the course of a few slides it evolved to look something like this:

That’s a scary amount of moving parts for just three services. I knew enough about Terraform, Kubernetes, VPCs and all of the other things needed to bolt my code together to know this was right, there’s a LOT more to micro-service architectures than just the code-lite services themselves.

The talk moved on to imagine the architecture for a web application that could be used for buying tickets and booking talks at the conference we were at. This time the architecture was expressed as a Wardley Map (remember Thought Lord & Mapper, Simon Wardley ?). This map had the value-chain as the vertical axis.

If you’re not familiar with value chains they’re the interconnected business activities that deliver value to a customer:

1. Customer needs a shop, shop needs a coffee machine, coffee machine needs electrical connections, electrical connections need a supplier etc.
2. Customer needs coffee, coffee needs a coffee machine…
3. Customer needs coffee, coffee needs a cup, cup needs cleaning equipment, cleaning equipment needs electricity.

Most businesses comprise of many chains which themselves will interconnect but they all start with what the customer needs.

The map developed further into a more complex set of services than our three above:

The point of having value chain visibility on the vertical access is to identify what parts of our system are visible to the customer. Customers will only value what they can see so the less visible the component in your system is the less point there is in spending time and money rolling your own.

When I’m in the coffee shop anything not visible (or tasteable) to me whilst I’m enjoying coffee in a nice environment should be serviced by whatever does it quickest/best/cheapest.

Similarly customers don’t care whether servers are real or virtual. It doesn’t matter whether the application is written in PHP, Django or Rails. The size of the devops team means nothing to them, nor does the operating system, nor the database. If the customer doesn’t perceive something as value then it’s a cost and as technology evolves things that drive cost are destined for obsolescence . Always be prepared to put these things down when the times comes.

With serverless architecture this scary complexity can now be safely moved out of the code and given to Amazon, Google or Azure as a Commodity (see the diagram above). They know how to manage these things a lot better and a lot cheaper than I can. I get all the developer benefits of micro-service development without the need for recruiting an elite force of devops engineers on a Netflix budget.

Here’s the video of Susanne Kaiser’s talk. It’s on the skillsmatter website so you may have to sign-up (it’s free). The slides from her talk are here and I have shamelessly borrowed a couple for this article.

I’d not come across Wardley Mapping before but I liked what it had done so I attended his keynote talk ‘Crossing the river by feeling the stones’ the next day. A highly entertaining tech talk that I would thoroughly recommend to anyone. I’ve watched it more than once…

Cutting a long story short

To get us up to speed and avoid another 8 months of study I managed to persuade the company I was working for at the time to pay for an in-house workshop by the AWS Serverless Hero Yan Cui - yes that’s a real title but don’t let it put you off Yan is super helpful and working with him is a great experience.

The Internet loves lists so I’ll stop narrating and list some of the benefits of bots & serverless micro-services that we’ve found on our journey.

Bots for Best User Experience: We want the systems we build to act like people not force people to act like systems.

Micro-services for Rapid Evolution whilst keeping the code Maintainable: forget speed or vendor lock-in this is what kills 99% of applications and/or makes them a misery to work on. Anything that can mitigate the effects of technical debt is always worth it.

Micro-services for robustness: services that communicate by message queues are easily built so that they don’t lose data when they go wrong. Work just builds up in the queue until the problem is fixed.

Serverless for Speed to Market: I love Terraform & Kubernetes but it would still take days to build and deploy even a simple Hello World app (weeks if we were setting up Kubernetes from scratch) plus all the costs attached. I deployed a simple, production-ready app within 2 hours of starting my first serverless course.

Serverless for low running Costs: how many staging and production servers are sitting there waiting for someone to use them ? Now we pay only for what we use which is a lot less.

Serverless for low maintenance costs: Even when servers are doing nothing someone needs to keep them spinning. AWS does that for us now and I get to sleep all night.

Serverless for rapid scalability: Traffic Spikes that would normally bring down our single site barely register across the entire AWS, Google or Azure stacks and our code will stay running.

Serverless for high availability: Our code runs across multiple data-centres out of the box.

Serverless for smart refactoring: We know how much each component in our code is costing us so it’s easy to justify why something needs refactoring.

Vendor lock-in…

One thing I certainly have learned about serverless is that you can’t mention the word without having some angry developer start flaming ‘what about vendor lock-in ?’ at you.

I’m not saying it’s not a problem but based on my 14 years of startup development using cloud platforms I see it like this:

1. I have never worked-for or heard of a company that switched cloud provider all at once. I’m sure they’re out there but I haven’t come across one yet.
2. Typically cloud provider costs are less than 3% of a development team salary so anything that helps developers get valuable things done quickly will always be cheap even if it gets more expensive.
3. Serverless is already a highly competitive business and that will only continue. AWS, Google, Azure & many others will be competing hard for your cash by offering the most cost-effective services. The notion that one of them will hike up the prices unnecessarily once they have enough people locked-in feels more conspiracy theory than credible outcome.
4. The whole premise of microservices is that “nothing should be so big you can’t afford to throw it away”. To paraphrase the safety demonstration given by most aircraft cabin crew “In the unlikely event that you should wish to switch cloud provider…” a well architected set of services with clearly defined contexts should be easy to replicate on other platforms. I’ll be covering things later in this tutorial that can help you do that.

Here are some real numbers to play with. Not so long ago I worked with a mixed dev team of 26 in-house and remote devs. We were spending £7000/month on AWS and £210,000/month on developer salaries.

If we assume an outrageous hike in AWS charges of 50% and a very conservative estimate of the impact of technical debt at just 10% (it’s hard to measure but the reality was that across the team we were committing code was actually 20% less often than the previous year) that would be £3500/month from outrageous vendor lock in and £21,000/month from conservative technical debt & complication — not counting the business fixed-cost or the loss of opportunity cost by new features not being shipped on time.

To finish off, micro-services are always worth pursuing because they will keep make your application robust, keep you out of technical debt jail and enable you to keep on shipping things to market long after your startup is no longer a startup. If you want to run your own services along with the multiple message queues, load balancers, databases, load balancers et al this is far from a trivial task and will need significant investment in the kind of people and systems able to take on that complexity and leave your developers with small, clean & disposable codebases.

If you don’t have that kind of money and backing from the business then serverless micro-services have all the benefits without the complexity and commitment.

Before I leave you, this article is the opening piece of a series that will culminate in some free online workshops where we build quite a funky event-driven serverless application that will drive a chatbot in Rasa. You can find out more about this course at brains-in-jar.org.

Thank you for reading & feel free to reach out, Steve Creedon.