The Case for Microservices and Docker Containers
Change is hard. To justify any changes in process or procedures the benefits have to outweigh the opportunity costs. Generally speaking we look at the tradeoffs and pick a route that seems most relevant for now and for the foreseeable future even if that means our processes change and what we are used to gets challenged. Sometimes that can cause frustration in many departments, not just development.
I’ll attempt to outline some of the benefits Imagine Learning has seen as we have moved some of our services from an Azure web app deployment to a Docker container .net core app on Kubernetes (k8s). The point is not to show how we configured our Kubernetes cluster, or how to create Docker containers, or how great .net Core is but rather to show the larger picture of why these technologies work to our benefit.
From there we’ll close with a few guidelines for building these better services.
Development Benefits
In no particular order let’s look at some of the immediate benefits that have been had by development which includes both engineers and members of a QA team.
Controllable environment — One of the difficult tasks for testers and developers alike is to duplicate the environment your service or application is running in. By creating a Docker container you have almost the exact same environment as will exist on the k8s cluster. All of the dependencies will have the same versions and if you need to debug something locally it is really easy to fire up a local copy of that Docker image and do some debugging. If you don’t know what Docker containers are you can think of them like lightweight VMs. The point is the environment for the application is the same locally or in a cloud deployment such as k8s.
Shorter time to production — The development department as a whole has one purpose, shipping software. If you aren’t shipping software often then you likely have lower job satisfaction and technical debt accumulates. Smaller services that fit into a larger ecosystem aid you in your quest to ship often. By breaking the monolith into pieces you can make large sweeping changes within a service quickly and not affect the rest of the overall application. If, for some reason, you need to use a different language, then do it! Since the service is smaller the build times become a fraction of what they once were. In our case we saw some builds go for 30+ minutes. Our microservices are built, tested and deployed within minutes of the pull request getting approved.
Clear separation of concerns — This is somewhat obvious, but separating data by their task also helps to use the optimal storage method for each type of service. For example, you may need a relational database for your rostering data but it wouldn’t be wise to store recordings or byte stream data into your database. Having separate services makes it clear what the service’s function is and provides additional flexibility. Need relationships? Use a relational database. Need ultra fast queries? Use a flat table such as Dynamo or Azure Table Storage.
Data accessibility — This isn’t tied to microservices specifically but while rewriting or breaking up the application it is an opportune time to consider data. Our new services have adopted a broadcast event methodology to accommodate communication between services when necessary or when anything interesting happens. While there is some extra complexity involved to make this happen you have separated yourself from having to directly talk to another service and therefore are immune from bugs or outages in other services. Having indirect dependencies again reinforces the ability to ship individual pieces of an application quickly. As another major benefit we now have data streams we can tap into if multiple services want to be informed of other changes or to train machine learning models. The point is that like good abstractions you can change the building blocks to accomplish tasks you haven’t yet thought of because the data is always available centrally.
Language/Technology flexibility— I always want to learn new technologies! One of my greatest fears has been to slowly become irrelevant because I didn’t take the time to learn new languages and tools. For example, if you are working in a large codebase, the chances of you making major upgrades to nuget packages or .net framework versions are slim because you don’t know what other issues you might encounter as part of that. Maybe worse, when you are forced to update a package it becomes a major headache because of all of the project interdependencies. Obviously, it’s best practice to upgrade versions for security patches and the like but sometimes we feel the urgency of the current feature or bug fix and quickly the application gets behind. The smaller the service is the easier it becomes to upgrade versions because there is less of a chance of unforeseen issues. Additionally, you have the ability to try out complete different languages when the occasion permits. Being a traditional .net c# shop it previously has been difficult to try out new languages. With containers running on the same k8s infrastructure you can easily say that anything that can be containerized we can run with the same deployment process on k8s.
Repeatable infrastructure and deployments — Infrastructure as code is desirable for all organizations today. Kubernetes enables this by making it easy to have a store of yaml files that describe your infrastructure from the ground up. We have actually completely destroyed and brought up our entire infrastructure in a matter of hours. We have the ability to have a pull request on every single infrastructure and code change which enables us to allow others to accomplish what they need to without fear of breaking things.
I should point out that getting comfortable with the yaml required for Kubernetes can be quite daunting. It’s take awhile to understand the options available and what everything means. To aid with that we use Kontemplate to generate yaml files that are suitable for Kubernetes and deployable via Spinnaker.
Business Benefits
Cheaper — Running in the cloud is expensive. As part of this overall change in process and development we are moving off of Windows servers to Linux servers. I know many of you are thinking that it’s about time but when you’ve been a traditional .net shop that change takes time and consideration. Not only will we save on the Windows license costs (Linux EC2 instances are cheaper than Windows EC2 instances) but we can achieve greater application density. Having highly available services implies you must have at least 2 services running at all times. That meant our Azure web apps had at least two instances and sometimes we mostly idle. With containers running on k8s we can run multiple services on the same node that would be idle and cut down the instance requirements without sacrificing reliability. I don’t have detailed numbers as to what those savings will be but I’d be surprised if we don’t save upwards of 30% of our cloud compute costs.
Opportunity for technology flexibility — One large concern for our company was being able to accomodate different backend technologies using the same CI/CD pipeline across the board. No matter the language or frameworks used we need to have high availability, reliability, security, etc. It is very possible to that this happens via acquisition where the company being acquired has a backend very separate than your own. Containers allow you to “lift and shift” pieces into your own infrastructure with relative ease. The complexity is “contained” in the Docker container. Then it is simply business as usual for deploying that container to production on k8s. Common deployments, shared costs, shared monitoring, all very possible.
When should services be split or how big should one service be?
Determining the boundaries for when a new service is required becomes a necessary task. There are a lot of opinions out there on when new services should be created. There are extremes on both side but generally speaking we have chosen to adopt a few guidelines for the size of our microservices.
- Logically grouped endpoints — If a service handles a Get request of some path, say GET /foo. That same service is expected to handle all of the HTTP verbs like PUT /foo/{id} or POST /foo. In other words a service per endpoint it taking things too far. Similar to good OOP, we want to make sure our endpoints make sense coexisting, otherwise, they should be two services. If I have a service that handles customer orders, that same service should NOT handle customer account information because they are separate concerns.
- Single Datastore — Having a single service talk to many datastores can be a sign that the service needs to be broken up. By having a single service with a single datastore the domain model becomes clearer and the service can be optimized for data access. As a potential exception to this rule a service may require access to file storage and then some kind of data storage for meta data on the files stored (think s3 files using dynamo for meta data). Since the data in the table is simply meta info for the files that can be acceptable to the service.
- Similar access pattern — This one might be less obvious, but if you have a service that contains apis that query a datastore and return a result with minimal transforms and that same service has apis that require intensive calculations or caching the service may need to be split.
