The Run Anywhere Mindset — A Discussion of Container Image Theory

Why Your Containers Should Be Built So they Can Run Anywhere the Underlying Container Engine Can

Published in

Capital One Tech

15 min readFeb 28, 2020

If you’ve read my previous articles about containers — The Whale in the Refrigerator and The Whale in the Grocery Store — you know I’m a container enthusiast (and I hope those articles helped educate and excite you about the potential that containers bring to computing and DevOps). In those articles I discussed what Docker is, went through some basic Docker commands, and then showed how to build your own Docker images.

This article will be a little different from those previous articles. Here I am going to discuss some of the theory behind building effective and useful container images that can easily be deployed to any environment — that is, they can literally “Run Anywhere.” This will include some of the recommended best practices from Docker themselves, as well as some learnings I have picked up in my experience using containers in numerous environments across companies both small and large. The article is based on my talk from DevOpsDays DFW 2018 which you can check out on YouTube.

A Quick Note on Language

As this is more of a theory discussion and not about Docker specifically, instead of referring to Docker Images and Docker Engines, I will use the more generic terms container (running instance), container image (basis for a container), and container engine (or container runtime, e.g. Docker). You can feel free to insert the word “Docker” here, but be aware that there are other container engines on the market beyond Docker and much of this theory could apply to those engines also. In fact, Docker has actually open-sourced their engine in the form of containerd, and if you are into containers, you should have some knowledge of containerd.

So, now that we have all of that out of the way, let’s get started.

Where is Anywhere?

The first question anybody should ask here is, “What do you mean by ‘Run Anywhere?’”

I literally mean that your containers should be built in such a way that they can run anywhere the underlying container engine can run. Essentially, containers with batteries included and the intelligence to configure themselves properly given a minimal set of parameters. This idea came from my work as a DevOps engineer. I was able to convince my team of the value of the idea that once built, our code should be able to literally run anywhere, and it became a powerful enabler for our team.

So, where is “Anywhere?” Anywhere, in this context, means any environment into which a running and useful container engine or container orchestrator can be deployed. This includes, but is not limited to, the following types of environments:

Local Workstation/Laptop (on Windows, MacOS, or Linux)
On-Premises Servers (like the ones under your desk)
Datacenter Servers
Virtual Machines (Local Workstation, On-Premises, or Cloud)
Orchestrated Clusters (Kubernetes, DC/OS, Docker Swarm)

Something to point out about all of those environments is that they are hardware, operating system, and cloud platform agnostic. If you can deploy a container engine to any of the above environments, you can run your containerized code.

Building a container image that can run in all of those different environments is not only possible but significantly easier than you might think. That said, achieving “run anywhere” definitely requires some planning. My goal here is to provide some guidelines and patterns to enable you to achieve this with your own container images.

Advantages of Anywhere

So now you’re saying, “Ok, that’s a neat idea, but why?” This is exactly the right question to ask here, so let’s take a moment and explore the “why” behind running anywhere.

Cost Savings

Let’s start with cost savings. Containers that can Run Anywhere have the potential to save an organization a substantial amount of money. These cost savings come in two areas, human time usage and compute usage.

Human Time Usage

The human side of your cost savings may not be immediately quantifiable because this will be more in time savings than anything else. For instance, if all of the applications in your ecosystem have implemented this Run Anywhere mindset, then your developers and DevOps engineers can spend more time writing new features and fixing bugs, because many of the bottlenecks that previously slowed them down have been mitigated.

Compute Usage

Compute usage savings is a different story. Moving to well built containers allows you to “densify” your application deployments; that is, you can run more than one application on a given server because those application processes are completely protected from one another via the container engine. The advantage here is that you can now avoid, or at least mitigate, “over-provisioning” of your Virtual Machines. You really begin to see this when you move to orchestrated clusters, where containers for several different applications will be running on any given server in the cluster, and unused nodes can be shut down automatically.

The Development Cycle

Now let’s talk about the Development portion of the software release cycle. As I said above, Run Anywhere theory states that any container image should be able to run on any container engine — this includes a developer’s workstation.

These days, software development is rarely done in a vacuum, and the software that is being developed is usually dependent on some other service. If those other services have been built with Run Anywhere in mind, then a developer should be able to pull the container images of those services and run them on a development workstation, perhaps even in some kind of “development” mode. The developer can then code, test, and iterate against those local services instead of having to deploy code into some kind of shared test environment. This saves the developer time as well as saving compute resources. Additionally, the developer has complete control over the versions of the services they are coding against. This enables quicker bug reporting and collaboration between teams, accelerating your development cycle.

Another advantage is ephemeral infrastructure. For example, let’s say you have chosen to develop an application with PostgreSQL as your backend database. In the past, developers would have either needed to install PostgreSQL on their local system and configure it (then uninstall it when they’re done developing, leaving a mess of configuration files) or ask somebody in the datacenter for a server running PostgreSQL. With Run Anywhere the developer can simply pull a PostgreSQL container and run it, then start developing against it. This pattern comes in handy in the QA and Testing cycle as well. Once done, deleting the container will remove the PostgreSQL database without having to clean up any lingering configuration, database, or log files.

In addition to development acceleration, there is a cost savings associated with not having to create cloud compute infrastructure or bare metal infrastructure to run, test, and iterate on code. This is just the beginning of the cost savings that Run Anywhere can bring, and we will explore this as we continue to discuss the advantages of this mindset.

QA and Integration

Once development is complete, code is pushed to some flavor of source code management system and some kind of “release” is created (think “tags” in git). If you have automated build systems (i.e. Continuous Integration) such as Jenkins, CircleCI, or Bamboo, the code will be pulled, built, and tested.

One of the basic tenets of DevOps practice and Continuous Integration/Continuous Deployment (CI/CD) is the idea that any artifact that passes all tests and that is successfully built should be deployable. Of course we all know that some of the artifacts we build should never ever be allowed near a production environment, but the basic idea here is sound. In that vein, a container image that has been built with the idea of Run Anywhere enables this.

Consider a generic CI/CD pipeline. The code is pulled, unit tests (and any other pre-build tests) are run, and if all tests pass, the code is built. At this point the code is usually deployed to some kind of QA environment for integration or validation testing. Since you built your container image with Run Anywhere in mind, with some on-the-fly configuration changes (or good service discovery) your code should “just work.” You’re not maintaining multiple configuration profiles because the code or the config is smart enough to figure out where it is and attach itself to the services it needs in the environment it has been launched in.

Also, if you need other back-end infrastructure for testing, it can be pulled and run in a similar manner to how the developer pulled and ran any necessary development dependencies. For example, if you are working with a PostgreSQL backend, you could pull that image and run it, then create and populate some test data into the database. Once done, you can run automated tests against the database. When you’re done, just destroy the container and you’ve cleaned up after yourself.

So now that you’re in QA/Integration you can run all of the validation tests you need to run, and if they all pass, your container image gets promoted to a release candidate.

Production

Your code just passed all its unit tests and all its integration tests and has been promoted to a release candidate. Assuming you have good tests (We all have good tests, right?) you are now poised for a fearless production deployment, and all using that exact same compiled code we started with. The value of this cannot be overstated! Well tested code that has survived in both the development and QA environments should give both developers and DevOps engineers strong confidence that their code is production-ready.

That said there always seem to be quirks in Production that don’t exist in lower environments for one reason or another, and usually that’s because of heightened security in Production. One thing that the Run Anywhere mindset allows for is keeping Production code as close to QA, Test, and/or Staging as possible. This keeps these environments as identical as possible, allowing issues to be discovered earlier in the development cycle and preventing high severity-incidents.

However, even with all of this in place, bugs can still make it through to Production. It happens. The Run Anywhere mindset prepares you for this eventuality because with a little bit of planning, it allows for dead-simple code rollbacks. If you’re using some form of versioning on your container images, then hopefully you’ve got backups of the last known working code. If you deploy a new release and it breaks things, reverting to the old code is as simple as killing the new container and starting up the old one (or re-pulling it if necessary).

Infrastructure

Another advantage of Run Anywhere is that it allows for the standardizing of server installs. If everything is in containers, then the only software necessary on the base server/instance is the container engine or orchestrator. If you deploy using something like Chef, Puppet, Ansible, AWS Cloudformation, Azure ARM Templates, or Hashicorp Terraform (or some combination of them) it means that you can minimize the number of configurations you need to maintain. Even in an Infrastructure as Code mindset, this can help additionally minimize human mistakes and ensure parity across servers and environments.

Achieving Run Anywhere

If you’re still reading, I’ll assume that I’ve sold you on the idea that a container image should be able to Run Anywhere. Now you should now be asking, “But how?” Let’s talk about how.

As everybody’s needs and environments are different, it’s difficult for me to list out a bunch of steps that you MUST follow. Instead I will be addressing some of the known hindrances to building robust and useful containers that are able to be Run Anywhere.

Containerize, Containerize, Containerize

If your application is not containerized, then you must first containerize it. That said, I would not suggest that you simply “lift, shift, and bottle” your app; that is, don’t just stuff it into a container image and say “done.” Take the time to make your application cloud and container aware. If you have the bandwidth, I highly recommend breaking up any monoliths into microservices, or at least into smaller chunks. Large container images are problematic because they take longer to start up, and also take longer to transfer over the network.

CI/CD

Before you write any code or fix any bugs, you should absolutely have an end-to-end Continuous Integration and Delivery pipeline for your application with automated testing. The advantages of CI/CD have been much lauded within the DevOps community so I don’t think it is necessary for me to spend much time telling you why you need this.

Instead I’ll mention some oft-overlooked pieces that you should consider adding to your CI/CD pipeline.

Create a scripted Pipeline. This “Pipeline as Code” can be checked into your Source Code management and the CI/CD platform can pull it dynamically.
Create Pipeline Libraries for often duplicated pipeline statements.
Remove any unnecessary steps, i.e. steps that add no value or have been deprecated because the code has moved on. This includes tests! Remove any tests that no longer provide value. It’s okay, you have my permission.
Add Automated Tests
Version every build (more on this below).
Parameterize your deployments, i.e. one deploy job should be able to deploy to any established environment (Deploy Anywhere!)
Remove as much friction from a non-production deployment as possible.
Add Deployment Metadata such as what versions of each component are deployed in a given environment (i.e. an Environment Manifest).

Pay Down Technical Debt

To kick your Run Anywhere journey into high gear, you must pay down your tech debt. Tech debt is like a millstone around your neck that will slow you down and frustrate your efforts to achieve Run Anywhere.

Here are some examples of technical debt that I have encountered. Your mileage may vary, but you should see the theme here of remedying certain bad habits.

Snapshot Builds

If you are using snapshot builds, you should migrate off of them and move to Dynamic Versioning so that every build has a unique version number. The advantage of this is it gives you a history of your builds and enables you to roll back quickly and easily in the case of a bad deployment — the value of which cannot be overstated. I have had great success using a combination of the build date and time stamp combined with the short git hash of the commit as a unique version. This approach gives you easily read breadcrumbs to the origin of a given build.

Update or Remove Dependencies

Take some time to update the sub-modules that your application is dependent on. This is important because your dependencies may have vulnerabilities that have been updated or that enable new features, some of which may help with your efforts to achieve Run Anywhere. Most of the time this will simply entail updating version numbers in your build automation system, e.g. package.json, pom.xml, or Pipfile. On occasion you will come across deprecated or replaced dependencies, which will require you to take the time to refactor any code that is using these modules.

If during your efforts you discover that you no longer need a certain dependency, remove it. There is no reason to pull code into your project that you no longer even use.

Add Health Checks

This is one of those little things that you really need to have in a container. Just a quick endpoint that can tell an external actor that this container is running normally and healthy. The most basic version of this is just an endpoint that returns a “200:OK”, but you should consider returning something a little more informative, like whether or not your container can reach its upstream datasources.

Externalizing your Configuration

The next step is to externalize your configuration. This will require a little planning, as you need to be able to dynamically configure the application from the environment, i.e. using environment variables or an external key/value store. You should start with environment variables, as there are no external dependencies on this feature. This will allow you to dynamically set any configuration parameters that would change based on environment, such as upstream data sources, logging levels, etc, and these are easily passed into a container. You will eventually want to support configuration via some kind of key/value store such as etcd or Hashicorp Consul, but this should not be a blocker in your journey.

Adding Service Discovery

This goes hand in hand with etcd, Consul, or perhaps even Netflix Eureka. If you have any of those tools available to you, use them to add automatic Service Discovery to your application.

In a nutshell, Service Discovery allows your application to automatically find and connect to the services that it depends on. This can be set up in such a way that if the application doesn’t find the environment variables it needs to configure itself, it searches for a service discovery service and configures itself from there.

Next Steps

You’ve containerized, you’ve paid down your tech debt, you have externalized your configuration, and your containers really can run anywhere. Now what?

Don’t Re-Architect (Right Away)

Now that your containers can run anywhere, you suddenly have lots of flexibility on how you can deploy them. So, now I’m going to tell you something you should not do. Do not change your production deployment architecture right away — especially if you’re new to containers. Even if you’re not new to containers, you’ve just added a layer of potential complexity to deploying your application. Instead, I recommend changing your underlying servers to support a container engine, and run your application in containers instead of directly on the OS (i.e. using docker run or docker-compose). Later, once you’re comfortable with containers and their quirks, and have built up the skillset to support containers, begin exploring orchestration engines.

Orchestration (When You’re Ready)

Once you’ve got your application deployed via containers, and you’re comfortable with the quirks of containers, your ultimate goal should be to move to an orchestrated container environment. Container Orchestrators are suites of software that manage your containers across a cluster of servers. Part of that management includes monitoring the health of the containers and automatically replacing any that have become unhealthy — that is how they become self-healing. The Orchestrator also handles load balancing, network ingress and egress, and will provide service discovery to your applications.

Deploying and managing a container orchestration cluster is not difficult, but it will require you to learn some new skills. That said, if you want to get a taste of container orchestration without worrying about administering a cluster, all of the major cloud providers have numerous hosted services that support different container orchestration technologies. These include AWS Fargate, AWS Elastic Kubernetes Service (EKS), Azure Kubernetes Service (AKS), and Google Kubernetes Engine (GKE). If none of these meet your needs, another option is to create and manage your own orchestration clusters via virtual machines. A non-Kubernetes based option is Mesosphere’s DC/OS, which provides similar functionality to Kubernetes.

I highly recommend that you do some research and choose the option that works best for your applications or perhaps spin up a few of these clusters and try them out.

Densification

I mentioned this in the “Cost Savings” section above, but this is where all that hard work pays off. Now that your applications are broken into smaller chunks and are running on orchestrated clusters, the cluster management software will place the running containers on various nodes in the cluster. This means that you could have any number of containers running on a given node, it’s the computer world’s version of a very complex game of Tetris!

Here are just a few of the tasks that the orchestration platform is continually doing:

Tracks where all containers are deployed within the cluster
Launches new containers as needed
Monitors for unhealthy nodes (servers), removes those nodes from the cluster and redeploys any containers that were deployed to those nodes
If the cluster is part of a cloud autoscaling group, those unhealthy nodes should be destroyed and recreated by the cloud platform
Monitors for unhealthy containers and redeploys them
Cleans up unhealthy containers after redeployment
Sets up and destroys network ingress and egress for each container as needed

The net outcome is that lots of containers run on fewer servers than would have been necessary if you had a 1:1 application to server ratio. That is to say, you have a more dense deployment of software per compute resource (i.e. server/instance), fewer net compute resources, and more efficient utilization of your compute resources. This is where the cost savings come from, and they can be substantial. The net outcome is that lots of containers run on fewer servers than would have been necessary if you had a 1:1 application to server ratio. That is to say, you have a more dense deployment of software per server, and that is where the cost savings come from.

If the idea that all these disparate application containers running on a single host could potentially interfere with each other has you concerned, worry not! This is one of the major advantages of containers, that each container is isolated from any other containers running on the same host. Additionally, the orchestration framework is smart enough not to overburden a given host by running too many containers on it.

Wrapping up

The purpose of this article was to give you a roadmap to creating well crafted containers that can literally Run Anywhere. In the process you will end up with better practices, less tech debt, fewer bugs in your applications, and hopefully a much easier deployment path.

Resources

DISCLOSURE STATEMENT: © 2020 Capital One. Opinions are those of the individual author. Unless noted otherwise in this post, Capital One is not affiliated with, nor endorsed by, any of the companies mentioned. All trademarks and other intellectual property used or displayed are property of their respective owners.