Keep your Microservices Clean Using the Twelve-Factor App Principles

Let’s imagine for a moment that you have the responsibility of contributing to the architecture of a new Software-as-a-Service (SaaS) using microservices. In order to keep your sanity over what will become an ocean of services, what are the first steps to consider?

The goal of this article is to outline a list of principles that will help put the first pieces together, while ensuring the application you are currently building remains scalable and resilient. These principles are applicable to multiple technologies because they form a methodology, not a technical direction on how to build your platform.

Let’s be honest, an architecture based on microservices adds its share of complexity to how things are developed. Often, it feels like going through the entire creation process of a new microservice is too labor-intensive for what we try to accomplish. Why not just add the feature to this existing service? After all, the entities are in the same domain.

On the other hand, since you are working with loosely coupled services, not respecting a minimum of standardization will most likely lead to fundamental problems in the future… and these problems can be hard to find a solution down the road.

0. Meet the twelve-factor

Today, I suggest you use the twelve-factor principles as a baseline in your architecture decisions. The contributors to this methodology listed the problems they saw during their experience and they came up with a proposal of 12 best practices that:

• Automate environment setup by using a declarative configuration
• Ensure a good portability between the execution environments (staging, production)
• Help maintain a good level of collaboration for teams that need to deploy on modern cloud platforms
• Minimizes divergence between development environments and production
• Make sure that the scale up of the application can happen without changing the processes or the tools used
• Most importantly, the rules help avoid the cost of software erosion

Software erosion is significant. As software development professionals, how many times have we seen an application that needs to be re-written because it aged poorly? With potential enormous costs associated with this, the ability to avoid software erosion can be a game changer for companies that own software.

In some cases, this migration does not have to be a one-off. There is a chance that functionalities can be peeled-off from a monolith and that smaller services can be created to make it easier to test, deploy, and support once it is in production. Following the principles in this migration can significantly help to avoid similar pitfalls later on.

1. One codebase, many deploys

Multiple apps sharing the same code in the source control is a violation of the twelve-factor. So for instance, instead of having your common code sitting in the same repository as your service, it should become a shared library that can be included in the dependency manager.

For instance, instead of keeping code for payments and finance together because they are closely related with common code, a library should be created to share common code and each service should be in its own repository:

Git repositories

This rule may look basic, but it forces a shift in the way that we design what we develop.

An app should consist of only one repository. If there are multiple codebases, it becomes a distributed system, it is not an app anymore. Each component in this distributed system becomes an app, and every one of them should comply with the twelve-factor.

By following this, you ensure that the same codebase can be deployed on any environment, either staging, production, or custom deployments made by developers.

2. Explicitly declare and isolate dependencies

In the previous rule, a dependency manager was implied to help us manage our shared libraries and our external dependencies. For most developers, this tool is already part of their day to day activities. For Node.js, the “package.json” file holds this information. Ruby uses a Gemfile, Java supports Gradle. No matter which technology you are using, every dependency should be managed by an entry in the dependency management tool.

Because dependency isolation is implicitly covered by this tool, using it for every deployment ensures that you are less likely to have implicit dependencies “leaking in” from another layer of your architecture. In other words, the version of libraries you will push will always be the same during development and in production.

This is very important to be able to diagnose a problem in production happening at 3 AM. Using few instructions, the person in charge should be able to have a reproducible environment on which they can investigate and not have to guess the state of the server in production.

The deepest root cause that may be hidden in one of the dependencies can be quickly tracked down because the list of dependencies will have changed and the situation will be resolved faster.

3. Store configuration in the environment

This rule is critical for security purposes. Otherwise, you end up having credentials hardcoded or stored in the codebase. Any security breach to your repository will potentially open the door to vulnerabilities in sections of your application.

Twelve-factor requires a strict separation of configuration from code. Developers should have a trivial way of deploying a new environment to build new features. By adopting the environment variables practice, you make sure that the configuration can easily be modified and that each environment can be created without having to add lines or files in the codebase.

Environment configuration is everything that will vary between deployments — such as staging, production, developer, etc. Typically, you will find:

• Resource handles to databases or other backing services
• API keys
• Application-specific information like hostnames, ports, etc.
• Channels used for alerts and monitoring
• Feature flags

By storing this information outside the codebase, the responsibility of holding the configuration can be shared by multiple services. Less sensible values can be stored inside a configuration management tool that will let other developers inherit from default values to get up and running quickly.

In order to ensure security, the credentials can be stored in a secrets management platform. Just like the less sensible configuration, developers can still use these required secured information without having the chance to visualize it and store it in a non-secure way.

It may look like a lot of complexity is added for something as simple as managing configuration, but in the long term, you can be sure that your credentials are in a safe zone and that configuration can be changed in a centralized location without having to investigate all your code repositories to see in which files you need to apply your modifications.

4. Treat backing services as attached resources

A backing service is any service that is consumed over the network by the application. The most frequently used services are:

• Databases
• Messaging or queueing systems
• Outbound emailing system
• Caching systems

They are usually provided by third parties, and it can be hard to ensure a good availability of an application with little visibility over these underlying systems.

By treating those resources as attached, they become easily interchangeable. Switching service should be as easy as changing the environment variable and restarting the instances. In case of database problems, it becomes trivial to restore the latest backup and update the configuration value to point to the new healthy instance.

Additionally, with the latest container tools available, starting and running a multi-container project is a trivial task. Most of the services used in web and application development are now available as containers, or alternative solutions can be used. It only takes a bit of configuration and local environments can be very close to what can be found in production.

With services separated from the application, it helps being more resilient to problems and adapting to changes. For instance, with a number of users growing every day, adding caching, load balancing or proxies might become a necessity. Instead of changing the infrastructure manually and taking risks, it is now possible to simply treat this new infrastructure as new containers. They will be installed in production and optionally in development environments.

5. Build, release, run

Every change that is submitted by developers should initiate a build process, which is composed of three stages:

• Build stage: through this stage, the code repository becomes a deployable asset. For a specific commit, the build stage will fetch the third party dependencies, bundle everything together, and possibly minify it.
• Release stage: the resulting asset of the previous step is taken and combined with the deployment’s current configuration to produce a release. At this stage, the package is being pushed in an execution environment.
• Run stage (or runtime): this stage can be as simple as starting an executable, but can also include a set of processes to launch at startup.

The first two stages will usually be run on a continuous integration and deployment (CI/CD) platform, but the run stage happens on each service deployed on an instance.

In this regard, efforts should be made to minimize the number of steps in the runtime. The two other stages (build and release) are used to validate and prepare the executable, they should contain all the necessary steps to ensure a good quality level and to make sure that the result package is working and ready to ship.

However, the runtime execution will occur once the release is complete in the environment. With today’s cloud platforms, this stage is also executed when the server is restarted, or if a crashed process is currently being replaced by the process manager. For this reason, it is very important to keep a low level of complexity at this moment. Most of the time, if your process encounters an exceptional situation, a limited number of developers are available. It’s clearly not a good idea to complexify the investigation.

Since the two first stages are initiated by the developers, they can be more complex. If errors happen, the developer driving the deployment will either be able to address them or find help to understand what is happening. They can also be longer if you are using lengthy automated testing to make sure that the version being pushed respects the quality standards.

6. Execute the app as one or more stateless processes

Twelve-factor apps are stateless and should not contain any persisted information. If there is a requirement to persist data, it should be done on a backing service, such as a database.

By respecting this rule, every packet of information transferred by the application can be understood in isolation. No process can write on the disk or in the memory for session data, for instance. If this is really required, it must be communicated in a way that no previously transferred information is required to process the packets.

Also, using a load balancer, the user session will most likely happen on multiple instances. Since the modern technologies are defining more dynamic execution environments, there are multiple cases that can cause the data in memory to be lost.

An interesting alternative is to use in-memory data storage systems, treating them as backing services and adding monitoring on them to offer visibility. It offers a quick database that is made for short lived information such as what can be found in session data.

7. Export services via port binding

A twelve-factor app is defined as self-contained, meaning that it does not rely on an application server to get started. For instance, a Node application does not need another software to start serving web pages. If a web application is declared, it can start a listener on a specified port by itself and serve pages autonomously.

A side effect is that for every microservice part of a system, there will be a port number that can be the same or different and it can even change in the future. When digging the source code to go get a port number that is required to integrate with another microservice, it should be a sign that there is a problem.

It’s specifically this problem that port binding is addressing.

To properly export this service, a proxy service such as an API gateway can be used to match paths to the internal port exposed by the microservice. For instance, if /api/users is the route requested, it could be bound to the port 32764 of a microservice.

By delegating this responsibility to a gateway, configuration can be added to use features such as token-based authentication, rate limiting and a fine grained access control that make it possible to assign more precise permissions to users. In the case of a changing port number, only the mapping found in this configuration needs to be changed.

8. Scale out via the process model

This rule makes an immense difference when the time comes to scale out. By having stateless and self-contained microservices, adding more concurrency to the infrastructure is a trivial and reliable operation. It becomes not only possible to quickly spawn new instances, but also to automate it based on configured thresholds.

For this reason, the scaling process must be implemented according to the UNIX process model.

Using this model, an application architecture is able to handle diverse workloads by using different process types. For example, the HTTP requests of the application may be handled by web processes. If processes are long running background tasks, they may be managed by a worker process.

By separating processes by type, the scaling operation is more granular and efficient. The web application might be very lightweight if all the long running processes are removed from it. It becomes possible to only have two web processes and six worker processes, for example. The list of these processes is called the process formation.

A completely separate process should then take care of changing the number of processes required per microservices and have a subprocess that will add or remove instances to match this number.

Containerized services in conjunction with a cluster solution is the best example of a tool that can manage a process formation. For instance, Kubernetes rely on a Horizontal Pod Autoscaler in order to automatically add or remove pods according to thresholds configurable by microservice.

9. Maximize robustness with fast startup and graceful shutdown

Elastic scaling and rapid deployment also means that processes will start or stop at any moment. To make this happen, the twelve-factor app must provide processes that are disposable and reliable.

That is why efforts should be made to minimize startup time as much as possible. By having small release packages and by limiting the processes run at boot, the deployment and the scaling operation benefits from more agility. It is faster to spawn new instances or simply move processes from physical machines if needed.

The other side of the process lifecycle, which is the shutdown of the process, can also be addressed by supporting graceful shutdowns. Doing so is making sure that on a SIGTERM signal, the interface stops taking new requests, completes the ones in process, and then exits when done. By ensuring the completion of execution before stopping, there are less chances of corruption.

This rule impacts directly the resilience of the application by allowing the scaling operations to act based on a reliable status of the number of processes running. No process is hanging, occupying a place in the formation without being able to take requests. If any problem occurs, the process manager can call a termination of the problematic process and start a new one.

10. Keep development, staging, and production as similar as possible

According to the twelve-factor, when the time comes to address discrepancies between development environment and production, there are three different areas in which gaps can be found:

• Time gap: the features that have been developed (or are still being developed) may take days, or even weeks to go into production.
• Personnel gap: Different people are involved in the development. Software engineers will produce the code and the ops engineers will deploy it.
• Tools gap: Development tools and server versions may differ from what is currently in production.

This rule of the methodology addresses these gaps by suggesting that the toolset and the people stays the same between development and deployment. Since the developers are responsible for pushing their changes in production, an increased attention is given by making sure proper monitoring is available and by following up when the change is deployed.

With the possibility of facing problems when deploying in production, the development time may be impacted by adding these additional responsibilities to the developers. In the long term, this time is greatly optimized because everyone on the team will give themselves tools to have better visibility and to eliminate any problems that may occur. The result will be a faster investigation on incidents and alerts being triggered sooner, way before the application could suffer.

For organizations which use a team of system administrators to push new versions in production, this is certainly an important shift in the mindset of developers. However, deployment and build processes are quickly automatized, and the time gap ends up being smaller because people trust the code being deployed.

11. Treat logs as event streams

An application that provides visibility through logging is adopting an event stream behavior to send events or information related to the runtime. In other words, a log file is simply an output format of this event stream.

A twelve-factor app will never have the responsibility of routing or storing this output stream. The logging output should simply be streamed to the stdout output stream. During the development, the terminal output can be used to observe the app running, and an aggregator can be configured for the hosted environments (staging, production) to store and process the streams.

Every event stream coming from any instance is monitored by a centralized solution to store it in a way that investigators can have an overall view of the platform based on a specific time. It becomes obsolete to consult logs on servers, and the added flexibility of introspecting the app make it possible to:

• Find specific events in the past.
• Large-scale graphing — for instance to monitor the requests per minute.
• Monitor and alert based on specific thresholds.

These streams can produce a lot of data so taking the right decision when dealing with the persistence can make a big difference. Treating this information using big data practices (like ETL) gives an advantage because it becomes trivial to produce dashboards to monitor specific metrics, or even use machine learning to detect intrusions or behaviors that could lead to problems.

12. Run admin/management tasks as one-off processes

Often, web applications don’t rely only on running web and worker processes to satisfy the requests from the users. It happens often to have a one-off task to apply a change to the database schema, or a script that needs to be executed to fix bad data.

For a twelve-factor app, these one-off processes need to be run in an identical environment as the regular long-running server processes. These scripts should be deployed with a release, using the same codebase and configuration as any other process run against this release.

Since the developer is the owner of the whole workflow for developing a feature, the admin processes part of their implementation needed to run automatically in any required way (one-off at startup or scheduled), with proper visibility to assist if ever a problem would happen.

Languages that come with a REPL shell ease the development of such tasks. This type of console interaction starts by reading a user input, then executes the implemented process and finally prints the result to the user. These steps form a loop that anyone that used the command line console to interact with a machine knows.

If the execution environment is taken into consideration, it becomes possible to run a shell script inside an application directory on local or development deployments to interact with the process. In the case of a production deployment, this interaction can be minimized to focus only on the required steps.

Conclusion

As initially mentioned, without providing a step by step guide on how to build a microservice architecture, this set of rules helps you make sure that you don’t go crazy trying to maintain a distributed system composed of many microservices.

It offers a checklist that you can easily add in the codebase with your services and perform incremental migrations as part of a wider initiative to respect the twelve-factor. Trust will most likely be gained from the developers for every service that goes through this migration because they will make sure that the services are resilient and they can rely on metric dashboards to prove it.

To learn more about the twelve-factor app methodology, I invite you to visit its website.

Editorial reviews by Deanna Chow, Liela Touré, and Mario Bittencourt.

Want to work with us? Click here to see all open positions at SSENSE!