The Twelve-Factor App, Abridged
Use declarative formats.
- API (“clean contract”) with the OS for portability
- Cloud deployable
- Minimize divergence between dev and prod, facilitate continuous integration (CI).
- Can be trivially scaled
One codebase tracked in revision control, many deploys
- One codebase per app
- Dependencies versioned and managed (e.g. Bundler, Yarn)
- Multiple deploys of the same app
Explicitly declare and isolate dependencies
- Never rely on implicit existence of system-wide packages. Declare all dependencies via a dependency declaration manifest.
- Use dependency isolation, e.g.
- One benefit of explicit dependency declaration is that it simplifies setup for developers new to the app.
- If the app needs to shell out to a system tool (e.g.
curl, that tool should be vendored into the app.
Store config in the environment
- config is everything that is likely to vary between deploys (staging, production, developer environments, etc).
- A litmus test for whether an app has all config correctly factored out of the code is whether the codebase could be made open source at any moment, without compromising any credentials.
- Another approach to config is the use of config files which are not checked into revision control, such as
config/database.ymlin Rails. This is a huge improvement over using constants which are checked into the code repo, but still has weaknesses: it’s easy to mistakenly check in a config file to the repo; there is a tendency for config files to be scattered about in different places and different formats, making it hard to see and manage all the config in one place. Further, these formats tend to be language- or framework-specific.
- The twelve-factor app stores config in environment variables (often shortened to env vars or env). Env vars are easy to change between deploys without changing any code; unlike config files, there is little chance of them being checked into the code repo accidentally; and unlike custom config files, or other config mechanisms such as Java System Properties, they are a language- and OS-agnostic standard.
- Another aspect of config management is grouping. Sometimes apps batch config into named groups (often called “environments”) named after specific deploys, such as the
productionenvironments in Rails. This method does not scale cleanly: as more deploys of the app are created, new environment names are necessary, such as
qa. As the project grows further, developers may add their own special environments like
joes-staging, resulting in a combinatorial explosion of config which makes managing deploys of the app very brittle.
- In a twelve-factor app, env vars are granular controls, each fully orthogonal to other env vars. They are never grouped together as “environments”, but instead are independently managed for each deploy. This is a model that scales up smoothly as the app naturally expands into more deploys over its lifetime.
My take on this would be to have the
dev environments for a Rails app, along with a
aws environment which encompasses all the “remote” environments where environment variables will be used. You could even consolidate
dev into a
local environment, and use a tool like
dotenv to set
IV. Backing services
Treat backing services as attached resources
- A backing service is any service the app consumes over the network as part of its normal operation. Examples include datastores (such as MySQL or CouchDB), messaging/queueing systems (such as RabbitMQ or Beanstalkd), SMTP services for outbound email (such as Postfix), and caching systems (such as Memcached).
- The code for a twelve-factor app makes no distinction between local and third party services.
- Each distinct backing service is a resource.
V. Build, release, run
Strictly separate build and run stages
A codebase is transformed into a (non-development) deploy through three stages:
- The build stage is a transform which converts a code repo into an executable bundle known as a build. Using a version of the code at a commit specified by the deployment process, the build stage fetches vendors dependencies and compiles binaries and assets.
- The release stage takes the build produced by the build stage and combines it with the deploy’s current config. The resulting release contains both the build and the config and is ready for immediate execution in the execution environment.
- The run stage (also known as “runtime”) runs the app in the execution environment, by launching some set of the app’s processes against a selected release.
- Every release should always have a unique release ID, such as a timestamp of the release (such as
2011-04-06-20:32:17) or an incrementing number (such as
v100). Releases are an append-only ledger and a release cannot be mutated once it is created. Any change must create a new release.
Execute the app as one or more stateless processes
- The app is executed in the execution environment as one or more processes.
- Twelve-factor processes are stateless and share-nothing. Any data that needs to persist must be stored in a stateful backing service, typically a database.
- The memory space or filesystem of the process can be used as a brief, single-transaction cache. For example, downloading a large file, operating on it, and storing the results of the operation in the database. The twelve-factor app never assumes that anything cached in memory or on disk will be available on a future request or job — with many processes of each type running, chances are high that a future request will be served by a different process.
- Asset packagers (such as Jammit or django-compressor) use the filesystem as a cache for compiled assets. A twelve-factor app prefers to do this compiling during the build stage, such as the Rails asset pipeline, rather than at runtime.
- Sticky sessions are a violation of twelve-factor and should never be used or relied upon.
VII. Port binding
Export services via port binding
- The twelve-factor app is completely self-contained and does not rely on runtime injection of a webserver into the execution environment to create a web-facing service. The web app exports HTTP as a service by binding to a port, and listening to requests coming in on that port.
- In a local development environment, the developer visits a service URL like
http://localhost:5000/to access the service exported by their app. In deployment, a routing layer handles routing requests from a public-facing hostname to the port-bound web processes.
- The port-binding approach means that one app can become the backing service for another app, by providing the URL to the backing app as a resource handle in the config for the consuming app.
Scale out via the process model
- In the twelve-factor app, processes are a first class citizen. Processes in the twelve-factor app take strong cues from the unix process model for running service daemons. Using this model, the developer can architect their app to handle diverse workloads by assigning each type of work to a process type. For example, HTTP requests may be handled by a web process, and long-running background tasks handled by a worker process.
- The share-nothing, horizontally partitionable nature of twelve-factor app processes means that adding more concurrency is a simple and reliable operation. The array of process types and number of processes of each type is known as the process formation.
- Twelve-factor app processes should never daemonize or write PID files. Instead, rely on the operating system’s process manager (such as Upstart, a distributed process manager on a cloud platform, or a tool like Foreman in development) to manage output streams, respond to crashed processes, and handle user-initiated restarts and shutdowns.
Maximize robustness with fast startup and graceful shutdown
- The twelve-factor app’s processes are disposable, meaning they can be started or stopped at a moment’s notice. This facilitates fast elastic scaling, rapid deployment of code or config changes, and robustness of production deploys.
- Processes should strive to minimize startup time.
- Processes shut down gracefully when they receive a SIGTERM signal from the process manager.
- Implicit in this model is that all jobs are reentrant, which typically is achieved by wrapping the results in a transaction, or making the operation idempotent.
- Processes should also be robust against sudden death, in the case of a failure in the underlying hardware. While this is a much less common occurrence than a graceful shutdown with
SIGTERM, it can still happen. A recommended approach is use of a robust queueing backend, such as Beanstalkd, that returns jobs to the queue when clients disconnect or time out. Either way, a twelve-factor app is architected to handle unexpected, non-graceful terminations. Crash-only design takes this concept to its logical conclusion.
X. Dev/prod parity
Keep development, staging, and production as similar as possible
- The twelve-factor developer resists the urge to use different backing services between development and production, even when adapters theoretically abstract away any differences in backing services.
- Alternatively, declarative provisioning tools such as Chef and Puppet combined with light-weight virtual environments such as Vagrant allow developers to run local environments which closely approximate production environments. The cost of installing and using these systems is low compared to the benefit of dev/prod parity and continuous deployment.
I comment: not so sure about this. I worked in an environment where a very savvy devops guy set up Docker environments locally and it was just too much of a pain to manage. After he left, people eventually switched back to just running the Rails app on the local file system. Divergent behavior (like file load order) was rare enough to be worth it.
- All deploys of the app (developer environments, staging, production) should be using the same type and version of each of the backing services.
Treat logs as event streams
- Logs are the stream of aggregated, time-ordered events collected from the output streams of all running processes and backing services.
- A twelve-factor app never concerns itself with routing or storage of its output stream. It should not attempt to write to or manage logfiles. Instead, each running process writes its event stream, unbuffered, to
stdout. During local development, the developer will view this stream in the foreground of their terminal to observe the app’s behavior.
- In staging or production deploys, each process’ stream will be captured by the execution environment, collated together with all other streams from the app, and routed to one or more final destinations for viewing and long-term archival. These archival destinations are not visible to or configurable by the app, and instead are completely managed by the execution environment. Open-source log routers (such as Logplex and Fluent) are available for this purpose.
- The stream can be sent to a log indexing and analysis system such as Splunk, or a general-purpose data warehousing system such as Hadoop/Hive. These systems allow for great power and flexibility for introspecting an app’s behavior over time, including:
- Finding specific events in the past.
- Large-scale graphing of trends (such as requests per minute).
- Active alerting according to user-defined heuristics (such as an alert when the quantity of errors per minute exceeds a certain threshold).
XII. Admin processes
Run admin/management tasks as one-off processes
- One-off admin processes should be run in an identical environment as the regular long-running processes of the app. They run against a release, using the same codebase and config as any process run against that release. Admin code must ship with application code to avoid synchronization issues.
- The same dependency isolation techniques should be used on all process types. For example, if the Ruby web process uses the command
bundle exec thin start, then a database migration should use
bundle exec rake db:migrate. Likewise, a Python program using Virtualenv should use the vendored
bin/pythonfor running both the Tornado webserver and any
- Twelve-factor strongly favors languages which provide a REPL shell out of the box, and which make it easy to run one-off scripts. In a local deploy, developers invoke one-off admin processes by a direct shell command inside the app’s checkout directory. In a production deploy, developers can use ssh or other remote command execution mechanism provided by that deploy’s execution environment to run such a process.