Stop Hammering Nails with a Screwdriver
Choosing the Right Provisioning Tool
You may have heard the saying “Choose the right tool for the job.” Sure, in a pinch you could pound a nail with a screwdriver or rock and it might get the job done. But you’d be no match for a skilled carpenter with a good hammer.
Choosing the right (or wrong) tools for the job can make a huge difference in your deployment times, frequency of outages, number of rollbacks, and risk profile. In some industries poorly built software may even endanger people’s lives. Not only are the wrong tools slower but they may produce unexpected results.
What’s in your toolbox?
Operations people love their tools. We rely on these tools to quickly and efficiently build and maintain our application infrastructure. We have tools for provisioning, testing, configuration, security, and monitoring. It’s easy to spot the DevOps people sporting a thick layer of stickers on their laptop.
Like that laptop covered in logos, there are so many different tools available to the modern IT shop. It’s hard to make sense of what they do and how they work (or don’t work) together. Engineers sometimes become very attached to particular tools, especially if they help relieve pain or solve a particular problem. Perhaps someone used a particular tool at their previous job and likes working with what they already know. Some companies even offer multi-function tools or platforms that hide all this complexity from the user.
What Outcomes Do You Want?
Take a step back and look at the big picture. Think about what outcomes you wish to achieve before deciding what tools to use. Be clear and concise, and write down your requirements. For example:
“We want to improve Linux server delivery time by 50%.”
“Developers should be able to create their own dev environments on demand, while remaining compliant with security policy.”
“Our application should be deployed in two public clouds within six months.”
Don’t get into lists of features and tools yet. Resist the temptation to immediately start debating the merits of various tools and platforms. Instead stick only to the high-level, specific outcomes that you wish to achieve. Once you have a clear idea of your goals, you can begin researching tools.
Which Tool for Provisioning?
In this blog post, we’ll focus on provisioning tools. Provisioning tools are used to build and configure OS or application infrastructure. Here are a handful of popular tools that you might be familiar with:
- Chef
- Puppet
- CFEngine
- Ansible
- AWS Cloudformation
- Azure Resource Manager
- Google Deployment Manager
- HashiCorp Terraform (disclaimer — I work here)
- Bourne again shell (Bash)
- Powershell
That’s just a sample of what’s available. There are platform tools that run on top of other tools, and some of these come in both open source and paid versions. Is it any wonder that people often throw up their hands and try to outsource the whole thing?
What is Provisioning?
In the technology world, provisioning generally means to provide some kind of service or infrastructure for humans and/or machines to utilize. This may or may not include the configuration of whatever was provisioned. For example, your systems administrator might build a Linux Virtual Machine and run some setup scripts on it. This is basic provisioning with a bit of config management.
To keep things clear we’ll use the following definitions.
Provisioning Tools: Build infrastructure and optionally prepare it for Day 1 operations.
Configuration Management Tools: Manage infrastructure or applications for Day N+1 operations.
Operations Tools: Manage application runtime settings, monitoring, security, reporting, etc.
Operations Tools
Let’s start with the Operations tool group. Scripting languages like Bash, Powershell, Python, Perl and Ruby all fit in this bucket. The benefits of scripting languages are that they can be used to write glue code that does anything from rotating log files, to sending alerts, stopping or restarting services, or configuring applications. The downside of scripting languages is that it’s easy to write code that nobody else can understand or use.
Can I use Powershell to provision virtual machines in the cloud? Yes, absolutely. But is it the best tool for the job? Probably not. Powershell and Bash can both be used to automate cloud provisioning, but you’ll end up with a bunch of scripts held together with duct tape and bailing wire.
Config Management Tools
Next, we’ll look at the family of Configuration Management tools. These are purpose-built tools that are designed to configure your operating system and application settings. Cfengine, Puppet, Chef and Ansible all fall into this category. Cfengine, Puppet, and Chef all share a similar feature set and philosophy about configuration management. Each of these tools runs an agent on every machine you wish to manage. This can be incredibly useful if you have to patch a server or continue to maintain it for months or years. The agent on the machine is autonomous, in other words each and every machine is responsible for keeping itself up to date. Typically the agent will run every 30 minutes or so and run through its checklist to ensure the machine is configured correctly.
Ansible is also used to configure servers but does not use an agent, instead, it relies on remote communication with each machine that needs to be configured. The benefit of Ansible is that it’s really easy to get started with and does not require any software to be installed on your target machines. The downside is that it requires remote access to every machine you want to manage, and machines are not able to repair themselves the way the other config tools do.
Provisioning Tools
Finally, there are the provisioning tools. These tools are designed to provision new infrastructure onto cloud or on-premise data centers. Provisioning tools include Terraform, AWS Cloudformation, Azure Resource Manager, or Google Cloud Deployment Manager.
The three big cloud vendors, AWS, Google, and Microsoft, each have their own provisioning tool. All of them work basically the same way. You create a set of template files in JSON or YAML format, feed the template into the cloud provider’s API, and your infrastructure gets built. Each tool is designed to work specifically with one cloud provider. The benefits of these tools are that they are supported by each cloud vendor, and generally work well if you follow the instructions. The downside of these very custom, specialized tools are that they are not portable. In other words, you can’t take your AWS Cloudformation template over to Azure and expect it to build anything. The cloud vendors each have their own API and native provisioning method.
Terraform is a multi-cloud provisioning tool that is easy to learn and easy to use. The Terraform language expresses your provisioning instructions as human-friendly, machine-readable code. Think of maybe a set of adjustable ratchet wrenches. Instead of having to carry around different tools for each cloud, you can simply use Terraform to provision onto all the different clouds. The benefits of Terraform are that it works on all the major public and private clouds, is easy to learn, and is open source software. The only downside is that it takes a few hours to learn how to use and configure it correctly. This is true of all the other tools as well!
Choose Your Tools and Learn to Use Them
You know your environment and employees better than anyone else. Do your homework, research the available tools, and understand the pros and cons of each. Ask your engineers, developers, and operations folks which tools they already know, try to understand where your bottlenecks and inefficient processes are, and create clearly defined outcomes. Try some tools out and see if they work for you. Ask other users of the tools how they like them. All these steps will help you choose the right tools for your infrastructure and application provisioning and configuration management. Soon you’ll be provisioning infrastructure like a pro with your handy tools.
Watch master carpenter Larry Haun drive the point home:
