Playing with chef, aws, vagrant and test-kitchen

Problem

If you are manually setting up your servers then you know manual configuration is tedious and it takes time. Even if you document what you did, it’s still very easy to miss a step the next time you try to replicate that same environment.

Solution

Infrastructure as code! There are a few tools out there that you can use to achieve this but we will use chef in this example. Chef uses Ruby as its reference language to define the patterns that are found in resources, recipes, and cookbooks.

Implementation

There are a few gems that we need to install, so let’s create a Gemfile and add chef then bundle install

Here is a simple script to test chef on our local computer. When we run the script it will create a file in our tmp dir and write “hello world” in that file.

To execute this on the local computer, run

$ chef-client --local-mode hello.rb

Verify the file is created and the content is written, then you can change the argument we are passing to the content method and rerun the command. You will see that chef will make the update.

In reality we would be doing more that just creating a file, and so we probably don’t want to test it on our local computer. Instead, install virtual box if you don’t have it on your computer. You will also need to install vagrant. Vagrant is a command line utility for managing the lifecycle of virtual machines. Basically it will make it easier to interact with virtual box. Run the following to see how easy it is to use virtual box via vagrant.

$ vagrant init hashicorp/precise64
$ vagrant up

Let’s add test-kitchen and kitchen-vagrant to our gem list so that we can run our chef script on our virtual box. Test Kitchen is an integration tool for developing and testing infrastructure code and software on isolated target platforms, and kitchen-vagrant is a test-kitchen driver for vagrant. Your Gemfile should look something like

Then run…

$ kitchen init

A .kitchen.yml file will be created in your project base directory. This file describes your testing configuration; what you want to test and on which target platforms.

Let’s do something more than just creating a file and writing to it, here is what my directory structure looks like

nginx.conf looks like

It’s a simple nginx configuration that returns “my app is awesome” when you make a GET request to the index.html page.

default.rb is the script that builds my infrastructure and it looks like

You can see that it installs two packages (nginx and curl), replaces nginx’s default config file for my nginx.conf file and starts the nginx service.

.kitchen.yml looks like

It describes my driver (vagrant), my provisioner (chef_solo) and recipe I want to run (my_App).

$ kitchen converge

Once it’s done, you can login to your virtual box by

$ kitchen login

Looks like my app is installed and nginx is returning the right content! Okay, time to push my app for the whole world to see. We need to add ec2 driver to our Gemlist.

…then update .kitchen.yml

Make sure you have set AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in your environment for your aws account before you run the script. Also notice I have added a security group (launch-wizard-1) that I have configured to open up port 80 (for my app) and 22 (for ssh).

After that you are all set, when you run kitchen converge, it will create a new ec2 instance for you and you will be able to access the app. Notice that not only have we automated our process but also because our infrastructure is described in code, we can actually test it just like we would test any other code.