Setting up for infrastructure development

Local MacOSX environment for development and testing with Test Kitchen and Vagrant.

goncalo pereira
Jun 2, 2014 · 6 min read

Introduction

In this tutorial you’ll see how to set up the local environment with KitchenCI using Vagrant as a driver, Chef-Solo as a provider andServerspec for integration tests under Mac OS X.

The idea behind this is that KitchenCI is flexible enough to add any types of tests (bash, rspec, etc) and Chef-Solo allows to try local changes without submitting to the source code repository or to the Chef Server.

KitchenCI also allows to pick other drivers like AWS. By testing the infrastructure under both Vagrant and AWS we can weed out configuration issues between the two types of instances instead of leaving the debugging for when CloudFormation is being developed.

Having a way to run new configurations and tests locally allows us to get feedback faster without having to push Chef changes to the CI environment (which might break the shared environments).

It also merges local configurations with all over environments making it visible to propagate changes and allowing the laptops to become just another test environment which results are reproducible.

Sometimes using this apparently more convoluted stack makes it more painful to develop locally if the configuration management is not healthy enough but this is only pushing forward all issues that would be apparent a few environments up. It is better to deal with broken instances locally than on other environments.

Stacks

The previous stack was made up of a mix of local installations and local vagrant configurations depending on each laptop, this was followed by an initial CI environment that would have its own chef instances over AWS using CloudFormation.

The new stack is made up of:

Local (laptop) that runs KitchenCI with Vagrant/Virtualbox — This allows to write tests, configure instances and be used to develop against.

CI agent that runs KitchenCI with Vagrant/Virtualbox — This allows to ensure a new local build will work and be tested and any commits are tested.

CI agent that runs KitchenCI with AWS- This is a copy of the previous one with EC2 instances. This allows us to find issues between Vagrant and AWS. Not used for application development.

AWS Development for CloudFormation — After ensuring the individual instances run we try the same environment with CloudFormation. This is for Cloudformation development, not used for application development.

AWS Development Integration — Now that we are certain that both AWS instances and Cloudformation are fully tested we can use a ‘releasable’ copy of it to bring up the Integration environment used for development.

AWS Development UAT, QA, Pre-Production, Production — Other environments are variations of the AWS Development Integration environment.

Mac OS X as a development environment

To set up all packages we’re currently using Homebrew — This managements installation of the most common tools and manages updates for those similar to Linux package managers Yum, Apt.

It is advised to have XCode command line installed and up to date to run Homebrew and other applications — not doing this might give you compilation errors and other problems in the future.

Some older versions of Mac OS X might not be able to run the following command in that case you’ll need to get it through Apple’s Development Portal.

To install Homebrew just run.

Adding the current configuration development repository

You’ll need to have git installed. If you don’t you can use Homebrew to install it.

Setting up a local Ruby environment

KitchenCI and Chef run with Ruby — The following are some best practices to ensure you have the same Ruby and Gem (Ruby’s libraries) versions as your Chef Clients. This is done by using RbEnv to manage Ruby versions and Bundler to install Gems locally so any other Ruby applications or updates do not corrupt this setup.

Thanks to Homebrew we can install RBenv easily. Ruby build is used to extend rbenv’s features around installing and removing Ruby versions.

You can have a look on the local available ruby versions with.

We can install more versions of Ruby with the install command. Bundler will have a header for the ruby version on its Gemfile, if the local version is not correct it will not run. In this case we’ll install Ruby 2.0.0.

Using the local command we enforce that the current directory always runs the picked ruby version. Rehash updates RbEnv.

After that step we’ll need to install Bundler. This will be done on the system level Gem library but after this step we’ll be able to use it to manage Gems locally.

Any Ruby application will use the file Gemfile to let you know about dependencies. Gemfile.lock will tell you what versions to install or if it does not exist it will be created when Bundler has a valid run.

You can run Bundler with the following command. Which will install the required Gems on the vendor directory

Due to some pains of setting this up on Mac OS X you can use this script instead.

Which should end with the following message.

Please run RbEnv rehash again to ensure that the changes applied.

A common problem seems to be a failure around using libxml2. This can be fixed by installed it with Homebrew and re-linking the files.

Running the chosen Ruby version with the local gems is done by running the bundle exec command, for example.

Vagrant and VirtualBox

You’ll need to use both websites to install these applications.

VirtualBox for Mac OS X over https://www.virtualbox.org/wiki/Downloads

Vagrant for Mac OS X over https://www.vagrantup.com/downloads.html

Using Test Kitchen locally

With Bundler we can call Test Kitchen to do several things like list available instances.

The drivers will be Vagrant, AWS or others. The Provisioner will be ChefSolo, Puppet or another. The Last action will be.

Not Created — Instance does not exist

Set Up — Creation or update running or not completed.

Converged — Last update ran.

Verified — Tests ran.

You can Create or update an instance with the Converge command followed by a regex which will apply to one or more instances.

Run the integration tests for the local environment with Verify.

You can login into an instance by using a regex that will only result in one instance.

Delete them completely with Destroy.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store