Our Ansible orchestration of the past

We recently changed our deployment strategy to use Docker Cloud, but before that, we dabbled in Ansible. About 2 years ago, we thought I’d be nice to be able to quickly replicate our environment, and went with Ansible (at the time using Docker on OSX wasn’t as easy as it is now).

Disclaimer: the project was left incomplete in that it wasn’t used in the end, this post will recall the basic structure and work done.

Services Used

• Vagrant: “a tool for building and managing virtual machine environments in a single workflow”
• Ansible: “a radically simple IT automation engine that automates cloud provisioning, configuration management, application deployment, intra-service orchestration, and many other IT needs”

Vagrant helped us to virtualise Linux environments, then we used Ansible to manage their orchestration and configuration.

Vagrant — Getting our virtual machine ready

Installation

Before anything, Vagrant and Virtualbox (the backend provider of our choice) need to be installed, you can reference the installation docs if needed.

Virtual machine

Choose a box instead of building a virtual machine from scratch for a faster process, for exmaple vagrant box add ubuntu/trusty64
Install the hostmanager plugin to be able to manage hosts files on our multiple machines: vagrant plugin install vagrant-hostmanager

Vagrantfile

The Vagrantfile allows us to configure and provision the machines required for the project, in our case this was:

Vagrant.configure(2) do |config|
  config.hostmanager.enabled = true
  config.hostmanager.manage_host = true
  config.hostmanager.ignore_private_ip = false
  config.hostmanager.include_offline = true
  # create ws1 machine
  config.vm.define :ws1 do |ws1_config|
    ws1_config.vm.box = "ubuntu/trusty64"
    ws1_config.vm.hostname = "ws1.witei.com"
    ws1_config.vm.network :private_network, ip: "10.0.15.20"
    ws1_config.vm.network "forwarded_port", guest: 80, host: 8081
  end
  # create ws2 machine
  config.vm.define :ws2 do |ws2_config|
    ws2_config.vm.box = "ubuntu/trusty64"
    ws2_config.vm.hostname = "ws2.witei.com"
    ws2_config.vm.network :private_network, ip: "10.0.15.21"
    ws2_config.vm.network "forwarded_port", guest: 80, host: 8083
  end
  # create db machine
  config.vm.define :db do |db_config|
    db_config.vm.box = "ubuntu/trusty64"
    db_config.vm.hostname = "db.witei.com"
    db_config.vm.network :private_network, ip: "10.0.15.11"
    db_config.vm.network "forwarded_port", guest: 80, host: 8082
  end
end

Boot the environment

Simply run vagrant up to start the virtual machine.

Install requirements

Enter the machine with vagrant ssh and install the requirements file:
pip install -r requirements.txt

Our requirements.txt contained:

Jinja2==2.8
MarkupSafe==0.23
PyYAML==3.11
ansible==1.9.4
boto==2.38.0
ecdsa==0.13
paramiko==1.16.0
pycrypto==2.6.1
sh==1.11
six==1.10.0
wsgiref==0.1.2
git+git://github.com/ticketea/forseti.git@a22af1d76d7d7657ed84085496573171bcf39dd5#egg=forseti

Ansible — orchestrating our machines

Installation

This is pretty straight forward, just make sure the machine meets the requirements. We included the installation of ansible in the requirements above.

Playbook

A playbook is a .yml file containing the instructions to configure and deploy our machines.

---
- name: Webserver playbook
  hosts: tag_Name_webserver
  sudo: yes
  remote_user: ubuntu
  vars_files:
    - vars/git_secrets.yml
  roles:
    - base
    - {
      role: witei-project,
      project_user: webserver,
      project_name: webserver
    }
    - {
      role: django,
      git_url: 'git@bitbucket.org:alvarovmz/witei.git',
      branch_name: master,
      project_name: webserver,
      project_user: webserver,
      requirements_file: requirements/prod.txt,
      environment_vars: {
        'DJANGO_SETTINGS_MODULE': 'housing.settings.prod'
      }
    }
    - {
      role: circus,
      project_name: webserver,
      project_user: webserver,
      template: wsgi,
      listen_port: 9100,
      wsgi_module: 'housing.wsgi.application',
      environment_vars: {
        'DJANGO_SETTINGS_MODULE': 'housing.settings.prod'
      }
    }
    - {
      role: nginx,
      upstream_port: 9100,
      project_name: webserver,
      redirect_to_https: False
    }
    - {
      role: witei-rotate,
      project_origin: "{{ PROJECT_FOLDER.stdout }}",
      project_name: webserver
    }

As you can see, we used a lot of roles in our playbook, these allow for abstraction. Each role is defined in a folder of their own, with vars_files, tasks, and handlers.

For example, our redis role (used in our development playbook) has just 2 files, a main.yml in handlers:

---
- name: restart redis
  service: name=redis-server state=restarted
  sudo_user: root

And a main.yml in tasks:

---
- name: Install redis server
  apt: name=redis-server update_cache=yes state=installed
  tags: packages
- name: Ensure redis is running
  service: name=redis-server
           state=started
           enabled=yes

To run the playbook simply ansible-playbook playbook.yml

Concluding

Vagrant provided us with a complete VM, and Ansible allowed us to structure our orchestration and deployment.

It has been 2 years since then, and when we picked-up the idea of automating our deployment, we didn’t rescue this project, but decided to use Docker Cloud instead.