Using Vagrant’s Ansible Provisioner to Build a FreeIPA Server

I worked on a project recently, which I thought had relatively simple requirements:

• Use vagrant to spin up a CentOS 7 box.
• Run an ansible playbook to build a FreeIPA server on that new box.
• Tightly couple the Vagrantfile and the FreeIPA server ansible playbook and keep them in its own git repository.
• Define any ansible variables and a static ansible inventory in a central location (/etc/ansible/), so that I could use them to not only to build a FreeIPA server, but also reuse them with other ansible playbooks to build FreeIPA replicas and clients (and not necessarily on vagrant boxes). Keep this configuration in its own independent git repository.

I assumed that these were reasonable requirements. However, I discovered, making this come to life was not as simple as it seemed it would be.

What does any of this have to do with network defense?

CentOS is to FreeIPA as Red Hat is to Red Hat Identity Management (IdM).

I find a lot of value in being able to quickly build up and tear down a central authentication server, and everything that comes with it (i.e. two factor authentication, LDAP, etc). It’s important to me to

It just so happens that FreeIPA is…well, free.

It would be swell to have a quick, zero cost, repeatable process to build up and tear down Microsoft Domain Controllers.

My Tools

Here are the tools I used while I was figuring this stuff out:

$ cat /etc/redhat-release
CentOS Linux release 7.6.1810 (Core)
$ vagrant --version
Vagrant 2.2.5
$ ansible --version
ansible 2.8.4
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/chris/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.5 (default, Jun 20 2019, 20:27:34) [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)]

The CentOS 7 Vagrant Box

I started with a simple CentOS 7 Vagrantfile:

vagrant init centos/7

And then trimmed it down and customized it a bit:

Note that I configured this box with a second NIC, bridged to eno1, and I explicitly defined its MAC address.

This box spins up on a network where DHCP statically maps the MAC address to a certain IP address. The network also has DNS configured so that the hostname test.my.domain resolves to that IP address.

This helps me cut down on what would otherwise be additional needed configuration.

With my new, custom Vagrantfile:

vagrant up

Everything worked great:

$ vagrant ssh
[vagrant@test ~]$ hostname
test
[vagrant@test ~]$ logout
Connection to 127.0.0.1 closed.

So I tore it down, and to get ready for the next iteration:

vagrant destroy -f

FreeIPA Ansible Roles

Vagrant has an ansible provisioner, which gives us the capability to call on ansible from within our Vagrantfile and run ansible playbooks on a new box.

And the FreeIPA project publishes ansible roles and playbooks that can install and uninstall FreeIPA servers, replicas and clients.

I realized that the FreeIPA ansible roles that I’ve been using are somewhat dated when I realized that I have additional ansible roles, related to FreeIPA, that are no longer a part of FreeIPA’s master branch on github.

My FreeIPA ansible roles seem to be closest to FreeIPA’s “v0.1.0” release.

If you’re playing along from home and want to get the same roles:

git clone --branch v0.1.0 https://github.com/freeipa/ansible-freeipa.git

Which matches closely with what I have:

$ ls -1 ansible-freeipa/roles/
ipaclient
ipaconf
ipa-krb5
ipareplica
ipaserver
ipa-sssd

If you happened to git clone FreeIPA’s current master branch today, you’d notice that ipaconf, ipa-krb5, and ipa-sssd are now gone. At some point down the road, I’ll have to upgrade the FreeIPA roles I have and try this whole thing over again to see if there are major changes.

(I know that if I used Ansible Galaxy I would have probably stayed up to date with the most current roles. I’m a late adopter, though. I keep all of my ansible roles in my own private version control system and upgrade when I’m good and ready.)

To make our new roles available to ansible:

sudo cp -r ansible-freeipa/roles/* /etc/ansible/roles/

The Playbook

After installing the FreeIPA ansible roles, I wanted to run a playbook like this against a new CentOS 7 box:

So I created the following file structure:

$ tree
.
├── provisioning
│   └── playbook.yml
└── Vagrantfile

I also updated my Vagrantfile to make sure that it calls provisioning/playbook:

Why the following line?

ansible.ask_vault_pass = true

Because I have a /etc/ansible/group_vars/all/vars.yml that includes pointers to an ansible vault. I want ansible to ask me for the vault password to decrypt the secrets it needs. If I did not include this, vagrant’s ansible provisioner would give up and die. It’s not smart enough to know that it should ask for a password, unless you explicitly tell it that it should. Which is not much different from how ansible-playbook works.

After reading the documentation for the FreeIPA ansible role, I knew that I needed to add my test vagrant box to the ipaserver ansible group in /etc/ansible/hosts.

I realized that I needed to define a few variables for the ipaserver group:

• ipaadmin_password
• ipadm_password
• ipaserver_domain
• ipaserver_realm

So, I…

• …added the following to /etc/ansible/hosts :

• …created a space for ipaserver group vars:

pushd /etc/ansible/group_vars/
sudo mkdir ipaserver
sudo touch ipaserver/vars.yml
popd

• …added the following to /etc/ansible/group_vars/ipaserver/vars.yml

I’m defining secrets in the clear, here. But in real life, some of the variables point to an ansible vault in /etc/ansible/group_vars/ipaserver/vault.yml.

When I spun up this new vagrant box:

vagrant up

Things did not go as planned…the setup sequence crashed and burned.

vagrant destroy -f

What Went Wrong?

With the benefit of hindsight and lots of research, I can see that this set up is a little too naïve to actually work.

Here are my key findings:

• Vagrant’s Ansible Provisioner Ignored Ansible’s Central Configuration. Vagrant’s ansible provisioner only seems to care about configuration files that are local to the vagrant project. I can see how, under most circumstances, this is probably desirable, but…that is not what I wanted.
• Vagrant’s Ansible Provisioner Used the Wrong SSH Username. Vagrant’s ansible provisioner has a mechanism available that is supposed to let you set the SSH username for the ansible provisioner. I thought I used it correctly. I did not.
• FreeIPA Needed a Box with a FQDN: The FreeIPA box must have a fully qualified domain name in order for installation to work correctly. But this configuration will only set up a box with the hostname set. In other words, I was getting a box with the hostname test , when I need the hostname to be test.my.domain in order to make the FreeIPA ansible role happy.
• FreeIPA Expects that firewalld Will Be Installed: The FreeIPA server role expects that firewalld will be installed on the box. The CentOS 7 vagrant box does not have firewalld installed.

Tricking Vagrant’s Ansible Provisioner to Use Ansible’s Central Configuration

I assumed that vagrant’s ansible provisioner would automatically consult with /etc/ansible/ , and so this tripped me up for a while.

Vagrant’s documentation for the ansible provisioner does point out that there is a mechanism to point a Vagrant file towards a custom static ansible inventory. So, in hindsight, I suppose I probably could have tried something like this:

However, I think this would have ultimately ended up wrecking havoc on some of the special sauce I needed to include in the inventory file, with respect to vagrant SSH keys. (More on this shortly.)

Note that in vagrant’s documentation for the ansible provisioner:

Since an Ansible playbook can include many files, you may also collect the related files in a directory structure like this:

.
|-- Vagrantfile
|-- provisioning
|   |-- group_vars
|           |-- all
|   |-- roles
|           |-- bar
|           |-- foo
|   |-- playbook.yml

I decided to use a brute force a solution to trick vagrant’s ansible provisioner into consulting with/etc/ansible/group_vars/ :

cd provisioning
sudo ln -s /etc/ansible/group_vars/
sudo ln -s /etc/ansible/hosts

Symbolic links, located in the provisioning folder, pointed towards my central configuration did the trick for me. It feels like a kludge, but, hey, it works!

Well…it almost works.

My central static inventory, /etc/ansible/hosts , still has some unresolved issues.

Before we created those symbolic links, vagrant’s ansible provisioner would auto create an inventory file for us.

Note that the generated inventory file is stored as part of your local Vagrant environment in .vagrant/provisioners/ansible/inventory/vagrant_ansible_inventory.

By inspecting that auto-generated inventory file, I realized that I probably want something like this in my static inventory:

Notice the relative path to the ansible_private_key_file . Vagrant dynamically generates that SSH key upon issuing vagrant up .

If we had specified:

ansible.inventory_path = "/etc/ansible/hosts"

in our Vagrant file, I was concerned that the relative path for ansible_private_key_file may have suddenly been a path relative to /etc/ansible and not our current Vagrantfile.

Vagrant’s Ansible Provisioner and SSH Usernames

Vagrant’s ansible provisioner looks for ansible_ssh_user in ansible inventory files, when it’s looking for the username that it should use to ssh to your new vagrant box:

From Vagrant’s Documentation

Confusingly, Ansible 2.0 has deprecated the use of ansible_ssh_user:

And unfortunately for me, quick Google searches resulted in things like this StackOverflow post, which boosted my confidence that all I needed to provide in my ansible inventory was an ansible_host.

This is what actually worked for me, in my provisioning/group_vars/ipaserver/vars.yml.

At this point, I cleaned up my inventory to look like this:

So even though ansible_ssh_user is depreciated with just plain old ansible, leaving it out, when using vagrant’s ansible provisioner, caused catastrophic problems.

Defining a Fully Qualified Domain Name on a Vagrant Box

This also stumped me for a while.

Vagrants documentation led me to believe that if I included the following statement in my Vagrantfile…

config.vm.hostname = "test.my.domain"

…then, I should expect to be able to do something like this, and see the corresponding result:

vagrant up
vagrant ssh
hostname
test.my.domain

However, I was seeing this, instead:

vagrant destroy -f
vagrant up
vagrant ssh
hostname
test

The lack of my.domain is a real problem for the FreeIPA server installation process.

I later discovered that adding the following to my Vagrantfile, gave me the FQDN that I wanted:

config.vm.define = "test.my.domain"

Making that change, my Vagrantfile now looks like this:

Which gives me the FQDN that I needed.

But we’re not out of the woods just yet…

vagrant up

And the FreeIPA installation still fails with the error:

The IPA Server hostname must not resolve to localhost (127.0.0.1). A routable IP address must be used.

Sure enough:

vagrant ssh
cat /etc/hosts | grep test
127.0.0.1 test.my.domain test

At this point, I’m tired of trying to find clever solutions and decide to go with a brute force option and decide to us my ansible playbook to delete /etc/hosts and create a new, empty one.

I also decide to do this to install and start firewalld. Which gives me my final playbook:

Note: deleting & creating /etc/hosts and installing & starting firewalld are in the pre_tasks section of the playbook. Ansible executes roles before tasks, so if we had made those items just regular tasks, they would be too late to be useful.

That’s It!

And that covers all of the moving parts to my working solution.

To recap:

tree
.
├── provisioning
│   ├── group_vars -> /etc/ansible/group_vars
│   ├── hosts -> /etc/ansible/hosts
│   └── playbook.yml
└── Vagrantfile

provisioning/group_vars/ipaserver/vars.yml:

I found it simpler to move all of the host variables to provisioning/group_vars/ipaserver/vars.yml I now have a nice, clean provisioning/hosts :

provisioning/playbook.yml:

Vagrantfile:

vagrant up

And a few minutes later, I have a brand new FreeIPA server: