Speed up Ansible with Mitogen!

The Rocket symbolizes speed

Ansible is one of most popular Configuration Management Systems nowadays after it was acquired by Red Hat in 2015 Ansible has reached numbers of thousands of contributors and became maybe one of most used deployment and orchestration tools. Its use-cases are quite impressive.

Ansible works by SSH connections to remote hosts. It opens SSH session, logs in to shell, copy python code via network and create a temporary file on remote hosts with this code. In the next step, it executes the current file with python interpreter. All this workflow is pretty heavy and there are multiple ways to make it faster and lighter.

One of these ways is using SSH pipelines which reuses one SSH session for copying python code of multiple tasks and prevent opening multiple sessions, which saves a lot of time. (Just don’t forget to disable requiretty settings for sudo on the remote side in /etc/sudoers)

The new way to speed up Ansible is a great python library called Mitogen. If somebody like me was not familiar with it — this library allows fast execution of python code on a remote host and Ansible is only one of its cases. Mitogen uses UNIX pipes on remote machines while passing “pickled” python code compressed with zlib. This allows to run it fast and without much traffic.

Mitogen in specific circumstances can speed up your Ansible in a few times and significantly lower your bandwidth. Let’s check the most popular use cases and figure out if it’s helpful for us.

The most popular use cases for me to run Ansible are:

• creating configuration files on a remote host,
• packages installation,
• downloading and uploading files from and to remote hosts.

Maybe you want to check other use cases, please leave comments to this article.

Let’s start rolling!

Configuring Mitogen for Ansible is pretty simple:

Install the Mitogen module:

pip install mitogen

Then either configure environment variables or set configuration options in ansible.cfg file, both options are fine:

Let’s assume our path to installed Mitogen library is:

 /usr/lib/python2.7/site-packages/ansible_mitogen/plugins/strategy

or

Prepare Ansible in virtualenv, with and without Mitogen enabled:

virtualenv mitogen_ansible
./mitogen_ansible/bin/pip install ansible==2.7.10 mitogen
virtualenv pure_ansible
./pure_ansible/bin/pip install ansible==2.7.10

Please pay attention that Mitogen 0.2.7 doesn’t work with Ansible 2.8 (for May 2019)

Create aliases:

alias pure-ansible-playbook='$(pwd)/pure_ansible/bin/ansible-playbook'
alias mitogen-ansible-playbook='ANSIBLE_STRATEGY_PLUGINS=$(pwd)/mitogen_ansible/lib/python3.7/site-packages/ansible_mitogen/plugins/strategy ANSIBLE_STRATEGY=mitogen_linear $(pwd)/mitogen_ansible/bin/ansible-playbook'

Now let’s try the playbook that creates file on remote:

And run it with Mitogen and without while creating 10 files:

time mitogen-ansible-playbook file_creation.yml -i hosts -e n=10 &>/dev/null
real 0m2.603s
user 0m1.152s
sys 0m0.096s
time pure-ansible-playbook file_creation.yml -i hosts -e n=10 &>/dev/null
real 0m5.908s
user 0m1.745s
sys 0m0.643s

Right now we see improvement in x2 times. Let’s check it for 20, 30, …, 100 files:

time pure-ansible-playbook file_creation.yml -i hosts -e n=100 &>/dev/null
real 0m51.775s
user 0m8.039s
sys 0m6.305s
time mitogen-ansible-playbook file_creation.yml -i hosts -e n=100 &>/dev/null
real 0m4.331s
user 0m1.903s
sys 0m0.197s

Eventually, we improved execution time in more than 10 times!

Now let’s try different scenarios and see how it improves:

• Scenario of uploading files from the local host to remote (with `copy` module):

• Scenario of creating files on the remote host with `copy` module:

• Scenario with fetching files from the remote host to local:

Let’s try the last scenario on a few (3) remote hosts, for example uploading files scenario:

As we can see the Mitogen saves us both time and bandwidth in these scenarios. But if the bottleneck is not Ansible, but for example I/O of disk or network, or somewhere else, then it’s hard to expect from Mitogen to help of course.

Let’s run for example packages installation with yum/dnf and python modules installation with pip.

Packages were pre-cached to avoid dependencies on network glitches:

With Mitogen it takes 12 seconds, as well as with pure Ansible.

On Mitogen for Ansible page you can see additional benchmarks and measurements. As the page declares:

Mitogen cannot improve a module once it is executing, it can only ensure the module executes as quickly as possible

That’s why it’s important to find where your bottlenecks are and if they are related to Ansible operations, Mitogen will help you to solve it and speed your playbooks up significantly.