How to configure a reverse proxy for Web Services on AWS using Ansible.

This article will cover provisioning of ec2 instances for web services and setting up Load Balancer for those Web Services over AWS using Ansible.

Before I cover what is load balancer and how to configure load balancer for web services, let’s visualize a scenario.

You have a web server that can manage 1000 clients at a time. Suddenly the requests to that particular server increase to 1500 clients. It’s likely that the website will crash or be terminated. To avoid this situation, the need arises to set up another target web server. In this scenario, connecting client to which of the target web servers and how, pose a challenge!

To solve this challenge, a master server is set up in front and the client never goes to the target web servers. Instead, their request goes to the master server, and the master server sends the request to the target web server. When the target web server replies to the master web server, it is sent back to client. This concept is known as reverse proxy.

What is a load balancer?

A load balancer is an efficient way to distribute the network traffic among various backend servers. It is also known as a server farm or server pool. It distributes client requests or network load to target web servers. Load balancers work on the round-robin concept, which ensures high reliability and availability.

Placing a load balancer in front of your web server infrastructure helps ensure any spike in traffic doesn’t bring down the site.

HA Proxy (High Availability Proxy)

HAProxy, which stands for High Availability Proxy, is a popular open-source software TCP/HTTP Load Balancer that can be configured as reverse proxy. Its most common use is to improve the performance and reliability of a server environment by distributing the workload across multiple servers (e.g., web, application, database). It is used in high traffic services like: GitHub, Imgur, Instagram, and Twitter.

More about HAProxy: https://www.haproxy.org/#desc

Let’s proceed to the configuration part.

In this article, you’ll learn how to create a setup on AWS similar to one shown below:

For this, you need to launch 4 ec2 instances on AWS, 1 as master server(load balancer) and 3 as backend web servers. Steps we going to perform:

1. Provisioning EC2 instances using Ansible
2. Retrieving IP Address of the instances using the concept of dynamic inventory
3. Configuring the web servers using Ansible role.
4. Configuring the load balancer using Ansible role.
5. The target nodes of the load balancer should auto-update as per the status of web servers.

To know how to launch ec2 instances on AWS and configure it for Web Server using Ansible playbook, go through the link below, as a pre-requisite before continuing to this article:

Automating provisioning and configuration of Amazon ec2 instance Ansible.

1. Provisioning of EC2 instances using Ansible

Playbook code to launch ec2 instances for webserver is shown below:

2. Retrieving IP Address of the instances using the concept of dynamic inventory

Go through my article below for, what is Ansible dynamic inventory and how to retrieve public IPs of instances on AWS using dynamic inventory.

Automating provisioning and configuration of Amazon ec2 instance Ansible.

To list public IP of instances running on AWS using dynamic inventory 
$ ansible ec2 --list-hosts

3. Configuring the Web Servers using Ansible role

For this, you need to create a Ansible role to configure these instances as Apache Web Server to work as backend/ target web servers. The purpose behind using Ansible Roles is, roles let you automatically load related vars, files, tasks, handlers, and other Ansible artifacts based on a known file structure. Also, after you group your content in roles, you can easily reuse them and share them with other users.

An Ansible role has a defined directory structure with eight main standard directories. You must include at least one of these directories in each role. You can omit any directories the role does not use.

Creating roles for Apache:

Create a directory named, roles in your workspace. Go inside the directory, and create ansible role using command, ansible-galaxy init <role_name> to create the role directory structure.

#Create ansible role, apache within directory, roles
$ ansible-galaxy init apache

Now you need to write down the below script for tasks, within “main.yml” file of the tasks directory as can be seen above:

• In the first step, using package module, httpd software installed on the target nodes.
• Next, using copy module, webpage is copied from source to destination, /var/www/html. The source file here, index.php, need to be put into the files directory, to be loaded automatically while running the role.
• Lastly, service module used to start and enable httpd service.

You might have noticed in the above tasks/main.yml file, I have notified handler to avoid unnecessarily restarting service again and again, instead handler will restart the service when it gets notified when there are changes in webpages. So, for this you need to write script for handlers in handlers/main.yml file as shown below (see the role directory structure above).

Now, as you have written all the necessary files, you can also remove the unwanted directories from the role directory structure(optional).

Now as you completed creating roles, you need to update the roles path into /etc/ansible/ansible.cfg file as shown below:

Further, you need to create a playbook where including the above apache role, you need to do httpd configuration on specified hosts. To get information about hosts group, run the dynamic inventory python scripts as shown below:

Use the highlighted group name to do the configurations for webservers. Run the below playbook code to configure this group of instances for httpd web server:

Superb!! Let’s move ahead to HAProxy configuration part!

4. Provisioning a EC2 instance for reverse proxy

To do HAProxy configuration, you need to provision an ec2 instance and configure it for HAProxy Load Balancer. Use the same script as above to launch an ec2 instance for the same (just change the tag name to “haproxy” and count to 1 in above file as can be seen below).

5. Configuring the Load Balancer using Ansible Role

As the instance is launched, now you need to configure it with the help of Ansible. For this purpose, create a Ansible role just like we did for webservers.

Creating role for HA Proxy LB:

Create a role named “haproxy” in the same “roles” directory of your workspace.

#Create ansible role, haproxy within directory, roles
$ ansible-galaxy init haproxy
#To list roles present
$ ansible-galaxy role list

Similarly, as you did before, create tasks to configure the instance as HA Proxy Load Balancer to work as reverse proxy. Write down the below mentioned script within “main.yml” file of “tasks” directory.

• In the first step, using package module, haproxy software installed
• Next, using template mode to copy the config file for HA Proxy, haproxy.cfg from the controller node to the managed node. Template module unlike copy module allows us to use Jinja2 to render a template to a file dynamically. Here, notify keyword used to notify handler to restart the service as there is any change in backend servers.
• Lastly, service module used to start the HAProxy service.

Now, you need to create your own custom config file, haproxy.cfg and put it into templates directory of the role.

Creating custom haproxy.cfg file:

You need to have haproxy.cfg file in your controller node to copy it into templates directory and further customize it.

#Copy haproxy.cfg file to templates directory of the haproxy role
cp /etc/haproxy/haproxy.cfg  /<workspace>/roles/haproxy/templates/ haproxy.cfg

By default, the frontend is bound to port 5000. Change the port number to 8080. Backend has ‘app’ block, where you need to give your backend servers’ IP to the backend app. Apply a for loop to configure the backend IP. Now you can launch as many web servers as you need, and there is no need to manually configure the IP inside the /etc/httpd/httpd.conf. It will automatically fetch the IP from inventory.

backend app    
balance     roundrobin 
{% for i in groups ["tag_Name_web_server"] %}    
server   app{{ loop.index }} {{ i }}:80 check 
{% endfor %}

Next, as in the above tasks/main.yml file I notified handler so, for this, we need to write below script into main.yml file of handlers directory of the role.

Now you have created all the necessary file, as you can check from the role directory structure below. Also, you can remove the unwanted directories from the role(optional).

Finally, you need a playbook to run against a host group for load balancer. Get the group name by running dynamic inventory python script as below:

Lastly you need to write and run a playbook code against the hosts group “tag_Name_haproxy” as shown below, to configure the instance for haproxy load balancer.

All set!! It’s time to see Ansible shine!

You can check from AWS Web console if all the instances are up and running or not.

Output

The playbook runs successfully, and the three target web servers can be accessed by the main web server using a load balancer as you can see below:

Tada!! You did a great job!

Let’s summarize what you did!

Using Ansible, we reached out to AWS, launched Amazon ec2 instances, thereafter, configured them for load balancer and reverse proxy respectively to get the desired set up. Adding HAProxy to your infrastructure not only enhances availability to your web services but also adds an extra layer of protection to backend servers.

One interesting thing to figure out is that, I did the entire setup with the help of automation tool without even going into the instances. That’s the power of automation!!

You can get the codes, I’ve used above from below GitHub link:

P-Bharti20/ansible_reverse_proxy

Thanks for reading………for any sort of queries leave a comment below. You can connect with me at:

Priyanka Bharti - Undergraduate Student - BIT Sindri | LinkedIn

Stay tuned for such awesome contents!