Deploying Web Server & HAProxy Load Balancer on AWS Using Ansible
Provisioning AWS EC2 Instances and Security Group using Ansible Playbook and then Deploying HTTPD Web Server and HAProxy Load Balancer Server using Ansible Roles.
This article will help you to learn, How we can setup EC2 Instances and Security Group on AWS using Today's Most Demanding Automation Tool — Ansible. Next we gonna see how to Deploy multiple HTTPD Web Server as backend server and HAProxy Load Balancer as frontend on top of these backend servers using Ansible Roles.
This particular demonstration will also helps us to learn Distributed Architecture. But before starting let's understand some basic terms…
HAProxy :
- HAProxy is a high-performance, open-source load balancer and reverse proxy for TCP and HTTP applications. Users can make use of HAProxy to improve the performance of websites and applications by distributing their workloads. Performance improvements include minimized response times and increased throughput.
- Load balancers can be used to distribute workloads across computers, networks, disk drives or CPU’s. HAProxy, included in many distributions of Linux, is one of the leading standards in software load balancing. HAProxy is used in high traffic services such as GitHub and Twitter.
- To know more about HAProxy : https://www.haproxy.org/#desc
Load Balancing :
- Load balancing is the process of distributing network traffic across multiple servers. This ensures no single server bears too much demand. By spreading the work evenly, load balancing improves application responsiveness. It also increases availability of applications and websites for users. Modern applications cannot run without load balancers.
- In production environment, companies have thousands of servers running the same Application (In our example this is HTTPD Web Server). This is because one server can server a limited number of clients at a particular point of time. But for bigger companies, their servers get thousands of traffic per second. So, to serve huge amount of traffic we need to use lots of servers, but the problem is each server will have their own IP address and for client it will be too much confusing to find which IP address is currently free to take their request.
- To, solve this problem we use Load Balancers (In our example it is HAProxy). We connect our Load Balancer with lots of backend servers and then provide the IP of the Load Balancer to the clients. So, for them it will be easier to reach out to our Application without any confusion. Next as we are using lots of servers as backend, so we will be able to serve huge number of clients at a particular point of time. Also if somehow our any backend server corrupts, we don't need to worry because our other backend servers will be there to handle the traffics. Likewise Load Balancer improves security, prevent Single Point of Failure, ensure disaster recovery and many more facilities.
Let's see the Problem Statement :
Statement: Deploy a Load Balancer and multiple Web Servers on AWS instances through ANSIBLE!
- Provision EC2 instances through ansible.
- Retrieve the IP Address of instances using the dynamic inventory concept.
- Configure the web servers through the ansible role.
- Configure the load balancer through the ansible role.
- The target nodes of the load balancer should auto-update as per the status of web servers.
Summary: One-Click Instance Launched, Web Servers provisioned and Load Balancer ready!
Pre-requisite :
To perform this practical, there are few pre-requirements we need to know beforehand. Let's see those pre-requisites…
- Basic knowledge of AWS is required. Also before beginning we need to make sure that we have respective “aws-access-key”, “aws-security-key” and EC2 pem format key on our local system.
- Basic knowledge of Ansible and Ansible Playbook is required. Also you should have Ansible Software installed in your local system. But don't worry if you don't have these things because just recently I wrote one Article where I described these things from very basics. Feel free to check out the below mentioned article…
- Lastly, This article is the 2nd part of my previous Article where I talked about Ansible Roles from very basics. Also shown how to provision EC2 Instance using Ansible Playbook and Setup HTTPD server using Ansible Role. So, those parts I will not talk in details in this blog, because this blog is focused mainly on Load Balancer.
Let's Begin :
On my previous article I had shown how to create Ansible Vault to store the AWS secret key and access key. Also I had talked in details about the Configuration file of Ansible. Those things are same here as well. I am considering you already know those things.
So, I am staring from the Ansible Playbook…
As always, I will breakdown my Ansible Playbook in small small parts and finally at the end we gonna merge everything and will run. Also at the end of this article I will provide the GitHub link from where you can download the entire Workspace…
Installation of boto3 and Provisioning AWS Security Group :
- This is the exact same code we used on my previous article. Only difference is here we gonna use this security group for our backend web servers.
Provisioning EC2 Instance & Creating Dynamic Inventory :
- Here also this is the same code we used previously and this 1st “ec2” module is launching EC2 Instances on AWS. Here only difference is we are launching 2 Instances as our backend server. You can launch as many as you want.
- Next using Dynamic Inventory Concept, we are storing the IPs of these Instances dynamically on an temporary variable called “webserver”.
Running Ansible Role to setup HTTPD on those backend servers :
- This is again the same code we used previously. At first we are waiting for SSH to come up and then we run the “webserver” role on the host group “webserver” which we created dynamically.
Security Group for Load Balancer :
- From here the new addition occur. Here I am running this playbook on my localhost which is creating one AWS Security Group called “LBSG”. Here instead of 80, I am using 8080 port for my Load Balancer, because I will configure my HAProxy Load Balancer to run on this port. Rest everything is same.
Load Balancer Instance :
- This code is creating one Instance on AWS. Here just to distinguish the backend server instances and frontend server instance, I used different tags. On previous Instances I used “webserver” tag, and here I used “lbserver” tag.
Deploying HAProxy Role :
- As like previous time Here also I created one temporary variable called “lbserver” to store the Load Balancer Instances IP.
- Next I applied the Ansible Role called “lbserver” to setup “HAProxy” Load Balancer. Till now everything is pretty much simple because these things we have already observed on my previously article on details. From here the most important part starts…
Creating Role for HAProxy :
Go inside the “roles” folder in your Workspace to create the Ansible Role like we did on previous time. Next run the below mentioned command and your Role will be created…
ansible-galaxy init lbserverCreating Task Inside lbserver Role :
Now we need to write the plays in the “main.yml” file inside the “tasks” folder. Write down the below mentioned script in that particular file…
- At first we are using “package” module to install HAProxy Software.
- Next we are using “template” module to copy our own customized configuration file. Here we can't use “copy” module because on the fly we will add our backend server IPs on this “haproxy.cfg” file and to do that we need to use “template” module.
- Also notice here we are using “notify” keyword. This keyword we use when some other future tasks depends on this task. So, using this notify we can trigger that dependent task.
- Lastly we are starting the HAProxy Service using “service” module.
Configuring “haproxy.cfg” file :
Frankly speaking, to do this practical I already installed the HAProxy packages on my local system and then copied the configuration file located at “/etc/haproxy” folder to “roles/lbserver/templates” folder. Because in Ansible Role whenever we use “template” package we need to store the file inside this “templates” folder.
Next we need to do little bit changes on this “haproxy.cfg” file. Just see the below screenshot and make the changes.
- Make sure on line no 68 of the “haproxy.cfg” file the port no is “8080”.
- Here we need to mention our backend server IPs along with the port number. If you notice the actual line that HAProxy require is 89 no line. On that line we mention where is our backend server.
- Now as we need to update this file dynamically, so we are using for loop. Ansible works on Python Embedded Programming called “jinja”. In “jinja” we mentioned the starting of the for loop using “{% for %}” keyword and end the for loop using “{% endfor %}” keyword.
- Next if we notice here “groups” is a keyword whose argument is “webserver”, that means we are using “webserver” host group. Next “hosts” is a keyword which picks the IPs from the host group.
- So, finally using for loop, we are picking one IP from host group “webserver” and creating one line in haproxy.cfg file dynamically. That means if we launch hundreds of backend server, our “webserver” host group will contain their IPs and we can easily connect them with HAProxy, through this configuration file.
Creating Handler :
On the “template” task, we mentioned one keyword called “notify”. Now in Handler we gonna mention that dependent task. Mention these below mentioned lines in the “main.yml” file inside “handlers” folder.
- Just notice one thing, Here using Handler we are restarting our “haproxy” service. Because suppose in future we add more backend servers and to add them on the HAProxy Load Balancer, we need to mention their IPs on the configuration file. Now in most of the servers when we update anything on our configuration file we need to restart the services.
Very important thing to note that the name of this task should be the same we mentioned after the “notify” keyword on previous task.
Creating Role for Web Server :
Next again run the same Ansible Role creation command and create a role for Webserver. I’m using the name “webserver” for this role. Next just to make sure both role is present and Ansible is able to locate them run the below mentioned command…
Creating Task Inside webserver Role :
Similarly like previous time again we are creating Tasks for setting up HTTPD webserver on our backend Instances. Mention the below script on the “main.yml” file inside “tasks” folder.
- This time along with “httpd”, we are also installing “php” package. Next I am deploying one “php” file from my GitHub repository. This “php” code will help us to see that how the Load Balancing is happening.
- Finally we have started the “httpd” service.
Let's Deploy :
So, finally we are done with our setup and you already know how we deploy our Ansible Playbook. Also here is the GitHub link from where you can download the entire Workspace…
Let's see the Output :
Hurry…
- we have successfully deployed our whole Setup of Web Server and HAProxy Load Balancer. Now let's go to the website and check what our HAProxy Load Balancer is showing us.
Go to the browser and type the Public IP of the Load Balancer Instance along with port no 8080, as HAProxy is working on this port.
Great…
- Our Load Balancer is able to forward our traffic to backend web servers. If you want to make sure from which instance this page is coming just check the IP address of “eth0” network card and check any of the Web Server Instance private IP from AWS Web Console. You will see it’s exactly same with one of the Web Server.
- Next to check the Load Balancing, just refresh the page and you will see it’s switched the traffic from 1st instance to 2nd instance and you will see the IP of “eth0” card changed. For reference see the below mentioned screenshot…
Congratulations… We have successfully deployed our Webserver and HAProxy Load Balancer on AWS Instance using Ansible Automation.
Final Words :
- Like I said previously, There are lots of future possibilities for this particular task. We can write bigger scripts to create more diverse architecture by adding more AWS services. Instead of HAProxy We can also deploy ELB service which is also very powerful.
- I tried to make it as simple as possible. Hope You learned Something from here. Feel free to check out my LinkedIn profile mentioned below and obviously feel free to comment. I write DevOps, Cloud Computing, Machine Learning etc. blogs so feel free to follow me on Medium.
Thanks Everyone for reading. That’s all… Signing Off… 😊
