Design Web Infrastructure (LEMP Stack)
In this opportunity I share with you the project that we carried out in our learning stage at Holberton School Peru together with my colleagues Diego Linares and Juan Salinas.
This project dealt with the Design of a Web Infrastructure which was diagrammed following some requirements that have been incorporated little by little throughout the entire project by Holberton.
We will use the LEMP stack which is a set of open source software that is widely used to create websites and web applications, where the acronyms refer to the elements that compose it such as:
- Linux with the Ubuntu Server distribution as the operating system.
- Nginx as web server.
- MySQL as a relational database management system.
- Python programming language with the Gunicorn service
Web Infrastructure in detail
The web infrastructure that was more scalable and redundant for a specific use case contains the following components as described below and as can be seen in the attached image:
Making the request
In Node 1 you can see the residential private network from which the request is made to a domain name foobar.com configured with a record www that points to its server IP 8.8.8.8.
Which, as shown in the graph, the router that is the connection layer between our private network and the external network communicates through the internet to a DNS server in which it makes the request about the IP associated with the domain name and this is returned over the internet (now we already have the address of the server that we have to reach where the files that we need to deliver to the client are hosted).
Then, as we see in the graph, the client makes the request through the web browser, the connection is established with the server that contains the request file through TCP (which is in layer 4: transport layer of the OSI Model) that works with IP (internet protocol) the two TCP/IP constitute a communication protocol that is the most used on the web.
Once the connection is established, a request/response can be exchanged between the client and the server through the HTTPS protocol (encrypted connection that protects the integrity and confidentiality of the data exchanged) through port 443, and this is possible because in the arrival server (Node 2) we have an SSL certificate to be able to identify with the browser and establish the encrypted connection.
Analyzing SPOFs
We analyze the SPOF (Single point of failure) that occurs when we have a part of the system that, if it fails, will cause the entire system to stop working. The idea is to strengthen the systems by adding redundancy in all potential SPOFs and that is why the following actions were taken in the system in general:
- In Node 2, 2 Load Balancers (HAproxy) were added instead of 1, these load balancers are configured as a cluster with one another, which means that both load balancers are combined to function as a single entity and their behavior is similar to a active-passive state, since if a load balancer fails immediately the other takes its place, thereby increasing the redundancy of the system in general at a key point.
- In Node 3, 4 and 5, we have servers that are separated from each other and thanks to this we manage to avoid the SPOF in the servers (web server, application server and database).
Firewall
We add a Firewall on arrival at each server that has an internet connection, which is a security system that monitors and controls incoming and outgoing traffic, without a firewall we could be attacked from the internet and we could be vulnerable to loss of information.
Load Balancers
Node 3, 4 and 5:
- As we can see, the load balancer through port 443 communicates with servers 3, 4 and 5, distributing the requests through the Round Robin algorithm which sends the requests to each server in turn in a loop every time it finishes with all of them the servers in the list and according to the idle capacity of each server.
High Availability
A concept that we apply when designing this web infrastructure is the concept of High Availability, which refers to the ability of users to access a system without loss of service. To solve the problem of high availability the most important mechanism is redundancy. For this we use a combination of solutions between active-active and active-passive which mean:
- Active- Active, that 2 or more active instances of the system can be implemented and this can improve scalability and provide high availability.
- Active — Passive, which is basically about implementing an active instance of the system that handles the requests and a passive instance that is waiting. In addition, between these two instances or servers there is a cluster software that is responsible for monitoring and automatically failover between the cluster nodes.
It was decided to use an active-active system (Node 3 and 4) and a passive (Node 5) of the second active (Node 4), thus achieving a system that has high availability, redundancy and achieve resource efficiency since the useful life of the system that is in passive mode will be greater due to its considerably lower use of its components, waiting for the failure of Node 4 or Node 3 to start up and be able to ensure the availability of the service without interruptions .
Components of each physical server
Each server has the following components:
- 1 Firewall (UFW)
- 1 SSL Certificate (AWS Certificate)
- 1 Web Server (Nginx)
- 1 Application Server (Gunicorn)
- 1 Database Server (MySQL)
- 1 Application files (codebase)
Web Server
The web server serves us to be able to store and transmit the data that is requested by the client (browser) to the Application Server, basically it is in charge of displaying the static content (HTML, CSS and Javascript). Also, Nginx was used because we had a choice of Apache or Nginx but the client preferred to use Nginx.
Application Server
The application server is the one that provides the dynamic part to the application, since it will be in charge of managing most (or all) of the business logic and access to the application data. In this case we use Gunicorn as the Application Server because we are using Python as the programming language.
Database Server
The Database Server will be in charge of keeping the data organized and related for easy and fast access to it. In addition, thanks to this we can achieve the persistence of the data in the application. In this case, we use MySQL because it is free software and its main feature is speed, since it was designed from the beginning with speed in mind.
Regarding the database, it was decided to have in Nodes 3 and 4, the databases configured as master and the database of Node 5 as Slave. This means that both the databases of nodes 3 and 4 have read and write permissions which are connected through the MySQL cluster.
And also in Node 5 we have the slave database that at first would have read-only permissions for being connected through Sync backup but that in the event of a possible failure of the servers to which it backs up, both could be changed manually or automatically its passage from Slave to Master database which would mean that it would have read and write permissions.
Monitoring Clients
In addition, Monitoring Clients were placed, in this case we will use Datadog which will be in charge of monitoring the servers, databases and also has services through a data analysis platform based on SaaS, it was decided to place in the following points already which are important points to monitor:
- In each server (Node 3, 4, 5) where the web servers, application servers, Database server and codebase are located.
- In addition, also in the server where the 2 Load Balancers are located, it is important to monitor them due to their relevance in the system and that always configure a SPOF, so it is important to have data on its status.
Thank you very much for reading and I hope you have enjoyed it as much as I have been able to write this article to share what I have learned on this topic.
