Day 5: Message Queue | Part 2 [PRACTICAL]

Hello Hackers, this is Part 2 of the message queue, if you haven’t read the first part, please head over to the below link.

Day 4: Load Balancers & Message Queue

In this blog, we will create a simple Python application and will use AWS SQS along with that. The simple Python application will scan for the given domain name's open ports. Let’s get started.

The flow:

The application will get input from the user -> send the user input(domain name) to the SQS queue -> SQS queue will receive the request and process the request -> return the result.

The first step is to create a virtualenv for our project and install the required packages

$ python3 -m venv venv_name
$ pip install flask[async]
$ pip install python-dotenv

Step 1: Create your Python application

Here in the above image we have created two methods namely get_user_input() and nmap_scan(), we are going to use that get_user_input() function with the API, that’s why it just returns the given input.

The next function nmap_scan() will do the scan for the given domain. The nmap_scan() function is performing a synchronous task. This means that while the Nmap scan is running, the application will be blocked, and it won’t be able to handle other tasks or requests.

If you want to make this function truly asynchronous and non-blocking, you can use the subprocess module with the asyncio library to run the Nmap command asynchronously.

And for the sake of example, we took a simple Nmap command

$ nmap -p 80,443 {domain_name}

The command will check whether ports 80 and 443 are open or closed for the given domain.

Step 2: Get the SQS queue URL

In our last post, we already created our queue using the AWS console, Now we need to get that URL. Most of the SQS APIs require the QueueUrl so we will use the get_queue_url() method to retrieve the URL of the queue using the QueueName

In the above function, we just have to pass the name of the queue to the get_queue_url() function. The name can be found on our AWS console.

Step 3: Send the user input to the SQS queue

Now that we got our SQS queue URL, we need to send our messages or user requests to the queue, so that our server will process those requests in that order(FIFO).

To send the message to the queue, first, we need to make sure that our message is converted into JSON format or key & value format. Then using the send_message() function, we can send our message.

The message sent will be converted into an MD5 hash.

Step 4: Receive the message from the queue

We will be using the receive_message() method from Boto3 to receive a message from the SQS queue. Some of the important parameters to keep in mind while using this method:

• QueueUrl: URL of the queue we want to receive a message from
• MaxNumberOfMessages: The maximum number of messages to retrieve.
• WaitTimeSeconds: Amount of time to wait for a message to arrive in the queue. Useful for long-polling of messages

After receiving and processing the message, we need to delete the message from the queue, so that queue won’t pick up that message again. Basically, to avoid duplication and to save some processing power, we need to remove it.

Step 5: Delete the message from the queue

It is important to keep in mind that receiving a message from the SQS queue doesn’t automatically delete it. Any other consumer can also retrieve the same message once the VisibilityTimeout period expires. To ensure, no other consumer retrieves the same message, it needs to be deleted within the VisibilityTimeout time period.

We will be using delete_message to delete the message from the SQS queue.

• We need to provide the ReceiptHandle as an argument to the delete_message method.

That’s it, our SQS queue operation is ready, Now we need to integrate both our application and SQS operation file with the API.

To create API, I am going to use Flask today.

Step 6: Create an API

The API is self-explanatory.

• First, we are getting user input using an HTML Form
• And then sending it to the queue
• Then receiving that message from the queue
• Processing and rendering the scan results in a new HTML template.

In order to get the user input via HTML form, first, we need to create the form class.

Step 7: Create a Form class

For this example, I took a simple text box(where the user will enter the input) and a submit button.

Now create an HTML template where the user input and result will be shown.

HTML template to show the results

That’s it, Now run the API.py file

Sample results

This is how you implement the SQS queue in your application.

Alright!!! Let’s explore a quick summary.

• First, we created our application
• Then we created SQS operations
• Then integrated both files with our API
• To get the user input we created a Form class
• To show the results we created HTML templates
• Finally, we scanned for a domain name and got the result.

If you want the full code, you can find it here.

GitHub - UVvirus/aws_sqs_implementation

That’s it… signing off for today…will meet you soon with another great article.