How we set up port scanning to secure our cloud data

In my years of working with distributed systems in a web service based environment, one thing I’ve noticed very frequently (especially in the newer crowd) is the neglect of ports or rather, the closing of ports. Security is of utmost importance at all times, with the prevalence of black-hat hacking and data leaks in this day and age. While ports may not be in and of themselves sensitive resources, they need to be secured and protected on priority.

Let me explain this with a little example. Imagine you’re going out of town, leaving your home locked for a few days. You’ll double check all the doors, windows, balconies, and make sure even the smallest of cracks are covered up. One would argue with you that a small opening between the window and the sill shouldn’t be a problem, why worry over it? No one would happen upon that crack by chance, and even then, there would be neighbors to watch over the house.

You’re not going to buy into that, would you? This is exactly the same situation with our servers hosted on the cloud. Each port is like a window or door into which some mischievous person can peer into and glean information that they shouldn’t know about. Maybe they could find out where a key is kept and potentially ransack all the precious valuables you have there. You may be basking in the sun, sipping your martini, while some hacker is emptying your possessions into his greedy hands.

Which brings us back to port scanning. To prevent the hackers from even seeing your open ports, you can and should regularly check for any open ports you may have used while operating your hosted services, and close them ASAP. However, this is a monstrous task to be done manually, since there are 65k ports for every instance you run. So what do we do? We write a script that can run a periodic check for any vulnerabilities and report them to the concerned authorities.

In this easy step by step guide, we’ll go through how we build such a port-scanner. One of the most acclaimed and well-known port scanner tool built is NMap. It’s a very handy weapon in any hacker’s toolkit. Many systems and network administrators find it useful for tasks such as network inventory, managing service upgrade schedules, and monitoring host or service uptime. Nmap uses raw IP packets in novel ways to determine fine details about systems on the network.

It offers a command line utility to scan any range of ports on any given IP address and return the status of the scanned ports. We’ll use this tool in combination with a python wrapper library ‘python-nmap’ to be used on top of nmap.

As for getting the IP addresses for every VM instance on the cloud, we’ll use a nice command line utility library offered by Google for managing your servers, gcloud. The idea is such that you run the gcloud utility from inside python script to get a list of all your instances from which you can extract their IP addresses. All we have to do is loop over the list of IPs and execute the nmap scan command for each IP in the list.

Since the scanning can take quite some time to cover all ports on all IPs, we can use python parallel multiprocess execution to divide the IPs among processes, most likely equivalent to the number of cores for best efficiency. The way we implemented parallel execution was using a shared mutable list to which each process would append the results. This allowed us to execute all scans within a reasonable time. Once all the results were collected, the only problem remained was to publish the data in a concise and actionable format. To do so, we used the Jinja2 templating engine to populate a simple embeddable HTML page populated with the scan results in a tabulated manner. This HTML output would be mailed to all concerned admins through a python mail client library, sendgrid. This would be configured to be run at a set interval through the day.

Once this was set up all we had to do was to wait for the timely reports to start rolling in. Our main aim was to secure any ports left open on our servers hosted on the internet and this scanner would be acting as our sentinel, watching for any vulnerabilities that hackers could exploit. This port scanner tool was implemented as part of the October Hackathon at Aasaanjobs, where the theme was IT Security. We had a lot of fun developing this and learned a lot, too. I hope you did as well.