What is an easy way to discover all of my external facing systems?

Problem: I’ve just taken over the Internal Security Engineer position at a company. We’ve never gone through a security audit before. We know which network ranges we own. However, we have no idea what our actual external-facing exposure is. There are some documented systems, but for all I know, people have been building and putting things on the internet for years and forgotten about them. Beyond that, what is even running on my IPs? What type of web applications do I have on them, and what is their exposure level? Even once we have our own ranges figured out, what about all the cloud services my team seem to be using? We have company data on them, but are they part of our “external” network?

Introduction:

A situation like this involves not only technically implementing a solution, but applying soft skills and teamwork in your organization. The process may not necessarily be hard, but it becomes an involved and continuous process to maintain up-to-date knowledge of your external facing systems inside of your organization.

The blog breaks down the solution into five steps:

Know Yourself — Identify the assets important to the business.

Know Your Team — Identify where the assets sit on your network and other services provided by your organization through information gathered from various departments of your organization.

Know What the World Knows — Attempt to find your public-facing systems. DNS reconnaissance and Network scanning are the main techniques applied in this blog posting.

Know How to Keep It All Together — Organize the information obtained from the previous steps.

Know What Lies in The Cloud — Determine your responsibility with respect to external assets and information hosted by a third party.

Know Yourself:

Discuss with your marketing team, business leads, etc., and establish what your assets are. What does the company do? What is their main source of revenue? This exercise will not only help identify and prioritize key assets and get an idea of what possible services are on your network, but you will have also established a communication channel with the non-technical division of your company.

[caption id=”attachment_1727" align=”aligncenter” width=”582"]

Image 1- Identifying possible assets from business a perspective[/caption]

Know Your Team:

Once you have the assets you would like to investigate in mind, find your organization’s Network Engineer(s) and ask them for their help. They may have more experience with the network and can help you answer various questions such as:

• How many nodes/devices do we have on the network?
• Does our network segregate traffic from our servers and workstations?
• If so, how is it implemented? I.E. with virtual local area networks (VLANs)?
• What is our current routing set up?
• What is our current domain name (DNS) setup?
• Do we have any firewalls or intrusion detection/prevention systems in place?
• If we do, what are our current policies for them?
• How are we keeping track of device information, especially on routers?
• How are we logging all that information?
• Are we implementing Simple Network Management Protocol (SNMP) to monitor the network?
• If so what information are we logging?

There are more questions that could be asked, but this gives you a rough idea of what information you should be aiming to obtain. Things like routing information, number of devices/nodes on the network and a rough idea of network flow. Again a discussion like this offers multiple benefits: you are getting to know the colleagues you will be working with to help implement your security policies and also creating a communication channel for when you do send a request to modify a firewall policy, request SNMP log information regarding open services, or have certain network traffic monitored.

[caption id=”attachment_1729" align=”aligncenter” width=”582"]

Image 2 — Identifying internal network infrastructure from networking perspective[/caption]

The next individual or group in your organization to talk to would be your System Administrator(s). If you now have a rough idea of how many devices are on your network and how the traffic flow should be, your System Administrators can help you determine what services are running on those systems and what servers and clients you may have on the network.

Things you may consider asking your System Administrators:

• Do we have Windows servers, Linux servers, Solaris servers, etc.?
• Which services/applications do we have running on these servers?
• How are we managing these servers?
• Are we recording the information anywhere?

As usual, there are many more questions that could be asked and discussed with your System Administrator(s), but the main concerns are what operating systems you are running, what possible services may be installed, and how they are managed. This type of information is not only useful for the initial external scanning, but will be helpful when implementing solutions to any issues that my come up in the future.

[caption id=”attachment_1730" align=”aligncenter” width=”582"]

Image 3 — Adding information obtained from administrative perspective to gain a good understanding of organizational network[/caption]

Know what the World Knows

Now we get into the technical aspect of investigating our organization’s external network. This consists of basic domain name system (DNS) reconnaissance, host discovery, and port discovery.

DNS Reconnaissance:

One of the initial public facing systems you should check on is DNS; how are your organizations IP addresses getting resolved? To properly answer this question, you need to investigate your DNS servers externally. A few things you might want to investigate are:

• What initial information do you obtain from a DNS lookup on the domain name of your organization?
• Can you request a zone transfer remotely?
• What are the mail servers, websites or addresses associated with your name, and are they yours?

These questions can be answered with various tools native to Linux. Since this is a blog post about ease, I will demonstrate a useful tool called Fierce, installed by default on Kali Linux — a Debian based system with a suite of tools useful for penetration testing. Fierce is a lightweight enumeration scanner based on DNS, Whois, and ARIN queries.

You can run a basic scan which will attempt to resolve the IP of a hostname, attempt a Zone Transfer, attempt to check if you have a wildcard DNS, and brute force possible affiliated websites.

Some of the output can be seen below and this can be called with a simple command line option of fierce –dns <hostname>.

[caption id=”attachment_1731" align=”aligncenter” width=”582"]

Image 4 — Output of DNS Enumeration through the use of the Fierce command line tool[/caption]

You would want to confirm this information with other tools such as Dig (domain information groper). Dig queries DNS servers with various requests. For example, it can make requests for any record information associated with the domain name.

This can be done with the command line option: dig <domain name> any

[caption id=”attachment_1732" align=”aligncenter” width=”582"]

Image 5 — Output of a DNS query through the use of the Dig command line tool[/caption]

You will want to confirm the output by manually checking the addresses because, even though they resolve to an IP address based on the information obtained previously, they may be only accessible on the internal network. With the information obtained, you can determine if those servers lie in the IP range known to your company, if the DNS configuration is providing too much information, or if one of your resolved names should no longer be associated with your organization.

Host Detection:

Now, with the set of IP ranges provided and information gathered through tools like fierce, you may conduct a host detection scan with tools like Nmap. Nmap is a Network Mapper utility that performs network discovery and security auditing. Nmap uses crafted IP packets to solicit a response from a target in various ways to determine: if the host is up, what ports are open and, in most cases, what services are listening on those ports. Nmap has many other features but for this blog post, those are the options we are interested in.

With the range of IP addresses provided you would perform a scan similar to:

nmap –sn <Ip Range> -oA <save_file_name>

The ‘sn’ option tries a Ping Scan which consists of an ICMP echo request, TCP SYN scan to port 443, a TCP ACK packet to port 80 and an ICMP timestamp request. This is a quick way to determine the number of possible targets based on the IP range received. The ‘oA’ option saves the output to various formats such as XML, text output, and ‘greppable’ output.

[caption id=”attachment_1733" align=”aligncenter” width=”582"]

Image 6 — Output of an Nmap host discovery scan[/caption]

A recommended option would be to establish a dedicated external server to scan the network from. If you have any Firewalls or IDS in place, consider whitelisting the target IP. If your IP range is large or the subnet is larger than a class C subnet, consider breaking the scans up — maybe half the ranges for one day, and the other half another day. Consider setting up a scanning policy.

Port Detection:

Now that you have a list of possible targets that may be up, you may conduct a more involved scan using the same tool — Nmap — to determine what ports and services are running on the hosts that are listed as “up”. Keep in mind that these scans may take from several minutes to several days, depending on the transport protocol you select, the number of hosts on the network being scanned, and number of ports to be scanned. With the knowledge obtained from communicating with your Network Administrators and System Administrators, you may want to scan the ports you think will be open first, for confirmation, then do a thorough and prolonged scan for verification and completeness.

Ideally you may want to run four sets of scans, one to determine what TCP ports are open and maybe apply a simple banner grabbing technique to determine possible port information. You also want to perform UDP port scanning even though it takes noticeably longer because it is a connectionless protocol. The ports to scan will depend on previous knowledge obtained from your Network and System Administration teams.

An example of a TCP scan would be

nmap –sS <host/host range> — top-ports=1000 –script=banner

[caption id=”attachment_1734" align=”aligncenter” width=”582"]

Image 7 — Output of an Nmap TCP port discovery scan[/caption]

The UDP Scan would be quite similar

Nmap –sU <host/host range> — top-ports=1000 –script=banner

[caption id=”attachment_1735" align=”aligncenter” width=”582"]

Image 8 — Output of an Nmap UDP port discovery scan[/caption]

These types of scans allow you to confirm the information provided in the “Knowing your team” exercise, and review the output, keeping in mind such questions as:

• If this is our main webserver do we really need to have these many services running on it?
• Do these services need to be public facing?
• If we can’t immediately turn off those services, can we talk to the Network Administration team to update the current Ingress and Egress rules in the Firewall policy?

After those scans, time permitting, a more involved scan can be done by running Nmap’s basic scripts for host detection and further port analysis. These types of scans may be directed at a specific port to learn more about what information an attacker could obtain about the service.

An example would be the command nmap –sV –A <host/hostrange> -p 80,8180

These ports were identified as web service or “unknown” in the previous scan.

[caption id=”attachment_1736" align=”aligncenter” width=”582"]

Image 9 — Output of an Nmap service discovery scan[/caption]

With this information, you can now confirm that both are indeed web services and with the Apache version information listed, you can determine if the services are following proper patch management security policies. Further analysis could be done with web specific vulnerability scanners and by manually investigating the application. However that sort of discovery is too detailed for this blog posting; we are focusing on discovering your public facing footprint.

In this section we applied technical techniques to map our external facing network but it is just as important to have this information organized in some scalable form. It is important to ensure the changes are recorded as scanning continues — either weekly or bi-weekly or however your policy deems fit.

Know how to keep it all together

Meaningfully logging and tracking information about your public facing servers is just as important as discovering what the world knows. There are multiple ways do this.

One of the ways I have seen to be effective for small data sets is to parse the information in Comma Separated Values (CSV) format and add it to a spreadsheet with various pages based on scan date. You may need to do some scripting to parse the information in a way that is meaningful to you. An example would be to use PowerShell (Microsoft scripting language) to parse the xml output of an Nmap scan and convert it to csv format with meaningful headers. Luckily, there is an old script developed by Jason Bolson to convert XML (Extensible Markup Language) Nmap output to various PowerShell Objects which you can modify to obtain the output that you need. Create subsequent scripts to convert multiple scans to and from these objects and output the format as CSV to be used in a spreadsheet.

An alternative way is to utilize the Zenmap tool the Graphical User Interface (GUI) to Nmap. With your recorded scans, you can easily determine what changes have occurred on your network by using the built in difference compare command located in the tools section of Zenmap.

[caption id=”attachment_1737" align=”aligncenter” width=”582"]

Image 10 — Output of Zenmap Compare Results tool[/caption]

The previous image compares the output of two scans using the Zenmap GUI. The green lines represent additions from the second scan and the red lines means the newest scan does not have the highlighted result. The files used to compare were a TCP scan and a UDP scan to emphasize the changes.

Multiple methods can be used to keep track of your scans — just ensure that the data is easily retrievable and presentable when the moment requires it.

Know what Lies in the Cloud

Understand your relationship with your Cloud Provider. This means:

• Identify what type of cloud service your organization is running.
• Ensure you are aware of the relationship your cloud service provider has to you.
• Determine whose responsibility it is to apply a proper security policy for the application or service.

A lot of organizations utilize the cloud in some way. The external service or application is still considered a public-facing entity of your organization. The level of responsibility you have for those services changes based on the type of service you are using. For example: are you using Infrastructure as a Service (IaaS), Software as a Service (SaaS) or Platform as a Service (PaaS)? In most cases the responsibility is shared for both the Cloud Service Provider and you as the client. An excellent reference point would be the PCI DSS Cloud Guidelines on page 11, which provide an excellent chart breaking down specific responsibilities such as applying a Firewall. In this situation, communication with your Cloud Service Provider is key. In all scenarios, it is still your responsibility to ensure due diligence is done to secure data left on any Cloud Service.

Conclusion

The easiest way to address public-facing systems is to take the challenge in stages:

• Identify the assets of the organization.
• Identify the individuals that will aid you in understanding the business goals of the organization, the network infrastructure and the services and systems used in the organization.
• Verify the information obtained through DNS recon with such tools as dig, and fierce.
• Establish a routine for constant scanning of possible hosts on the networks and the ports that are publically available.
• Record this information in whatever format you find easily scalable (e.g. as spreadsheets or in databases)
• Identify your cloud providers, communicate your interests as an organization, and understand their obligation is to you.
• Rinse and Repeat.