A Guide to Open Source Intelligence Gathering (OSINT)

Hi Guys, this article is all about how significant a perfect reconnaissance is for bug hunting. From my experience, I have coined a list of some of the best tools and techniques that I use for reconnaissance.

Passive Reconnaissance or OSINT or Information Gathering is to gather information about a target using publicly available information. Extracting relevant information about the target always helps and plays a significant role during bug bounties. The primary tasks during reconnaissance include:

Identification of IP addresses, subdomains, ports and services that can increase our attack surface.
Identification of technologies used, application platform and other infrastructure details
Identification of sensitive information for e.g. API keys, AWS S3 buckets, leaked credentials, etc.
Other data includes identification of Log files, Backup files, Database files, Client-side code, Javascript libraries and Configuration files

Usually one of the primary step during Reconnaissance is to identify the net ranges and sub-domains associated with the target. Search engines like Shodan and Censys can be used to a great deal for finding IP addresses, sensitive files, server information, error messages and other exposed services.

A whois lookup helps to find admin contact and other email addresses. Email addresses can be searched through database leaks or through a search service like HaveIBeenPwned that tells you if your email was found as part of a breach.

This whois site https://ipalyzer.com/ is extremely useful. You can get tons of information about the target like GeoIP information, provider information and can query different spam lists.

Enumerate subdomains using virustotal and Sublist3r. Some other tools like dnrecon, subbrute, knock.py can also be used.

Wappalyzer is also a good utility to uncover technologies used on websites. It detects content management systems, ecommerce platforms, web frameworks, server software, analytics tools and many more. Analyzing HTTP status codes and response headers using burp is a must. Check for CSP headers as they contain domain names and sources from where the script loading may be allowed.

Post this analysis, look out if the target is vulnerable to any publicly available exploits. For exploits and tools search, you can use Sploitus and it is absolutely brilliant!!

It is also imperative to check records about people associated with the target. Gather information about people names, email addresses, social profiles, address, phone numbers etc.

Check the below link for a comprehensive list of OSINT tools and resources:

You should also check the html source of the application and the requests/response using burp and see if the application hosts any of its content in the cloud. Few applications might store images and other static files in S3 buckets. Scan for open S3 buckets using S3 Scanner.

Check the below guide for testing the security of AWS S3.

One should always look for code hosting services like github, gitlab, bitbucket etc. Web vulnerabilities, configuration issues, AWS and other secret keys can sometimes be found in these services. While making commits, developers leave credentials or unique “Access tokens”. It’s easy for attackers to retrieve this sensitive information by using commit logs and checking out specific commits and host an attack on the target infrastructure.

Tools like Github Dorks can be used to to search sensitive data like private keys, credentials, authentication tokens etc in the github repositories.

Google Dorking or an advanced search is also very useful. Exploit db gives an integrated list of Google Dorks https://www.exploit-db.com/google-hacking-database/

In the field of cybersecurity, using the right tools for your OSINT investigation can be really effective if you combine it with critical thinking and have a clear OSINT strategy. Whether you are running a cybersecurity investigation against a company/person or if you are on the opposite side working to identify and mitigate future threats, having pre-defined OSINT techniques and clear goals can save you a lot of time. Every organisation should embrace OSINT as one of the cybersecurity defenses to identify and detect app, services, and/or server threats.

Thanks for reading the article. If you think of any other useful techniques that can be added, feel free to leave a comment below:

Minali Arora