OSINT Tool to know

In the Previous Blog, we know What is OSINT now we have to know what tools are used to do OSINT.

This blog gave you insights about those tools. So, let’s have a look into it.

Shodan

Shodan Seach Engine

Shodan is called The Google of IoT Devices. Shodan is a search engine, like Google, but instead of searching for websites, it searches for internet-connected devices — from routers and servers to Internet of Things (IoT) devices. Security experts use Shodan to analyze network security.

Shodan works by requesting connections to every imaginable internet protocol (IP) address on the internet and indexing the information that it gets back from those connection requests.

Shodan crawls the web for devices using a global network of computers and servers that are running 24/7.

An IP address is your device’s digital signature — it’s what allows Google to tailor searches to your location, and it’s what allows all internet-connected devices to communicate with each other

Internet-connected devices have specific “ports” that are designed to transmit certain kinds of data. Once you’ve established a device’s IP address, you can establish connections to each of its ports. There are ports for email, ports for browser activity, and ports for printers and routers — 65,535 ports in all.

Shodan works by “knocking” at every imaginable port of every possible IP address, all day, every day. Some of these ports return nothing, but many of them respond with banners that contain important metadata about the devices Shodan is requesting a connection with.

Censys

Censys helps organizations, individuals, and researchers find and monitor every server on the Internet to reduce exposure and improve security.

Attack Surface Management (ASM)

At Censys, we can see it all. Our world-leading attack surface management platform gives organizations the sixth sense — relentlessly monitoring assets, seeing the unseen, and proactively giving security teams an opportunity to solve issues before they have a chance to take place.

This isn’t security by the defense. This is a system of vigilant offense that constantly looks at everything from HTTP hosts to message brokers to remote desktop exposure to network printers. Seeking potential breaches, shoring up leaks in your protocols, and mapping any potential weak points.

Discover & Investigate Internet Assets

Including, hosts, services, SaaS logins, websites, buckets, and ICS/IoT devices — regardless of cloud, account, network, or location for the ultimate system of record.

Conduct Rapid Response

Rapidly identify and secure Internet assets that may be exploited by a critical vulnerability.

Prioritize & Remediate Risk

Uncover, prioritize, and remediate critical risks (e.g., potential data loss, critical vulnerabilities, exposed devices/APIs/logins) within hours of coming online.

Identify Cloud Exposure

Pinpoint weaknesses in your cloud across all providers.

Manage Acquisition, Subsidiary & 3rd-Party Risk

Understand security risks associated with uncontrolled companies — acquisitions, subsidiaries, contractors, and other dependencies.

ZoomEye

Zoomeye It’s one of the search engines that allows you to search for devices, sites, and services. it’s a hacker search engine There was a lot of high-profile news related to Zoomeye on the web. The search engine has convenient functionality, flexible search settings, and its own API.

Zoomeye is a search engine developed by Chinese security company Knownsec Inc. The first release was released in 2013. ZoomEye uses Xmap and Wmap at its core to collect data from open devices/web services and for fingerprint analysis.

It can continually scan and identify numerous service ports and protocols 24 hours a day and finally map the entire or local cyberspace, thanks to a massive number of global surveying and mapping nodes that use worldwide IPv4, IPv6 address, and website domain name databases. You can also check the vulnerability impact assessment and search for specific themes. Databases, industries, Blockchain, firewalls, routers, network storage, cameras, printers, WAFs, network storage,

Maltego :

Maltego makes complex OSINT easy with great maps and transforms. Maltego is software used for open-source intelligence and forensics. Maltego focuses on providing a library of transforms for the discovery of data from open sources and visualizing that information in a graph format. Maltego is one of the best information gathering and data mining tools.

In fact, you can gather information on just about anything — people, chemical weapons, IP addresses, terrorists, bank account numbers, etc.… Maltego uses transforms to fetch the information required. The Transform Hub is a large number of websites where data is fetched (e.g., Shodan, VirusTotal, etc…). You have to manually install each transform in most cases as they don’t come pre-installed. Further, transforms are pieces of code that take an input and chuck out a visual output that is related to the input in a particular manner. The data mining is then rendered visually on a blank canvas. Maltego contains hundreds of transforms. And as such, you can sift through data in real-time.

The Harvester

We can call the harvester OSINT for Networks. the harvester is a simple to use, yet powerful tool designed to be used during the reconnaissance stage of a red team assessment or penetration test. It performs open source intelligence (OSINT) gathering to help determine a domain’s external threat landscape. It is used to find email accounts, subdomain names, virtual hosts, open ports/banners, and employee names related to a domain from different public sources (such as search engines and PGP key servers).

Furthermore, it will help paint a picture of just how big the target’s Internet footprint is. In addition, it’s useful for organizations who want to see how much of their employees’ information is available to the public on freely accessible web pages.

Essentially, given certain criteria, the Harvester will run around on the Internet as your surrogate, snatching up any and all information that fits certain criteria. I would also like to point out one more thing before moving forward. This tool can be used to gather email addresses, which could be incredibly useful to an attacker trying to crack online login credentials or gain access to an individual’s email account.

Recon-ng

Recon-ng is a full-featured Web Reconnaissance framework written in Python. Complete with independent modules, database interaction, built-in convenience functions, interactive help, and command completion, Recon-ng provides a powerful environment in which open source web-based reconnaissance can be conducted quickly and thoroughly.

Where we use Recon-ng

• Recon-ng is used to detect the target’s IP addresses.
• Recon-ng can be used to find sensitive files such as robots.txt
• We used Recon-ng to detect Content Management Systems (CMS) using a target web application.
• Recon-ng contains several modules which we can use to gather information about the target.
• Recon-ng port scanner modules find closed and open ports that are used to maintain access to the server.
• Recon-ng subdomain finder modules are used to find subdomains of a singer domain.
• We can use Recon-ng to find information related to the Geo-IP lookup, port scanning, Banner grabbing, Sub-domain information, DNS lookup, reverse IP using WHOIS information.
• We can use Recon-ng to look for error-based SQL injections.

UserSearch :

UserSearch is the largest free, reverse lookup tool online. UserSearch is a huge network of search engines present on the web which find an exact match of a username or email address. It has an exhaustive list of services, such as lookup for emails, forum users, crypto site users, dating site lookups, and adult website lookups. It can even analyze a webpage and pull out all the emails on that page.

Mitaka

Mitaka is used for finding IPs, MD5, ASNs, and bitcoin addresses.

Mitaka is used for identifying malware, determining the credibility of an email address, and finding if a URL is associated with something wrong. You can select and inspect certain pieces of text and indicators of compromise (IoC) that one thinks may be suspicious, by running them through a variety of different search engines, all with just a few clicks here and there.

Mitaka is used to perform In-Browser OSINT investigations to identify Malware, Sketchy Sites, and Shady Emails and help individuals get aware and secure.

By using Mitaka you can do these things:

· Email Address inspection

· Suspicious URL

· Malware Analysis

· Checking If Site Is Sketchy

Spiderfoot

Spiderfoot is an OSINT tool that automates the whole OSINT process. It automates a huge number of queries that would take a long time to do manually. As there are tons of data available on the web on different services, networks, and protocols… gathering all this information from every single place and one at a time becomes a pretty time-consuming task.

That’s when SpiderFoot comes to help, as it can be used to automate the OSINT gathering process to find anything about your target, centralized in one single tool.

To automate OSINT, Spiderfoot queries over 100 public information sources and processes all the intelligence data from domain names, email addresses, names, IP addresses, DNS servers, and much more.

Specify the target, choose the modules to run, and Spiderfoot will perform the whole job for you, collecting all the data to build a full profile of anything you are investigating.

Spiderfoot main features

• Open Source: This security tool was written in Python, and it’s hosted on Github. The best part is that it is open source, which means anyone can contribute to making it better.
• Multi-platform: Spiderfoot can be run on both Linux and Windows operating systems.
• Web interface: By default, Spiderfoot can be run from a CLI interface, however, it also supports a cool web interface for those who want the ease of use, fancy icons, and rich graphic visualizations.
• Module support: it works including more than 100+ modules, which can help to run almost any kind of test against the target network. SpiderFoot modules were programmed to interact with each other, allowing all related modules to share the same data about the target.
• Documentation: unlike other OSINT tools, Spiderfoot was not only well written in terms of code, but it also has a brilliant documentation area that will allow you to discover, read, and understand how everything works including the installation process, and usage, modules, etc.
• Spiderfoot HX: while the standard version will work from any environment, you can also choose to run Spiderfoot from its own self-hosted cloud platform, which includes more advanced features than the self-hosted version.

Spyse

Spyse is a cybersecurity search engine for finding technical information about different internet entities, business data, and vulnerabilities. It’s an all-in-one platform for fast and effortless reconnaissance without using any additional tools.

One of the features that make Spyse so unique is its data gathering. Spyse uses 60 servers around the world to gather data. Placing these servers in a geographically distinct area avoids rate, geolocation, and ISP blockage. Spyse uses qualitative data gathering with 38 self-developing scanners that unite their data into a single scanning pipeline.

Spyse also performs web spidering on the target domain, therefore, information such as the links, robots.txt files, and HTTP headers can also be retrieved. This can aid in fingerprinting the existing technologies in use by the website in scope, identification of sensitive URL’s and mapping the application. Spyse has also the ability to discover other domains that exist on the same IP address. This is a common finding in penetration test reports since multiple domains on the same host increase the attack surface.

Spyse can also perform vulnerability discovery by identifying open ports and matching the port discovery with a CVE (Common Vulnerabilities & Exposures) number. The search functionality also allows users of the service to search by CVE number:

Intelligence X

We can actually call Intelligence X a database of literally everything OSINT.

Intelligence X is a first-of-its-kind archival service and search engine that preserves not only historic versions of web pages but also entire leaked data sets that are otherwise removed from the web due to the objectionable nature of content or legal reasons. Although that may sound similar to what the Internet Archive’s Wayback Machine does, Intelligence X has some stark differences when it comes to the kind of content the service focuses on preserving. When it comes to preserving data sets, no matter how controversial, Intelligence X does not discriminate.

Searchcode

Searchcode is also known as The Google of Code. Searchcode is a unique type of search engine that looks for intelligence inside free source code. Developers can use Searchcode to identify problems related to the accessibility of sensitive information in the code.

The search engine works similar to Google, but instead of indexing web servers, it looks for information between the lines of code of running apps or in apps in development.

The search results can help a hacker identify usernames, vulnerabilities, or flaws in the code itself.

Metagoofil

Metagoofil is an information-gathering tool used for extracting metadata of public documents of the targeted company or organization that are readily available on websites.

It is used for extracting information from documents like PDF, DOC, XLS, PPT, ODP, and ODS that are found on the target’s webpage or any other public site.The tool uses Google to find the documents, then downloads them to the local disk, and extracts all metadata.

Metagoofil analyzes the metadata of these documents and collects a good amount of information. It can find sensitive information like usernames, real names, software versions, emails, and paths/servers.

Hackers can use Metagoofil to gather username information and perform easier brute-force attacks. It can also show a hacker the file paths that can reveal OS, network names, shared resources, and more.

Tineye

TinEye is another excellent tool that helps to do an image-related search on the web. You can even check if the image is available online and where that image appeared. It uses machine learning, neural network, and watermark/pattern recognition to look for similar images on the web rather than keyword matching. This tool helps to search images on social media websites using the reverse image search option

The image search can be made using an image URL or an image itself. Instead of accessing web applications repeatedly, API and browser extensions can also be used to look for a particular image directly.

Exiftool

Exiftool is a free and open-source software program that is used to read, write and update metadata of various types of files such as PDF, Audio, Video, and images. It is platform-independent, available as a Perl library as well as a command line application. Metadata can be described as information about the data such as file size, date created, file type, etc. ExifTool is very easy to use and gives a lot of information about the data.

Exiftool is a tool with an enormous array of features able to work with over 23,000 tags over 130 different groups. You can even define custom tags too!

OpenVAS

OpenVAS or Open Vulnerability Assessment System is a Pen-testing framework whose collection of tools allows you to scan and test systems for known vulnerabilities. OpenVAS uses a database containing a collection of known exploits and vulnerabilities.

OpenVAS consists of:

• A database comprised of results and configurations
• A Scanner that runs various Network Vulnerability Tests
• A Collection of Network Vulnerability tests
• A Greenbone Security Assistant, a web interface that allows you to run and manage scans in the browser

It is to perform comprehensive security testing of an IP address. It will initially conduct a port scan of an IP address to find open services. Once listening services are discovered, they are tested for known vulnerabilities and misconfiguration using a large database. The results are compiled into a report, including detailed information regarding each vulnerability and notable issues discovered.

Once you receive the results of the tests, you will need to check each finding for relevance and possibly false positives. Any confirmed vulnerabilities should be re-mediated to ensure your systems are not at risk.

Nmap

Nmap means “Network Mapper” and is one of the most popular and widely used security auditing tools. Nmap is a free and open source utility utilized for security auditing and network exploration across local and remote hosts. This would be the first program an attacker would use to attack your website or web application.

Nmap operates by detecting the hosts and IPs running on a network using IP packets and then examining these packets to include details on the host and IP, as well as the operating systems they are running.

The main features of Nmap include port detection (to make sure you know the potential utilities running on the specific port), Operating System detection, IP info detection (includes Mac addresses and device types), disabling DNS resolution, and host detection.

WebShag

Webshag (the Free Web Server Audit Tool) is a multi-threaded, multi-platform web server audit tool. Written in Python, it gathers commonly useful functionalities for web server auditings like website crawling, URL scanning or file fuzzing.

Webshag can be used to scan a web server in HTTP or HTTPS, through a proxy and using HTTP authentication (Basic and Digest). In addition to that it proposes innovative IDS evasion functionalities aimed at making the correlation between requests more complicated (e.g. use a different random per request HTTP proxy server).

BuiltWith

BuiltWith is an OSINT which used to find what sites are built with. If you need to gather the technical details on a particular website you can use this tool. This OSINT tool is a website profiler, competitive analysis, and business intelligence tool. You can use their free search engine to analyze the technology profile of a website. Just enter a domain name and click search to find out which content management system (CMS) or content framework a site is using, including widgets and plugins.

Wappalyzer is one of the other alternatives in case you won’t use BuiltWith.

Grep.app

How do you search across half a million git repos across the internet? Sure, you could try individual search bars offered by GitHub, GitLab, or BitBucket, but Grep.app does the job super efficiently.

Grep.app can also be useful when searching for strings associated with IOCs, vulnerable code, or malware (such as the Octopus Scanner, Gitpaste-12, or malicious GitHub Action cryptomining PRs) lurking in OSS repos.

HaveIbeenPwned

HavIbeenPwned is one of the go-to tools for every security researcher. It maintains a database of leaked passwords, emails, and phone numbers over the past few years.

Security breaches are happening very frequently recently, which demands security professionals to be vigilant all the time. HaveIbeenPwned can be of great use when you need to check if the latest public breach has your credentials or not.

One of the attractive features in HaveIbeenPwned for the organization is Domain search. It will notify the domain owners every time the system finds data in the breach against a specific domain. But to use that feature, one has to first verify that he owns that particular domain. After that, HaveIbeenPwned will send automated emails every time the system finds data in the breach.

BeenVerified

BeenVerified is a market-competitive application that looks through individuals in open web records. It’s pretty helpful to inquire about more important data regarding any individual on the planet when leading an IT Security investigation to find details regarding an obscure individual.

When you are done, the search results will consist of every individual that matches the given name, alongside their subtleties, geographic area, telephone number, and so on. After the system gathers the results, BeenVerified will generate a comprehensive report of the data.

The incredible thing about it is that BeenVerified additionally incorporates data from government sources and criminal records.

BeenVerified’s data sources incorporate data from different data sets, liquidation records, vocation history, web-based media profiles, and surprisingly online photographs.

So, BeenVerified can serve as your first stop when you’re on a hunt to get specific information about an individual that is available on the internet

Google Dorks

Search engines do provide us with a lot of information, and Google is popular of all, which is used to gather information about a target. Google Dorks is one of the most useful open-source intelligence tools, which provides such information using some amazing operators. Dork helps you to make more targeted searches efficiently.

Google Dorks are often referred to as GHDB (Google Hacking Database) and are specifically intended for pen-testers during the information-gathering phase.

The following are some of its operators:

• Ext: This is used to define what file extension to look for specifically.
• Intext: This is used to find certain text on a page.
• Filetype: This is used to find specific file types that a user needs to look for.
• Inurl: This is used to retrieve web pages with a certain text in their URLs.
• Intitle: This is used for retrieving web pages that have a certain text in their title.

Search engines also index log files and dorks can access them to find vulnerabilities and hidden data.

CheckUserNames

Social media holds a lot of data, so looking for a username on different major social networks is a time taking task. By using CheckUsernames, you can perform a search for a username on several social networks at the same time. It will search for a particular username on more than 160 websites. The users can check the presence of a target on a specific site to make the attack more targeted. This tool is very useful for cybersecurity kind of activities

So, finally, it’s a quick and useful tool for checking your brand across major social networks, in one simple search.

Creepy

Image sharing is one of the most used features of social media platforms. But while sharing sometimes we even share the exact location of where the image was taken. Creepy is a python application tool that extracts that details and presents the geolocation on a map. It collects geolocation data through various social networking platforms and presents reports as a search filter based on the exact date and location.

Creepy supports the search for Twitter, Flickr, and Instagram, and reports can be downloaded in CSV or KML format for additional analysis.

Fierce

Fierce, a fantastic network mapping, and port scanning tool. Often used to locate non-contiguous IP space and hostnames across networks, Fierce is far more than just a simple IP scanner or a DDoS tool. It is a great reconnaissance tool that is used by whitehat communities all over the world.

Fierce is specifically designed for corporate networks and is used to discover likely targets in the networks of their systems. Capable of scanning for domains within minutes, Fierce is becoming the preferred tool for performing vulnerability checks in large networks.

Some of its defining features include:

• Performs reverse lookups for the specified range
• Internal and external IP ranges scanning
• Capable of performing an entire Class C scan
• Enumerates DNS records on targets
• Excellent Brute force capabilities coupled with reverse lookups should brute force fails to locate multiple hosts
• Name Servers discovery and Zone Transfer attack

How Fierce performs Scanning

Despite being such a resourceful and effective recon tool, it’s working is relatively simple. It starts the scanning process with brute force attacks if it is not possible for it to readily perform zone transfer of the target domain. Fierce uses a predetermined wordlist that contains possible subdomains it can detect. If a subdomain isn’t on the list, it will not be detected.

Foca

FOCA stands for Fingerprinting Organizations with Collected Archives. FOCA is a tool used mainly to find metadata and hidden information in the documents it scans. These documents may be on web pages and can be downloaded and analyzed with FOCA.

It is capable of analyzing a wide variety of documents, with the most common being Microsoft Office, Open Office, or PDF files, although it also analyses Adobe InDesign or SVG files, for instance.

These documents are searched for using three possible search engines: Google, Bing, and DuckDuckGo. The sum of the results from the three engines amounts to a lot of documents. It is also possible to add local files to extract the EXIF information from graphic files, and a complete analysis of the information discovered through the URL is conducted even before downloading the file.

ReconSpider

ReconSpider is the most Advanced Open Source Intelligence (OSINT) Framework for scanning IP Addresses, Emails, Websites, and Organizations and finding out information from different sources.

ReconSpider can be used by Infosec Researchers, Penetration Testers, Bug Hunters, and Cyber Crime Investigators to find deep information about their target.

ReconSpider aggregate all the raw data, visualize it on a dashboard, and facilitate alerting and monitoring of the data.

Recon Spider also combines the capabilities of Wave, Photon, and Recon Dog to do a comprehensive enumeration of the attack surfaces.

Why it’s called ReconSpider ?

ReconSpider = Recon + Spider

• Recon = Reconnaissance

Reconnaissance is a mission to obtain information by various detection methods, about the activities and resources of an enemy or potential enemy, or geographic characteristics of a particular area.

• Spider = Web crawler

A Web crawler sometimes called a spider or spiderbot and often shortened to a crawler, is an Internet bot that systematically browses the World Wide Web, typically for the purpose of Web indexing (web spidering).

Overview Of The Tool

• Performs OSINT scan on IP Address, Emails, Websites, and Organizations and find out information from different sources.
• Correlates and collaborates the results, and shows them in a consolidated manner.
• Use specific script/launch automated OSINT for consolidated data.
• Currently available in only Command Line Interface (CLI).

Any type of comments are welcome. Thank you for your time :)).

Happy Hacking !!!

If you enjoyed reading the article do clap and follow:

Twitter: https://twitter.com/i_amsphinx

LinkedIn: https://www.linkedin.com/in/pathakabhi24/

GitHub: https://github.com/pathakabhi24