Practical approach to Open Source Intelligence (OSINT)
Earlier we talked about OSINT and what it is. Today we will look into practical considerations involved in typical OSINT tasks as well as walk through some examples.
OSINT processes are not just analysis of data. Many times OSINT techniques are investigations and they are run as such. This means that there is no single method involved that will churn out intended results like a mathematical process. A lot of times, further decisions are made by analyzing the current information and connecting the dots based on the real world meaning the information represents. In general, a typical OSINT task will go through the following phases:
- Planning and direction: Defining a target and clear objectives so as to focus the scope of the operation.
- Data collection: This is an essential part of any OSINT project. (This blog too will mainly be concerned with this part.)
- Processing and analysis: Assessment of data needs to be carried out in line with the goals of the project. Connecting the dots to as to generate some form of intelligence is the prime concern at this stage.
- Dissemination of information: This is not the most talked about the step of the process. But the intelligence obtained needs to be structured properly. Many times some proof of work is demanded and efficient note taking tools, capturing screenshots, and presentation of results becomes important for completion of an OSINT job.
Foundational concepts:
Here we will discuss some of the basic practices followed by OSINT practitioners. These practices can be followed regardless of the problem at hand. Some of these topics require discussions of their own.
- Prepare your computer: General OSINT tasks do not require high-performance or CPU/GPU-intensive machines. But attention needs to be paid to the environment in which these tasks will be performed. Primarily because of security reasons MacOS or certain versions of Linux are preferred more than others. But whichever operating system is used for the purpose, it is essential to remove all malware from it and keep it protected using antivirus programs, scheduled scans, regular updates, setting up a firewall, or maybe even periodic restores of the whole system.
- Operational Security: OpSec deals with the ideas to mitigate the risks and threats one can have while performing OSINT tasks. For protection, it is highly recommended to make use of Virtual Private Networks (VPN) tools. Also using virtual machines inside the operating system and performing tasks from them is also a good practice to safeguard from a threat actor.
- Web browser: A lot of times, OSINT practitioners are surfing the internet for relevant information. Traversing links, entering passwords, using search bars, etc. on websites are quite a typical routine for an OSINT job. A good web browser comes in very handy for all such purposes. Though this blog does not comprehensively establish a contest amongst browsers, Firefox is considered to be pretty good for the job. It has a wide user base and as such has enough credibility to access websites. Its default configurations are better at offering privacy to even most novice of users. It also comes with a security-focused community base of users and offers a range of extensions to enhance productivity specifically for OSINT purposes.
A few mentionable firefox extensions are as under:
HTTPS everywhere: encrypts communication between major websites and your browser.
resurrect pages: links to archived versions of web pages that are deleted, modified, or unavailable.
SignalHire: extracts contacts on a click from social and professional networks like Facebook, Twitter, LinkedIn, etc.
Nimbus screenshot capture: Supports capturing even while scrolling through web pages. - Sock puppets: Fake accounts or social media aliases are particularly useful to access data that is otherwise available only to service users. The idea behind using these accounts is to maintain online anonymity. With the advent of bot and spam detection tools, it is vital to learn how to create these accounts properly and maintain their usage. Tools like fakenamegenerator and thispersondoesnotexist are very helpful resources in this regard.
With the brief discussion above, we move on to a practical demonstration of some tasks that are frequently performed in OSINT jobs.
Disclaimer! The information in this blog is for educational purposes only. The tools used here are all open source and example scenarios are covered using freely available data. Any illegal or malicious usage of this information is not intended by the author.
Example OSINT tasks
OSINT tasks are really not very different from the regular usage of the internet. With the purpose of collecting information, most of the routine tasks involve performing searches and using tools to either filter through the search results or collect them in simple formats for further viewing and analysis. For example, a typical OSINT roadmap to find contact information of a person would undergo following steps:
- Search Engine OSINT: invoke search engines like Google, Bing, Yandex etc with special operatives like “phone” “email” “address” etc.
- Social media OSINT: run searches on social media platforms like Facebook, reddit etc or get tools that run searches and collect information.
- Breached dataset OSINT: Many websites provide information by searching the leaked or breached datasets. This data is not shared freely but the searches can be made to check if the data against a person appears in specific breaches already.
- People OSINT: Many companies are in the business of maintaining contact information on a person e.g whitepages, truePeopleSearch, fastBackGroundCheck, voter records etc. Their services can be employed for the specific purpose of looking up a person.
Though this is not an exhaustive list of searches, the next step would be to view all results and check them for connections. The connections made could be made to run through again against the search engines. Many times, a concrete contact is not extracted with this practice but still it is a source of leads that furthers an investigation.
OSINT techniques readily make use of tools to facilitate usage of existing web services like search engines and filter results from them. Because of this reason, the tasks are regularly plagued with the idea of a tool becoming obsolete. There could be a number of reasons why a said tool might stop working. That is why there is a continuous effort of sharing and improving tools by the OSINT community. Nevertheless, the reader is informed that the tools used for proceeding examples might become obsolete in said time and their usage here is mostly to demonstrate OSINT methods in the modern world.
Geo-locate a photo
Let us look at an example task to actualise our discussion so far. For this task, I have made use of a kind of challenge that was posted by Jhon McAfee on his twitter. The post is pasted here for reference and the idea is to find the address of the place seen in the photo.
Immediately, a few things to realize in the photo are as follows:
- Photo is taken at a petrol station which has red and white in its branding and possibly blue petrol dispensing units.
- The destination of McAfee’s journey is London.
- Also because of the nature and type of truck (flat-nosed) in the background of the picture, the road trip seems to have been somewhere in Europe at the time of the photo.
- A few other things to note are in the background. Clear horizon, House/building with a pointy roof and a pole in the front.
As a first step, let us run an image search on the photo. The image search did not bear good results and the results are not included here. Possibly because of the faces in the photo, search results are not homing in on location or background in the image which might have been more helpful for our purpose. Nevertheless, making an image search on google or yandex is a good step to learn about the image and especially about its sources.
Moving forward, we still have features in the image to inquire about. First up is the petrol station. The photo itself does not relate any brand logo or name but we can make a search based on the features we can see.
A likely candidate of our search is highlighted in yellow. Upon exploring the link, it becomes quite convincing that the petrol station is most likely an Esso petrol station.
Having knowledge that McAfee’s destination is London, it would be interesting to see if we can find where he started his journey from. Taking a look at his twitter account, we see a very relatable tweet that could hint us a possible starting location. It seems that just two days prior to McAfee’s post from the petrol station he was somewhere called “hotel schlicker ”. Said tweet from McAfee is pasted under and the logotype is highlighted.
On searching for the hotel, it turns out that there is a hotel schlicker munich whose logotype is very similar to the one in the picture. So we have our first guess for a possible complete story in which McAfee was going from Hotel Schlicker in Munich to somewhere in London and he stopped along the way at an Esso station where he took a photo.
This is good. We just have to scour through the ways from Munich to London by road and look for an Esso station. The problem is that the distance between the two places by road is more than 1000Km. Here is where an intelligent use of open source maps comes into play.
We are very familiar with Google maps to find our route from point A to point B. It is a great tool to do that. We take a guess that McAfee and his team might have done the same. We chart a course from Munich to london.
While Google maps is great, what most people do not realise is that a map as an entity can be for many different things. As long as you can ascribe a reference point in a defined coordinate system to any form of data, that data can be mapped. That is what google does too. Their application is geared to deliver efficient solutions for way finding, satellite view, street maps, traffic congestion etc.
For many use cases, Maps works just fine but if we are interested in things other than wayfinding, say we are interested in Pharmacies at a location, There are other options apart from google maps to look into. I would quickly relay an example here.
In both map views, I have tried to show the Danube river as a reference. You can immediately see that openstreetmap organisation has data that is more fitting for particular needs. In the image “Pharmacy_osm” the query made to get the required data is shown in the left-hand side panel. The right hand side panel presents results as well as define the bounding box area in which the search will be focused. Rather than only using google maps for our McAfee problem, we will try to run queries on openstreetmap’s dataset as well.
Moving forward with our investigation, I ran some queries to look for “nodes” in OpenStreetMap that had the “brand” tag with the value “Esso”. Because the road journey from Munich to London covers large areas, I decided to scroll along the route and use sections of maps, making the same repeated queries on each section. I decided to cut the overall route into as large a section as to whose results were delivered by the overpass tool in less than a minute. Roughly measured, I sectioned the route into stretches of about 250–300 Km each.
For the result of each section, I exported the results and imported them into google maps tool. If you sign in to google, the tool allows importing your own markers and overlaying them on the regular google maps. That is exactly what i did for each section as shown:
Already one can appreciate the picture we are trying to form. Though there are 1000s of Esso petrol pumps or nodes between Munich and London, a few dozens of them follow the route that is highlighted in the picture. If McAfee was at an Esso station, it is more likely that he would be at a station close to the route that is going to London. From this point onwards, it was a gritty work of checking each marker close to the highlighted route against its street or satellite view on google maps.
The effort did pay off though and i was able to identify a possible location close to Munich actually and did not have to surf all sections of the rest of the route. The relevant node is shown as under:
And the corresponding view of the node on Google Earth is:
We can see in google earth’s view that the house/building with a pole at its front, as well as a clear field between it and the petrol station, give us clues that this can be the location. Some visual cues that other people on the internet have ascribed to this location as being the possible location are shown as under:
In conclusion, we can say that the photo was taken from an Esso station at the coordinates 48° 48' 39.6'’N 8° 11' 2.4'’E
I hope you enjoyed reading this blog. In the next blog we will talk about some industry leaders and their recent works.