Week in OSINT #2019–24
A week filled with Facebook fails, Twitter news and some nice tools and sites
I have been extremely busy the last two weeks running investigations 10 days straight, at least 12 hours each day. So most of this newsletter was written in the last 72 hours or so, and I almost didn’t make it today! During last week a lot of things were going on in regard to Facebook, I also saw some nice links from the community, but didn’t have enough time to include them all. Ah well, there is always a next time right? For this week, the subjects are:
- Buildings of the World
- Dollar for Docs
- Twitter Disinfo Dataset
- Twitter Disinfo FAIL
Site: Facebook
Do you still know here were you were when Facebook changed their graph search? You probably did, and so do many other online sleuths out there.
There has been a lot going on in regard to Facebook searches the last few weeks and this won’t be the last time. According to the changelog of the graph API there have been many changes since April, so the fact that querying directly via the URL would break eventually shouldn’t really have come to a surprise, looking at this huge list of deprecated endpoints!
Tip: Always — I really mean ALWAYS! — check the documentation of Facebook, because they are pretty meticulous when it comes to that! I always read API documentation from whatever website I am looking into. This is where I get most of my knowledge about how things are set up from a technical perspective and I usually find inspiration to find new ways of using those API’s to my advantage.
While playing around a bit with Facebook in the developer mode of Firefox, it becomes clear straight away that a lot of functionality is still around but the way to use it has changed quite a bit, especially for the non-technical people out there. The direct querying in the address bar is gone, but by combining some API endpoints, their values, some JSON formatting and encoding them, there’s a very good alternative and maybe even a more flexible way of working!
Inti de Ceukelaire started adopting his scripts when the changes went live, but it seems he was merely constantly changing to the URL’s that were not affected yet, soon rendering his stalkscan tool defunct. But then D Nemec and Henk van Ess among with some others dove into the way the URL’s were constructed and were able to create a nice set of queries. Thanks to the extensive work of D Nemec there is even a huge list of parameters or endpoints available. Soon after that a user called ‘sowdust’ stepped in and created a Javascript based page that now runs all of the links underneath here, so kudos to them!
In short: At this moment there are a few tools online that use the new way of querying and ALL are using the script created by sowdust. So don’t worry, there’s no need to encode the more difficult query parameters, just fill in the desired information and let it run. Play with it to find out how it works — it’s not that different than before to be honest — and maybe even try to find some new tricks. And if you are looking for more in depth information about the way of working and want some real life examples, then there’s also Henk van Ess' bootcamp early July in Amsterdam.
FB-Search: https://sowdust.github.io/fb-search/
Graph.tips: https://graph.tips/facebook.html (full sowdust copy)
Intelx.io: https://intelx.io/tools?tab=facebook (slimmed down version)
Bootcamp: https://www.linkedin.com/feed/update/urn:li:activity:6544861700514996224/
Site: Buldings of the World
For a Quiztime challenge I sent out the other week (sorry Christiaan for taking your spot!) people needed to locate a billboard somewhere in Africa. One of the ways to solve this challenge was to dive into the phone number and license plates, and then find the buildings in the background. Twitter user ‘mcoumans’ found the site Emporis, that can help you locate high buildings. A second site that popped up again in a Quiztime challenge was Phorio. So again these sites were helpful to solve a seemingly impossible geolocation. So you have a challenging photo with some high buildings in the background? Then think about these websites.
Link: https://www.emporis.com/buildings
Link: https://en.phorio.com/
Site: Dollars for Docs
Another great addition to my newsletter via AccessOSINT, this time it’s a public database by ProPublica with information about any form of payments by pharmaceutical companies towards doctors or hospitals in the U.S. from August 2013 to December 2016.
Site: Twitter Disinfo Dataset
Twitter decided to share information in regard to banned accounts that — according to their investigations — were run by foreign campaigns and were meddling with the elections or other state backed operations to spread disinformation. And it is a vast amount of data too! To give you a little insight, here is a small snippet from the readme that comes with the information:
Each folder includes 6 types of files / folder, comprising the full archive of information disclosed about that information operation:
* _tweets_csv_hashed.zip - all tweets and metadata
* _users_csv_hashed.zip - list of users and profile metadata
* _profile_banner_hashed.zip - profile photos and profile banners. Users with the default Twitter profile pic and/or banner are not included
* _tweet_media_hashed - folder containing tweet media. Within the folder, there are a number of .zip files where each contains a number of users' tweet media (Number of .zip files varies based on the volume of media; See _tweet_media_hashed_README files for details about which users are in which numbered .zip file)
* _tweet_media_hashed_README - file that details which users' tweet media are in which numbered .zip file
* _periscope_hashed.zip - Periscope broadcasts, where each sub-folder contains the users' broadcasts (users without a Periscope account are not included; users with a Periscope account with no broadcasts have an empty sub-folder)Link: https://about.twitter.com/en_us/values/elections-integrity.html#data
News: Twitter Disinfo FAIL
Besides the fact that Twitter is opening up and trying to do its best to share the fruits of their efforts to fight disinformation, there is also the other side of the medal. The fact that authoritarian states — that Twitter itself is trying to fight — are abusing Twitter’s option to flag a user for their behaviour. So if you are tweeting the truth about human rights violations (Cameroon, Saudi Arabia, etc) or touching countries with ridiculous strict laws (India, Russia, etc) than be prepared to have your account suspended, until you meet the demands of those countries. Maybe a lawyer can explain to me what an American company (Twitter) has to do with local laws of countries that for instance have a proven track record of completely ignoring human rights? And maybe it’s also a good idea for Twitter starts sharing information on which countries send in the most take down notices and which accounts are targeted. Because those accounts might hold the most valuable truth for the people!
Have a good day and have a good search!
