Top OSINT sources and vishing pretexts from DEF CON’s social engineering competition
This year, I was invited to be a judge for the vishing competition at the Social Engineering Community at DEF CON, the world’s largest hacking conference. Having participated as a contestant in the past, this gave me insights into all OSINT and vishing reports that contestants had to complete ahead of time. Having visibility across all reports gave me the chance to answer some questions that had been on my mind for a few years:
- What are the top-producing OSINT sources?
- How difficult are objectives to find?
- What OSINT source is best for what type of objective?
- If a contestant cannot find an objective, is it because it’s not discoverable by OSINT or because they haven’t tried hard enough?
- Which pretexts yield the best results?
I’ve collected and analyzed some of the data from this year’s competition to answer these questions — and more. Huge thanks to the organizers of the Social Engineering Community and the contestants for making this research possible!
A primer on the DEF CON social engineering competition
The DEF CON conference includes many sub-conferences, so-called villages. Each village focuses on a specific topic, such as WiFi hacking, lockpicking, or social engineering, and features a combination of hands-on labs or demonstrations and talks. The village organized by the Social Engineering Community includes a vishing competition. Vishing is a portmanteau of the words voice and phishing, in other words, using social engineering techniques over the phone.
This year, 16 contestant teams were selected to compete. They were informed of their target company several weeks in advance so that they could research their target company to write up two reports. In the OSINT report, contestants had to find 25 pieces of information (“objectives”) in public sources. The vishing report included the people they are planning to call, the person they are pretending to be, and their pretexts. Each team typically spends between 40 and 120 hours to write the reports, including researching the objectives, documenting sources in a way that judges could verify, and writing up professional reports. Contestants were not allowed to contact employees of the company, only to consult publicly available resources. They also had to prove that the information was no older than 2019, which added another hurdle.
At DEF CON, the contestants entered a sound-proof box and called their target company in front of an audience of hundreds of people. Their goal was to extract similar types of objectives (pieces of information) from a target company employee.
Analyzing 16 OSINT reports
The 25 objectives for the OSINT reports are picked to help a social engineer to either get into the physical building or infiltrate the digital systems of the target company. At DEF CON 30, these objectives were:
- Determine if an access control system is in place.
- Document examples of their employee badges posted online.
- Does the building have anti-tailgating signs up?
- What is the process to replace a badge?
- What is the name of their janitorial company?
- What is the name of their waste management?
- What is the pickup day for the waste management?
- What is the name of their shredding company?
- What is the pickup day for the shredding company?
- Do they have security guards?
- If they do have security guards, what hours do they work?
- What is the vendor check in process?
- Do they get phishing tests at work?
- What vendor provides Social Engineering or Security Awareness training to employees?
- What is the companies email address format?
- Find the corporate brand or style guide
- Internal company lingo
- What Operating System do they use?
- What Web Browser do they use?
- What Anti-Virus do they use?
- What VPN do they use?
- What is the Wi-Fi SSID name?
- How often do they change their password?
- Is Multi-Factor Authentication is in use?
- If they use MFA, what accounts is it tied to (email, VPN, intranet, etc)?
How difficult were the objectives to find?
I was interested to see how difficult these objectives were for the contestants and where they were getting the information from.
Generally, contestants had an easier time finding objectives connected to digital security than physical ones. However, all teams were able to figure out if the corporate building had physical access controls.
Most common OSINT sources
Reading through the reports, I coded each one of the objective findings because I wanted to figure out which online resources were the most helpful in completing objectives. When a team used several sources, I counted them all. However, if a team used the same source twice for the same objective, I only counted it once.
To my surprise, YouTube was the top-producing OSINT source. From personal experience, I have found lots of objectives from YouTube. However, it is typically difficult to target a specific objective in that way. Rather, I reviewed all videos available by and about my target company and tried to spot objectives — frame by frame. The most helpful videos are either corporate recruitment videos that show day-in-the-life at the company (yielding access control, badge designs, hardware, operating systems, browsers) and webcasts for employees or customers (revealing operating system, browsers, VPN, anti-virus solution).
The runner-up was LinkedIn, which I fully expected. Lots of people post details about the technologies they are responsible for at work. The Google search inurl:linkedin.com/in/ company vendor can quickly surface profiles of employees advertising their expertise in a particular technology. I often looked at the leading vendors on a Gartner Magic Quadrant to identify likely vendors in large companies. Once you find a “chatty” LinkedIn profile, you can usually discover many more objectives from the same employee. Job postings can be similarly revealing but authored by the corporation.
Google dorking was the third type of OSINT source. I coded a source as such when the starting point of the search was not a website but a Google search, and the website that was found was not a source otherwise categorized. If you are not familiar with dorking check out this Google dorking cheat sheet or go all-in and purchase Michael Bazzell’s book Open Source Intelligence Techniques.
The target company’s building website was a great source for all sorts of physical objectives. Landlords looking for commercial tenants will often host a website disclosing all sorts of useful information. Similarly, vendor websites can disclose customer lists as well as customer-specific login pages and documentation. They were also a great source for waste management pickup schedules, providing lookups by ZIP code.
Google Street View is great for any physical objectives, such as access control systems, waste management and document shredding companies, and security guards. Pro tip: You can travel back in time on Street View to show images captured in previous years. For the competition, objectives had to be from 2019 or more recent.
Corporate infrastructure was a collective set of OSINT flags where the digital infrastructure was leaking information, typically in the form of subdomain names, web applications hosted at the subdomains, and MX records (for email phishing protection services). Contestants used online services such as crt.sh, DNSdumpster, Security Trails, and MXToolBox.
My favorite new method looking at corporate infrastructure was by the team Spilt Beans (@bngrsec, @_jacoff, and @_seahop) who used the cache-snoop module in recon-ng to query the company DNS server for cached hostnames of anti-virus update servers.
Wigle.net was the most used source for obtaining a company’s WiFi SSID. A very useful service for this objective, Wigle.net provides a physical map of the world with SSIDs by location. However, the most second most common way to capture an SSID in the competition was finding the SSID in YouTube videos.
Future contestants may find this matrix useful: How often were which types of objectives found through which OSINT source:
The OSINT answer is out there — maybe?
Knowing when to stop your OSINT search can be a struggle. Is the answer simply not obtainable using an OSINT source or did you simply not search hard enough? This year’s data set had a partial answer to this question because each target company was assigned twice, so I was able to compare teams going after the same objectives.
For each target company, I looked at the two teams and determined the low and high score for the target. Then I looked to see if the higher scoring team had missed any objectives found by the lower scoring team. And yes, this was the case for all of the target companies. As frustrating as it is, each of the high scoring teams could have found between one and four objectives more than they had identified in their report. Next time, just keep digging!
How to mitigate OSINT risks
The report also asked contestants to advise the corporations on how to mitigate the risks of the leaked information. Most contestants recommended educating the users on how to keep this information private. With YouTube and LinkedIn being the two primary sources, that would mean training corporate communications and human resources on scrubbing videos of sensitive information and training employees on the risks of publishing technologies on their LinkedIn pages.
While this is not a bad idea, OSINT info will still leak from corporations in these and other ways. Additionally, companies should seek to OSINT-proof their processes. For example, authenticating employees or customers needs to involve information not readily available online. To know what’s available online, companies may want to conduct an OSINT search on themselves, or hire a firm to do so.
A little though exercise for you: How would you make your company’s processes more OSINT-resistant?
Learnings from vishing reports
Contestants also had to present vishing reports to the judges that would go into their final scores. These reports were also useful to flag any pretexts that were either unethical or outright illegal, such as using fear or impersonating a law enforcement officer.
Pretexts planned in vishing reports
As expected, IT-related pretexts were among the most frequent in the pretexts planned by the contestants, with a mix of IT/security audits, IT helpdesk reaching out, and IT/security surveys. Five contestants suggested pretexts following up on a potential security incident. The judges strongly advised against these because these could easily lead to a fear-based pretext, which was not permitted under the competition’s ethical guidelines.
Pretexts actually used in calls
At DEF CON, the contestants focused on a subset of the pretexts they had explored in the vishing reports. All contestants had access to several hours of coaching sessions from experts provided by the village, and many people adjusted their planned pretexts as a result.
The most frequent pretexts all involved an IT person reaching out to an end user, such as IT helpdesk reaching out, IT/security survey, and IT/security audit. Having observed similar contests for a number of years, these are really the work horses that produce year over year because they align well with the targeted objectives.
During the competition, I wasn’t able to consistently track which pretext produced what scores. However, I tracked who used what pretext and was able to calculate low, median, and high scores per pretext.
This chart is a good illustration that the pretext can help a ton with high scores but that there’s still quite a lot of skill — and a bit of luck — involved in getting a top score.
My personal take on pretexts
In addition to the quantitative analysis, I have a few takes on good vs. bad pretexts for the competition:
- Surveys sometimes struggle to drive compliance because there’s no authority / pressure. Promising gift cards or an upgrade for their laptop can help. Avoid threatening with negative consequences if they don’t comply.
- Phone numbers of individuals are less likely to pick up because there’s only one person who could answer the phone. It’s better to select general office or service numbers that are staffed and paid to answer the phone. I recommend general location numbers over specific services (e.g. helpdesk) because they typically don’t follow a rigorous process.
- Make sure that the pretext matches the target. The best pretext will be one where it’s the target’s job to help you.
- Pretexts that come from interns/students get low compliance because of the low status of these roles. While asking for help from a low status may get you there, using a high-status role is a better bet. You should still be nice and ask for help rather than command authority, but you’ll be likely to receive the help.
- Don’t just have an opening paragraph and then list out objectives for them to answer. Try to chain the objectives together in a narrative that makes sense in the context of the pretext. Plan ahead and put a lot of thought into this.
- Some of the questions were too generic and not believable, e.g. “I’ve heard viruses are a big thing. What do you use to scan viruses?”. Be specific: “I got a PDF via email. What should I use to scan this for viruses, or can I just open it?” Provide a reason. Why would you ask this? “I had a virus on my computer a month ago and now I’m a lot more careful.” You can plan ahead but if you’d like to learn how to justify impossible situations, take an improv class.
Recon tool recommendation
If you’re interested in OSINT, recon or pentesting, you may also enjoy testing out runZero, a network discovery and asset inventory solution created by my co-founder and Metasploit creator HD Moore. Although runZero (formerly known as Rumble Network Discovery) mostly provides asset inventory behind the firewall, many pentesters and other security professionals find it very useful for mapping out the external attack surface of a company (check out my blog post on how to scan your external attack surface).
Our Starter Edition is completely free for up to 256 devices, both for private and commercial use. If you want to try our advanced features, such as external scanning and integrations with EDR and vulnerability scanners, get the 21-day trial of the runZero Enterprise Edition.
