Please Pick Up: Experiences from the DEF CON 30 Vishing Competition
Hello, this is _____ from corporate IT. I’m speaking to _____, right? I’m reaching out because we’ve had negative feedback at your location on IT measures impacting productivity. I’m wrapping up a report for this week and just need a few minutes to hear about your experiences.
This pretext landed me second place in the DEF CON 30 Social Engineering Community Vishing Competition. Over the course of 25 minutes, I called about a dozen employees of my target company to see what information they would divulge over the phone. Objectives ranged from browser versions to security guard hours; multi-factor authentication (MFA) systems to office badge descriptions. By the end, I was in disbelief at what I accomplished, despite seeing others do it before.
Here are my experiences from the competition, how I got there, and the lessons I took away. I hope this gives an interesting peak behind the scenes, and helps future contestants prepare, or simply work up the courage to compete.
Getting Started
Human security has interested me for a while. Since the start of my career, I’ve been hyper-aware of how important people are to security. Creating strong relationships between security teams and others was a huge goal of mine. In small companies, I built a security champions program to get engineers more involved with security, learned to manage bug bounty programs in ways that were fair and grateful to researchers, and generally did my best to leave a fun, positive impression of my team on everyone I interacted with.
As a shy introvert, I secretly wanted to challenge my own confidence and social comfort zone. I think this still drives a lot of what I do today, including my participation in this competition. When I first attended DEF CON 26 in 2018 and saw that year’s vishing competition, I was wowed and knew I wanted to try it. Contestants sat in a soundproof booth, called employee phone numbers, and managed time after time to solicit security information with enthusiasm. It looked like magic!
The technical aspects interested me, such as spoofing phone numbers. Contestants could place phone calls from any number they wanted, and while I’d been getting scam calls for years using my own area code, I was interested in just how difficult it would be. I set up a private branch exchange (PBX) telephony server, connected it to Twilio to place spoofed calls, and wrote a blog post about it. Soon after, Twilio started requiring verification for outbound phone numbers, so I guess it wasn’t allowed intentionally like I thought. At least I had fun testing it on myself and family for a while.
As DEF CON 27 approached in 2019, I applied to join the vishing competition myself. Applications are a little unique in that they require a short video, and it was known that fun ones stood a better chance. With that in mind, I left my college apartment for the local record store to pick up a copy of The Way It Is by Bruce Hornsby And The Range. You see, there was a running joke on a social engineering podcast at the time where one of the hosts would play snippets of the song Mandolin Rain off that album at opportune moments. I thought it was pretty funny, so I based my concept off of that. My application wasn’t accepted, but I managed to dig up the video for its long-overdue public premiere. Enjoy!
The denial hurt, but I was determined to persist. Unfortunately, COVID-19 threw a wrench in any plans to hold the competition in 2020 or 2021. When this year came around, the DEF CON village was under new management, and I… kind of wanted to try again?
By this point, I wasn’t as interested in social engineering as I had been in the past. Human security was just as important to me as ever, but it had been a while since the magic of those competition phone calls inspired me. After putting it off for a while, and a lot of internal debate, I took a day to quickly throw together a new video. With a much bigger recording space, 4k camera, tripod, and support of my cat, this one did the trick! Though I consented to showing it at DEF CON, it never aired, so please enjoy its debut as well.
After a while of waiting, and some pushed-back decision deadlines, I was in.
Heck.
OSINT and Vishing Reports
105 pages: The length of my open-source intelligence (OSINT) report.
19 pages: The length of my vishing plan report.
I severely underestimated the amount of time and effort I’d need to put in before I could place phone calls at DEF CON. I procrastinated for a while, then spent some good weekends and evenings finding as much information on my target company as possible. While the tools were simple, it wasn’t easy! I ended up in third place with 429 points for these reports, behind second with 431 and first with 461. In the end, I moved up a position and first place kept theirs.
Note that I won’t include company names or specific evidence anywhere in this post. Target companies, including mine, do not volunteer or consent to be involved with the competition, and while contestants do our best to conduct calls ethically, it’s best to keep things anonymous. There’s a reason audio and video recordings aren’t allowed in the DEF CON village!
Ethics are the backbone of the competition, and we all agreed to stay far away from fear-based pretexts, authority impersonation, collection of personally identifiable information (PII), and more. The goal was not only to showcase social engineering to the DEF CON audience, but also to maintain professionalism and avoid disruption at the target companies.
OSINT Objectives
OSINT reports contained 25 possible objectives. Here’s a slightly-abbreviated list:
Access control, employee badges, anti-tailgating signs, badge replacement process, janitorial company, waste management company, shredding company, waste management pickup day, shredding pickup day, security guards, security guard hours, vendor check-in process, phishing tests, security awareness training, email address format, style guide, internal lingo, operating system, web browser, anti-virus, VPN, Wi-Fi network, password policy, MFA, MFA accounts
I found and documented 20 of them, and was finalizing the report just hours before the submission deadline. If I procrastinated less, I might’ve found more, but I was happy that I got at least half and surpassed 100 pages!
Going down the list seemed like the easiest way to complete the report, but I quickly realized some of the objectives were much harder to find than others. Most data came from Google (including Images and Maps), DuckDuckGo, YouTube, and LinkedIn. Some data also came from miscellaneous tools: ASN Lookup, DNSdumpster.com, MXToolBox, Twitter, and WiGLE. Some of these services may be unfamiliar, but they’re not complicated! It really is possible for most people to do this type of research and succeed.
All data had to be from 2019 or later, which presented a challenge and ruled out a lot of potential evidence. With that in mind, let’s dive into some of the findings and processes.
Access control: A few YouTube videos contained blurry glimpses of badge readers, but this was enough to identify the main vendor in use. In one video, I could easily discern a black rectangle with a red light bar at the top and blue logo at the bottom. This immediately brought the company HID to mind, and my hunch was correct. A quick browse of products on their website confirmed the exact model and appearance. You’ve almost certainly seen HID readers in office buildings or public spaces. There were even some at Caesars Forum this year!
Employee badges were much harder, as I could mostly only find indiscernible sources on YouTube. Still, the work I did finding them helped a lot of my future research. The company had posted tours of many of its offices, along with a few unofficial videos floating around. There were a few different badge formats, but I was able to piece together enough of an idea on appearance.
Waste management company: Google Maps Street View was the key to this finding. I looked around the perimeter of the company’s headquarters, and found a loading area with dumpsters and trash compactors. Logos on the dumpsters were easy to trace back to their companies. One dumpster had a phone number that also turned up a company in Google. Sadly, I was unable to determine pickup day or details on shredding.
Security guards: Simple Google searches for the company name and “security” turned up a number of job postings on the official website. Those gave me the exact job title which I used for further Google and LinkedIn searches. With those, I was able to find details on hours, including evenings and weekends.
Security awareness training: Searching Google for the company name and “security awareness” revealed an employee post with a certificate of completion on LinkedIn. The title was pretty vague, and I didn’t have much hope for a finding, but a followup search with the text on the certificate revealed an exact match from KnowBe4, a popular security awareness vendor.
Figuring out if the company conducted phishing tests was harder, and I’m not sure if I got points for this one. There were no direct results for phishing information, but mail server (MX) records from DNSdumpster.com pointed to Proofpoint, a popular security email vendor. I looked at MXToolBox and found SPF records for the company, which specify the third-parties that can send email for a domain. Some IP addresses were owned by Proofpoint, and a final Google search for the company name and Proofpoint turned up an old cached job posting mentioning the vendor.
I couldn’t be sure that Proofpoint was used for phishing, as they’re typically a sort of email spam filter, but there was at least a chance. Submitting the info I had was better than submitting nothing, and may have gotten me a few points!
Email address format: This was probably the easiest objective on the whole report. Many of my searches started on Google with the company name and “filetype:pdf” to look for documents. Almost every single one had an email address in the format “first.last@company.com”. A few random LinkedIn posts and Tweets added confidence to the finding.
Wi-Fi network: I was stumped on this one until I was informed of a nice service WiGLE to search public Wi-Fi networks. WiGLE relies on crowd-sourced data, but had a lot of results for my company name in prominent office locations. I was only able to conduct a few searches before my trial account was restricted, but I got everything I needed. Be sure to use your search credits wisely!
MFA accounts: This was probably the most shocking finding of my report. A Google search for the company name and “sso” found an SSO portal that immediately looked like Azure Active Directory (Azure AD). I confirmed this with a few other searches for similar-looking Azure AD setups, and I was confident Azure AD was using MFA. Interestingly, the SSO portal offered a list of websites to log into. Usually this happens after login, but anyone on the internet could see this list! This was a treasure trove of many applications in-use by the company.
These are just a few of the objectives found, but they were some of the most interesting. Many others came from the same sources and techniques.
Vishing Objectives
Vishing plans were focused on a similar set of information, but with 29 objectives instead of 25. Here’s another slightly-abbreviated list:
Work from home or office, access control, employee badges, anti-tailgating signs, badge replacement process, waste management or shredding company, waste management or shredding pickup day, security guards, security guard hours, vendor check-in process, phishing tests, phishing test sentiment, security awareness training, security awareness training details, security awareness training frequency, email address format, operating system, web browser, anti-virus, VPN, Wi-Fi network, MFA, MFA solution, MFA accounts, MFA-lacking accounts, fake website visit, most utilized software
While I found many of these during the OSINT phase, all of them would need to be solicited from live phone calls at DEF CON. My vishing plan report focused on collecting as many objectives as possible on one call. I ended up with a single primary pretext that I was really happy with, and a few more backups that I threw together at the last minute. I only ended up using my main pretext during the competition, as it seemed to be flexible and working well.
Some of the best advice I got was to use a pretext that felt comfortable and natural to me. One of my first ideas was a cybersecurity analyst collecting feedback on recent cybersecurity initiatives. If you look back at the top of this post, you’ll see that it didn’t change a whole lot! I’d never really done social engineering before, but this goes back to my passion for human security. I’ve always cared deeply about how people perceive security teams, and I find it easy to sympathize with complaints. Nobody likes clicking through the boring security awareness training slideshows each year.
I was told that mentioning security so early on might raise some suspicion, which made a lot of sense to me. With only a few seconds to create a positive impression, I wanted to leave as little room for questioning as possible. I changed my persona from a cybersecurity analyst to a generic corporate IT team member, and refined the intro to focus on recent negative feedback impacting productivity. This way, I could still hold a natural conversation, and build trust by letting people talk about their security perceptions.
A key to this pretext was asking for sentiment around objectives, rather than directly asking for objectives themselves. For example, one of my questions might be “Have you experienced any slowness with the VPN?” instead of “Do you use a VPN?”. I can still solicit further details, but this leads to a much more natural conversation, and call recipients will feel like their experiences are being heard.
Asking for objectives like this made me fear taking up excessive time on each one, but I think it was worth it. During the competition, there’s a lot of time spent dialing and hitting voicemails. Every recipient that picks up is extremely valuable, so it’s okay to spend extra time building trust and getting the most out of each call. Here’s what my final list of questions looked like:
I definitely improvised on wording during the actual calls, but having an informal script like this helped a lot. You’ll notice that I lead some questions with the answers themselves, like naming Windows and Google Chrome. Getting people to confirm what you already know can be easier than asking outright, and they may even be eager to correct you if you’re wrong.
What happens when someone becomes suspicious half-way through the phone call? How can I keep their trust and continue getting objectives from them? Our reports contained a section on handling pushback, but it turns out I worried about this a lot more than I needed to. My plan was to create sympathy — it was Friday and I had a report due at the end of the day, with 12 more responses to go. In reality, once I had initial trust established, people were more than willing to continue answering questions.
Phone Numbers
With my pretext ready, the last piece of the puzzle was the set of phone numbers to call and spoof as. I mentioned earlier finding a lot of PDFs with email addresses during OSINT, so I used the same strategy to find phone numbers. I searched Google for PDFs on the company website and collected a little over 70 phone numbers.
Every number had a name and job title associated with it, and some had locations. I started Googling names and found official company directory pages confirming those names, numbers, titles, and locations. We were allowed to call numbers to validate them, but had to hang up immediately and avoid all interaction.
All of my numbers were for individuals, not offices, so I didn’t have to worry about phone menus and options. I called to make sure each number worked (with *67 to not give away my own cell phone number), and even confirmed some names upon answering or hitting pre-recorded voicemail messages. I then started ranking my phone numbers based on the following criteria:
Phone number: Confirmed on official directory > differs from official directory
Job title: Common job title > mid-level sounding job title > high-level sounding job title > executive sounding job title
Location: Confirmed on official directory > differs from official directory
Attempted call: Answered with name > answered > voicemail with name > voicemail
I compiled a final list of phone numbers that I’d successfully validated, and had enough confidence in for a successful call. I split them into timezones, and mainly focused on Pacific Time (PT) numbers. This got me my reference sheet for the booth:
I cut a lot of the original 72 phone numbers I had, and reordered a bunch based on rankings. I also had numbers to spoof as, which came from different office locations. I created some 1-800 lookalikes, but ended up using spoof number C for all of my calls:
Coaching
I haven’t talked about coaching yet, but it was one of the coolest parts of the competition. As a contestant, I had access to four different coaches, and about an hour of time with each one. They were incredibly helpful with advice on OSINT, vishing, and more.
I did most of my coaching sessions as I was wrapping up my OSINT and vishing plan reports. For OSINT, they pushed me in the right direction on some objectives I was having trouble with. I hadn’t thought of looking at Google Maps Street View for dumpsters, and didn’t even know WiGLE existed for finding Wi-Fi networks.
A lot of the advice I mention throughout this post comes from coaches, like using comfortable pretexts and confirming already-known information. I learned that artificial time constraints are really helpful — telling the recipient that you only need a few minutes of their time.
I was told to practice my pretext heavily, which I did a little bit with one of my friends, and even with one of the coaches. I probably could have practiced a lot more, and tried it out on some real phone calls, but I was happy enough with my preparation. One of the coaches told me I was unlikely to get much pushback, and if I did, it might not be worth trying to continue the call. That turned out to be true!
I’m incredibly grateful to all of the coaches for spending so much time helping me and the other competitors.
Competition
Getting into the booth was pretty scary. I showed up to the DEF CON village a little early to watch some other calls and scout ideas. We had just found out the day prior that out of 16 teams, there were only eight target companies, so every two teams had the same company. The two teams would go back-to-back, in an order decided by a coin flip.
As I had more points on the OSINT and vishing plan reports, I got to choose the coin side, and ended up winning the flip with heads. I then chose to go first in the booth, as I wanted to get it over with, and was afraid the second team might end up calling some of the same people as the first.
When I got into the booth, I unpacked my reference sheets and iPad, which had my latest pretext updates on it. I was incredibly nervous, so decided to just stare at my pretext and clear my head as much as possible. No matter what happened, I at least had some sort of initial plan to follow.
I started calling phone numbers from my list, jumping around groups a bit and hitting a number of answering machines. When people finally answered, I had two very successful calls, with the first one netting me almost every objective on my list. I only received pushback twice — one person told me they didn’t have time, and another doubted my employment at the company. The suspicious person actually made me happy, as it’s nice to see security-aware responses!
Successful Call 1
I got a majority of my points from a single call. When I introduced myself as an IT team member responding to recent negative feedback, the recipient mentioned it may have been related to problems they reported. They were instantly on-board with my pretext. What great luck!
My plan to relax my nerves worked. I focused on the intro I had written down, and then focused on each question in the moment. I tried to think about the call one step at a time, and not think about collecting every single objective.
I started going down my list of questions. I made a point to ask if the recipient had seen recent slowness with any of the things I asked about. I don’t remember all of the person’s responses, but it seemed like they were generally happy with everything. The pretext was working exactly as planned, giving us a chance to talk naturally about each objective.
We got to the end of the call, and the person was even willing to visit the fake website set up by the village, earning me a fair chunk of final points. I was really happy at that point, and thanked them for their time before hanging up.
Successful Call 2
This call likely only got me around half the objectives as the last one, but definitely played a big role in my final score. When I introduced myself, the recipient responded that they didn’t think they could help as they were working from home. I told them that’s actually exactly why I was calling them!
I again went down my list, but chose to skip some of the more office-related questions. I think I tried to fit a few of them in still, but don’t remember which ones. This person also seemed happy enough to talk to me, and didn’t push back. Sadly, I let them go at the end when they resisted visiting the fake website. If I had pushed a bit more, that objective would’ve landed me first place.
After the Calls
When I exited the booth, I was really proud of myself, and relieved that it was done. I had put in so much work up to that point, and challenged myself just like I wanted to. I thought I did alright, and looked forward to seeing how the team after me would do with the same target company.
When the judges held up their bonus point awards, I was shocked. One judge gave me 9/10, and the other two gave me 10/10! Was it sympathy for my nerves, or did I do better than I thought?
The judges gave me some nice feedback on my performance. When I said I was very nervous in the booth, one of them said they couldn’t tell as I captured so many objectives. I’m not quite sure I believed that, but still appreciated it a lot! I got some advice on using more internal company lingo in my pretext, which I agree might have helped establish trust.
I never realized just how well I did until I got an email the next day.
Results
I woke up from a nap to this email in disbelief. The night before, scores had been posted for the first eight competing teams, including mine. I was in fourth, and guessed I’d be bumped down even more when the second half of teams finished. It turns out the scores were calculated incorrectly, but I didn’t have much hope for better placement.
Over the course of the next day or two, the excitement turned into regret, as I was just 43 points away from first place. It haunted me to think that I would’ve won if I got another person to visit the fake website (55 points), or dressed up in the theme of my pretext (80 points). Was deciding not to wear a costume really what cost me the competition and a black badge?
I have nothing but praise for the winning team, and they really deserved it! We all had our plans, and if I decided to do something worth a few more points, they could’ve done the same just as easily. Still, I won’t pretend that getting so close didn’t hurt, and I’m pretty sure I experienced every stage of grief from the moment I saw the results to the morning after my flight home.
Denial: After the first posting of incorrect scores, I thought maybe these ones were wrong as well. But then would I really get an email confirming my second place? I partly hoped I was lower down the scoreboard, because I’d have fewer regrets ignoring the wardrobe points.
Anger: Did I really lose because I chose not to dress up? And is there nothing to be said about a team of one competing against a team of three? Well… yes and yes. These were the rules of the competition from the start.
Bargaining: Surely I should have gotten some wardrobe points! Before the competition, I even went back to my room and changed into joggers with a tech t-shirt. This fit the lazy IT worker style right? I knew that was a weak argument, or I would’ve asked for those points in the first place.
Depression: I found it difficult to be happy about my second place victory, or talk to anyone about anything related to the competition. You may have seen me at hacker karaoke the night of the results, singing Helena by My Chemical Romance to bid “so long and goodnight” to the black badge. Even joining the contestant panel after the award ceremony was a bit difficult.
Acceptance: By the time I woke up the morning after I flew home, my head was pretty clear again. Some personal regrets may haunt me for a while, but I really feel nothing but happiness for the winning team. I think about all the work I put in, and how the three people on that team probably each put in the same amount of work or more.
Looking back on my grief feels very silly, but I’d rather acknowledge the feelings than ignore them. I’m sure previous contestants felt similarly, not to mention future ones. I’m incredibly proud of my accomplishment and how I pushed myself to second place!
Lessons Learned
Wear a costume!! I’m only half joking… but in all seriousness, it is worth examining every possible opportunity to earn points. I decided not to go for wardrobe points because I didn’t think I’d do well anyway, so they wouldn’t make a difference. You never know how close it will be. If I ended up in third, I would’ve wondered why I didn’t push for a few more points to get that sweet second place trophy.
Confidence is key. Social engineering is all about confidence, and it turns out that doesn’t just apply to phone calls. I limited myself because I didn’t think I could win. Improving confidence was one of my goals, so I guess this lesson certainly checks that off. You can be very successful even if you’re introverted and have no prior social engineering experience.
Don’t be afraid to participate solo. My second place proves that you can be just as successful no matter your team size. It’s a lot of work, but I really value the control I had over every part of my plan, and would roll solo again in a heartbeat.
Don’t procrastinate! I put off my OSINT and vishing plan reports for a while, so it really felt like cramming when I did get around to them. Try to dedicate at least part of your day to research and writing leading up to the deadlines. This probably wouldn’t have won me the competition, but if I found a few more objectives, who knows.
Utilize coaches. I cannot express enough gratitude to the coaches for their help. I met with all of them, but still ended up with one or two sessions I didn’t use. Knowing how useful each one was, I wish I took full advantage of these.
Stay determined. There are times when all the preparation will feel like a drag, but it’s worth it. If I participate next year, I’ll certainly turn this year’s experiences into motivation to work even harder.
Closing Remarks
I considered splitting this post into a few, but I was excited to share my full experiences. I hope it helps anyone considering competing in the future.
Challenging my confidence and social comfort zone continues to be a huge goal of mine, and I can’t believe what I accomplished in this year’s vishing competition. I don’t use Twitter, but let me share a few small shoutouts:
Thank you @jaimefilson (WiK) for the photos and congrats!
Thank you @RachelTobac for another photo and congrats! Also for the nice challenge coin which I have on display at home along with my trophy.
Thank you @sec_defcon (Social Engineering Community) for hosting the village and organizing everything in such a short amount of time. This includes all the organizers, judges, coaches, and other volunteers.
Thank you to my cat for being the face of my team, and providing support along the way. Here’s the full photo that was cropped to be my team logo:
Team Name
Finally, I got a lot of questions about my team name. I participated as “Arty Boy”, which still felt weird every time I heard it instead of my name. It’s simply a song by one of my favorite artists Flight Facilities, so here’s one final video for your viewing pleasure!
