From the Judge’s Seat: A TraceLabs CTF Recount
Today is August 10th, 2020 and we are a day past the TraceLab’s August CTF, which happened to kick off on the third day of DEFCON SAFEMODE, which is a myth because we all know that DEFCON was cancelled this year. This is a quick write up on how the TraceLabs OSINT for Good CTF goes from the perspective of a judge.
This was the second time I’ve judged for TraceLabs. In fact, the first time I participated in the CTF I did so as a judge. It was an interesting perspective; seeing hundreds of flags submitted over a few hours that felt more like a few minutes. Seeing people learn about the missing persons, and learning about them yourself, it challenges how you think about what a CTF is.
But I’m getting ahead of myself.
What is the TraceLabs CTF? Well, a CTF is a capture-the-flag competition where contestants battle for the highest score through a range of challenges, usually, and this holds true during the TraceLabs CTF. But this isn’t the main point. The TraceLabs CTF is focused on real missing persons (MP) cases globally with the goal of helping Law Enforcement (LE) find out what happened to these people. It’s not only a fun and interesting contest, but the work is highly rewarding.
So let’s break down what happens day of the CTF. First, you get up — tired because you couldn’t sleep, there is too much excitement leading up to this day! If you’re me, you head to the gym and lift things until you’re awake. Otherwise, maybe you eat food. I don’t know what sane people do on a Saturday morning.
Now, we’re an hour out from the CTF and it’s time for the TraceLabs team to start breaking it down for the contestants. While they’re doing that, you, a volunteer judge, are logging into the CTF platform and making sure your sock accounts are still working. Even if you checked Facebook the day before and it was working, that does not mean it’s working today.
First thing you see when you log in is probably a blank screen. It’s a while until the CTF opens up, after all. Once the gates are open and the flags start flying, you’ll be able to see the cases for that CTF. These range from 5–15, depending on the CTF. The last two CTFs I’ve participated in (once as a contestant, once as a judge) each had 8 cases. The first CTF I judged had 5. It all depends on what cogs are moving on the back end of TraceLabs and you, even as a judge, won’t know how many until it all starts, or right before hand.
Again, these cases are all real people who are currently considered missing. The cases can range from a few days old to years old, and the ages of the individuals vary widely. Here’s the point that really needs to be driven home for both future contestants and judges: If the source of a flag is a missing persons alert organization or a news source, it is not a viable submission! This information is often given to you in the CTF platform, but there are multiple sites in the wild that have MP profiles up, and some may vary slightly.
What this means is that submissions should be new data for LE to follow up on. Details that may have been missed or were unknown to the investigators. Information that the Law Enforcement Agency (LEA) released themselves is not new information and thus is not considered a flag itself. It can, I would argue, be used as supporting evidence if you find something that needs to be updated (such as eye colour, hair colour, etc).
These details would be considered “Advanced Subject Information” (at the time of writing this) and can be very important for LE to know when searching for MP.
So let’s say a contestant submits a few photos from Instagram and Facebook to prove that an MP is listed as having brown hair, but recently dyed their hair blue, or that the MP has a history of dying their hair. The contestant hasn’t submitted screenshots (note for contestants: please always submit screenshots in addition, it’s so helpful), and Facebook is giving you a hard time if you’re not signed into an account. What do you do? Well, just like a contestant, you need to have sock accounts (personas, fake accounts) set up on multiple social media platforms to investigate clues that come in.
Contestants aren’t the only ones using OSINT to find more information about the MP in these cases; judges are (or should) actively be doing this as well to verify flag submissions.
Setting up socks can be difficult or strange the first time. I’m not going to get into it here, but if there is interests, I may make another post in the future, let me know!
If a judge is having a hard time verifying a flag, which happens often with the dark web submissions, or you want a second opinion on a flag, the TraceLabs slack will always have an active channel for judges for any specific CTF. This means judges can constantly share information, get opinions from others, and ask any questions they may have. The volunteer judges are always a mix of repeaters, seniors, and new judges. It’s important to keep the dialogue open and spread the knowledge.
Communication is key. Communicate with other judges and communicate with the teams you are assigned as a judge! If you are in contact with the teams it will just help you both. They can argue for why something should be accepted, and you can better understand what’s going on, on their side. This is a great opportunity to help the contestants fix any errors they may be making with submissions too, such as a lack of context or detail which leaves submissions vague.
All the above can be very overwhelming. As submissions start pouring in it becomes harder and harder to keep up. Know that you have an hour or two to address any flag that comes in, this is expressed in the CTF rules so that contestants expect a wait period. Not only does this give you time to talk to other judges if you need more eyes on it, but this gives you time to keep up with back log.
My first CTF I judged up to 7 teams at any time (grabbing extras when other judges needed to step out). This flooded my submissions box. When I judged this past Saturday, I went an hour or two with just one team, calmly going along. Then, suddenly, I had another team and they had two hours’ worth of submissions that were going unchecked now suddenly in my submission stream! Halfway through those and I had another team with a similar situation pop up! This was crazy!
And then what happened?
I was scheduled to do a DEFCON talk right after those two teams showed up so I had to step away. I got as many as I could done before the talk then skedaddled. After the talk I had more submissions from the three teams than I could count (I’m almost convinced these three teams submitted more than the seven I had at any given time my first CTF). This was stressful to say the least. I didn’t want to leave these teams hanging and there was some confusion over what was and was not acceptable sources (see: MP announcements and news sources).
But I’m back, all is right in the world while I go through and update submissions (accept…accept…reject, explain, accept…). Then it happens. The CTF ends…for the contestants. Up to this point, you may not have understood stress as a judge. Sure the above is stressful, but this is on a new level you may not have been prepared for!
When the timer runs out for the contestants, the flood gates open. Every submission, from every team, opens up for every judge to look at. All of the volunteer judges are then diving into this stream to work out if a submission is acceptable or not. Gone are the associations with specific teams you were working with. Gone is most of the ability to reach out to teams about what they sent over. And gone is the chance for future resubmissions with more context. It’s time to take it as it is or drop it.
That is not so bad, you may be saying. And sure, it isn’t the worst thing in the world. The added stress level comes from the new clock for the judges. You see, every contestant (usually over 600 on a global stage) is waiting to see how they’ve placed. And they’re waiting on you (and the other volunteer judges). The score isn’t finalized until the last submission is reviewed. From there, the TraceLabs team takes it away to finalize the tally. So, you scramble. Do you start at the top? The bottom of the submissions? How about the middle? Are you really good with one type of submission or is your Facebook sock down so you can’t check those? It’s a mad dash at this point, and it feels like it may take forever to get through what could be hundreds of submissions…
And then it’s all gone.
The submission stream is blank.
You remember to breath, to blink, and your fingers relax off the mouse or touch pad.
And it’s done. The team will come back shortly with the results and everyone will be excited. Some will sleep. Some will drink. Everyone will reflect on how it went from either side. We’ll all hope we help LEA find these MP.
And we’ll clean up; clear our cache, maybe destroy socks if we feel the need. Regardless, it’s finally time to relax after hours of running down leads and verifying submissions. Congratulate each other. Be happy, you did a great job!
Then we remember.
One month until the next one.
