“Tricking” our developers into liking application security
Are you in the application security field?
A security engineer? Maybe a frustrated CISO?
Developers are often curious people, very passionate about topics such as performance, testing, DevOps, and more, but somehow when it comes to security, they seem to lack the same passion. Why is that? It’s usually surprising to those of us in Application Security since we see it as such an interesting, challenging, and creative field.
If developers don’t show much interest in application security, then lectures or awareness training won’t do the trick…
A different approach must be taken!
This post will cover some of what we learned from the event (both good and bad), as well as give some more technical pointers on how to set such an event up for other Application Security teams who might want to try it with their developers.
The Impact (or, “why should I?”)
I gave a few AppSec presentations to our developers, but I felt like we could do better with a different approach, and so I decided to organize a Capture-The-Flag (CTF) event instead and put the developers in our shoes for a day.
The impact on our developers went far beyond our expectations. I am here to tell you, based on this experience, that this approach should be adopted by more companies as an industry standard to get people excited about topics they don’t know.
In just one day, we managed to get 200 developers stoked about secure coding. They were so into finding bugs and winning the competition, that they all forgot to eat lunch that day. And developers never forget about their lunch!
The overall engagement of our security CTF was higher than some of the biggest productions at WalkMe that you would typically expect developers to be more naturally connected to.
Moving forward, we decided to conduct a similar event every year.
Let’s walk through how we did it.
Step 1: Intrigue them
As we said before, developers are curious. They will do amazing work if they find it interesting enough. And they definitely want to be the best.
We need to establish the fact that writing secure code is part of being a good developer. Based on this, we assumed they would come to us :)
Pro tip: Let them know that security flaws are like bugs.
When customers ask for a feature, they don’t have to say, “Oh, and can you please make sure that it doesn’t contain any bugs in it?”.
It goes without saying.
So let your devs know that the customers also won’t say, “Oh and can you please make sure that it is secure?”.
Discuss it in your next guild session. Show some examples of security coding standards from other respectful companies.
The key point here is to make them realize they will be better developers once they understand secure coding.
Step 2: Make it great
Application Security can’t be taught briefly, so you have to make some time for it. We actually dedicated an entire day for all the developers to participate.
As for the choice of hosting a CTF: Let me tell you something, nothing beats hands-on workshops. They are the best way to learn for us techies.
Another pro tip: Get your higher management on board for such an event, and get help from other teams (Engineering Excellence, Marketing, Operations, etc.).
Thanks a lot, Anat, Dror, Omry, Yana, Oded :)
A big part of our CTF’s success was that it was well supported by the company and all of the various teams we asked for help setting it up. (And also, we had amazing prizes.)
Each team in our CTF consisted of up to 5 people, and each member of a winning team received:
1st place: PlayStation 5
2nd place: Oculus Quest
3rd place: Portable Projector
But just having awesome prizes isn’t enough. If you want to host a successful event you have to be prepared.
The first question you have to figure out is: Should you use a commonly known CTF or write a new one?
Pros for a known CTF: well tested, less time and effort, probably high quality
Cons for a known CTF: solutions are out there :(
JuiceShop was my go-to for our event.
OWASP JuiceShop is a great vulnerable app to practice security on. It is very progressive and highly maintained by the community.
JuiceShop contains all kinds of security bugs that you want to train your developers to be careful of, and the best thing about it is that it has a super easy CTF configuration.
JuiceShop has an infrastructure platform called MultiJuicer which is a great way to deploy a lot of instances for your R&D.
For the overview and scoreboard of the challenge, you can use a CTF platform such as CTFd, FBCTF, RootTheBox (those 3 are best suited for JuiceShop). It’s even nicer if you could have the scoreboard displayed somewhere throughout the event, so people can watch as points are being awarded live!
Step 3: Get them pumped
You’ve already chosen an existing CTF or created one yourself, picked a CTF platform, and created the infrastructure.
Now it’s time to let the developers know about the event coming.
Teasers
To do that, we sent small teasers with challenges related to security every couple of days (starting about a month before the event itself), with increasing difficulty as the teasers progressed.
We leveraged our own product and sent ShoutOuts using WalkMe Workstation to increase engagement with the users.
Tip: I recommend starting with challenges that are more of coding questions that are somehow related to security. They have a better chance to catch the developers’ attention.
And with each teaser, move towards security-based questions.
Each teaser directed them to a Google Form where they could submit their answers and we could see the engagement, as well as which questions were easier and harder for them to answer.
For the more complex teasers, we also created a Slack channel that we invited them to join, where we would post a more elaborate explanation the day after the teaser was sent out. This is also the Slack channel we used later during the event to communicate updates and offer help.
Tools
We’ve all spent time learning how to configure SSL on Fiddler at one point or another, and we didn’t want the developers to waste that time and be frustrated on the day of the event.
For our event, we gave them a list of helpful security tools that they could download a few days before the event if they wanted to get a head start. I also created a page for each tool in our knowledgebase explaining how to install it and basic usage/tips:
We suggested the following: BurpSuite, Fiddler, Postman.
Step 4: CTF Day
This is for all the marbles now. If they have fun, they will ask for more training down the road, if not, you had a good run but…
We started our day with some basic application security training that covered some of the more common vulnerabilities, the general idea and how to start looking for them, as well as some general explanations about how the event is going to work (remember: they don’t know what a CTF is yet).
At the end of the day, when they’ve had more time to see how the vulnerabilities work in practice, we had another session about how to fix these vulnerabilities (how can you fix what you can’t exploit?)
Side-Channel attack
During the day, we had little workshops at the office and on Zoom for people that wanted to get some extra tools that will maybe help them find more bugs, or just expand their knowledge.
Ideas for workshops could be:
Advanced Burping, blind SQLi, API security, etc.
I recommend being concise (don’t exceed 30 minutes), they don’t want to waste a lot of time from the CTF itself. Everything should be with real examples and a demo by you, so they will fully understand. It is not a lecture.
Some tips for the day:
- Make sure you have a communication channel for questions and technical issues (it’s 2022, don’t forget your remote employees!)
- Ask for help from security co-workers to help teams throughout the day- We recruited our friends from the larger security team to help us (you don’t have to be an AppSec specialist to be able to help point people in the right direction!)
- As part of the starting presentation, show a full live demo (create a team, find a vulnerability, submit the flag), it’s the easiest and fastest way to explain the process.
- Cool prizes are a great help in motivation!
Technical Part
To host JuiceShop for different teams simultaneously you should host it using MultiJuicer. It runs on a Kubernetes cluster and can create instances of JuiceShop on-the-fly as teams are created on the platform.
If you chose JuiceShop, as I did, a great place to start is here.
Setup:
- Create a Kubernetes Cluster on your favorite cloud provider
- Publish MultiJuicer on your cluster
- Create a domain for your CTF and redirect it to MultiJuicer Load Balancer
- Deploy a CTF platform of your choice (I highly recommend CTFd)
- Create another subdomain and redirect it to the score server
- Restrict all traffic for your internal network (optional)
Cheat detection
When choosing an open-source CTF there is always the possibility of cheating. There are some technological controls that you can put in place, the most important one being: generate new flags!
If new flags were generated, you can’t simply find them online and submit them, even if someone found their way to a walkthrough, they would still need to be able to understand and reproduce the steps, so even cheaters learn. :)
You can try to block traffic for known write-ups (https://pwning.owasp-juice.shop)
But at the end of the day, technological controls can only go so far, the best approach is to remind people what it’s about. To quote our VP of Engineering Excellence: “If you cheat, know that you are stealing a PS5 out of the mouths of your colleague’s children”.
Finally, if you are too afraid of people cheating maybe consider offering more moderate prizes for the winners.
Conclusion
Assuming developers are curious and passionate people, we just need to find the right tools to make them as ambitious as we are about application security. It’s not as hard as we sometimes think it is. Show them that security is another milestone for being a great developer. Plan ahead for an awesome day of AppSec training. Train them and get them interested before that day. Pick the right platform or gamification that suits you best.
Get developers that can find the next XSS for you :)
