Part 4 — Delivering an Application Security Training Course
Going by the BSIMM7 chart below (further out to the edge is more mature), companies lack maturity when it comes to training and awareness programs. Running a web application security training course for development teams who are designing, writing and testing applications can provide a lot of value and should be incorporated as part of your overall application security programme.
The goal of a web application security training program is to raise security awareness and teach technical teams about security concepts, so that security issues are less likely to turn up in production code.
Benefits of a Training Program
- The tech team will learn about web application security concepts which will help them avoid introducing security bugs and flaws into production.
- Even if they come away having learnt only a few new security concepts they are now more confident in fixing and understanding security issues. They are also aware of the tools and resources they need to learn more and to practice their skills outside the training course. At the very least they know that security issues are a real threat so if they are developing an app they know to add the word “security” into their search terms when researching best practices for the database, language, framework or library they want to use.
- If a developer knows who to contact about a security related question they are more likely to do so if they know who you are and how to contact you. Running multiple training sessions forces them to listen and stare at you for hours on end so they will definitely get to know who you are!!
- Almost all the activities and objectives in your application security program will involve the tech team in some form. So educating them is a necessary and crucial first step to implementing a successful application security program.
- It aligns with most security frameworks including OWASP SAMM, NIST and PCI so if you are trying to achieve a certain level of maturity or compliance then training will need to be part of that process.
Security Maturity Models
It’s worth mentioning that there is a whole section on training in both OWASP SAMM, BSIMM and the Microsoft Secure SDLC so i would recommend reading the details in these sections before you start planning your training program, as it can help with direction and may give you some different ideas on where to begin.
Making an Effective Training Program
Most people have completed compliance training at some point in their life when they join a new company. How effective was it? How fast did you want to finish it? How many reminders did you need to start it? Security training has to be engaging, relevant and at least bearable if not enjoyable to be effective. Here are a few techniques that might help:
- Start with the why. Why am i here and why do i need to learn about security. Teach them about some recent breaches, why hackers hack, their motivations and if possible demoing real security issues affecting their applications.
- Make it practical and challenging for all skill levels. Get them to setup tools like OWASP Zed Attack Proxy and give them offensive and defensive security challenges based on the concepts you teach.
- Snacks, snacks and more snacks. If you are going to make them sit down and listen to you for hours on end then provide snacks or even coffee’s for morning sessions!
- Break up the course into small 2–3 hour sessions. People can only take in so much information, shorter sessions spread over a week or month give the content time to sink in. It’s also an easier sell the training to Product Manager’s and Delivery Mangers’s if the tech people are not missing in action for 2 days straight!
- Make it specific and relevant. Assess the current security knowledge of the students as they may already know about some of the OWASP Top 10 issues. So it might be worth going a little bit faster through those issues and also covering attacks that are likely to affect the application they are building. For example if the company is using OAuth and JWTs for the authorization and authentication flows, it might be worth teaching them things like JWT None algorithm, session injection, OAuth open redirect issues, etc.
Training Program Constraints
Not all of the techniques described above will work, as designing and implementing a training program will depend on your constraints:
- The training program budget. Can you spend a month designing and writing the material or will you have to bring someone in to help out. Do you have budget to pay for pre-made or custom training or online practical challenges?
- The resources available. Do you have internal security people that can deliver the training or will you have to bring in consultants to help?
- How many people do you plan to get through the program? Having a 3000 person tech team to train will be very different than a tech team of less than 200 people. If it’s a large enough team you might have to stick with online based training instead of in person training. However sometimes you can get the best of both worlds by delivering custom onsite training to a select few and then recording the sessions via a screen recorder such as Webex which then allows you to distribute the sessions at scale to others.
- How much time can you get from the tech teams to give the training? If the business decides that you can only take a 1 day of their time this will constrain what you can teach.
- How thorough you want to be with the content you cover? If you don’t have 5 days to train them on every single web security issue and concept you might have to pick and choose what content is most important to the company. For example you could use data collected from pen tests, bug bounty programs and code reviews to figure out what the most common issues are and start there.
With those constraints in mind let’s go through how to build a training program, some of the lessons learnt and how you can measure the success of the training program. The constraints for this example are as follows:
- 8 hours of training time per tech person.
- ~200 person tech team to get through the program.
- 2 months of a security person to build and deliver the training.
- Attendance was not compulsory but highly encouraged by management.
- Not including the 2 months of resource time the budget was minimal.
- Success criteria of 70–80% attendance rate and >80% “would come again” rating from students.
Only 8 hours of training time had been allocated in this example training program, so given the small timeframe it’s best to break up the training into a few, 2–3 hours workshops and spread them over a few weeks or days. This approach has the benefit of getting more people to attend as fitting a 2–3 hour workshop into their day is a lot easier than wiping out a whole day (remember the not compulsory constraint!). It also makes it more bearable and won’t overload the students with too much information. Also I don’t know about you but presenting for a whole day is tough too! One of the down sides to this method though is that there are more scheduling overheads because more invites will need to be sent and if your sessions are prerequisites of each other you have to track who has attended which sessions and organise your invites accordingly remembering to account for no shows.
So with the timeframes in mind we can start forming the agenda and content for the sessions. Using issues found via a bug bounty program, penetration tests and code/design reviews will help you decide what content to focus on. This will usually lead to content that aligns with the OWASP Top 10 so if you don’t have the luxury of analysing past issues then this can be a good place to start. Notice that the below high level course structure doesn’t cover all possible security concepts. Apart from not having enough time you will also need to break up the content with practical security challenges, and if past student feedback is anything to go by it will make the course more enjoyable and give them time to cement their knowledge of the concepts being taught.
Thinking Like a Hacker
With a goal of making the course fun and in turn more effective you have to work out what content would make them want to attend, but also give them enough information to be confident dealing with basic web security concepts. Starting with the “why we need to care about security” instead of diving straight into web security fundamentals (SSL, HTTP, Cookies, etc) will give them context as to why they should be learning about security in the first place, meaning they will get more out of the training and be excited about attending further workshops. So in a way you are trying to spark their interest and make them realise that security can be fun but also important to learn. Of course you can’t convince everyone to be interested in security but we only need 80%! It can also be a good way to single out your security champions :)
Ok so what topics should we cover to get the students interested in security?
- Cyber security trends. How many companies have breaches? What types of data are they after? Which countries are most affected? What are some recent examples of breaches that made the media?
- Hacker motivations. This is fundamentally why we need to care about security in the first place so go through what motivates attackers to hack into companies in the first place e.g. money, politics, fun. What are the different types of attackers e.g organised crime, script kiddies, activists.
- How hackers think and why it’s important to be able to think like one when designing and coding applications.
- What they do with the data after a breach? E.g. sell on darknet markets.
- Using lots of demos and examples of issues that have or still affect their application. This makes it relevant and real, once they see first hand what impact security issues can have to their application and the business.
Most tech people don’t read about security in their spare time so for them learning about things like darknet markets and what types of data and services are sold on them is quite interesting! Here is an example of a cut down and slightly modified version of what this first workshop could look like.
Web Security Fundamentals
It’s important to cover web application fundamental concepts like HTTP, TLS and cookies with a security lense so that it stays relevant to security but also gets the students up to speed, as you can’t assume they have been taught these low level concepts. To help students get up to speed you could also add to the training email invite a list of articles to read beforehand.
Common Web Vulnerabilities
After covering the “why” and the necessary web fundamentals it’s time to work out how to prioritise what else to teach and the ratio of theory to practical. The OWASP Top 10 as mentioned above is often a good place to start even if it won’t teach them about every web security issue. Also after analysing all past issues at this company it turned out that 97.5% of them were in the OWASP Top 10.
Practical Security Challenges
Student feedback from previous sessions highlighted that practical challenges are very important to include in any security training program. There are lots of vulnerable VM’s and web applications on the internet to choose from however they all fit into two different categories. Defensive, where you are asked to look for vulnerabilities in code to solve the challenges and offensive where its testing for issues in a black box environment.
Offensive based web challenges like OWASP WebGoat, OWASP Security Shepherd and OWASP Juice Shop are good options to start with. Then on the defensive side you have commercial services like Secure Code Warrior or Code Bashing. Defensive challenges give students online access to gamified security coding challenges with several different programming languages to choose from. Mixing defensive and offensive challenges gives you the best of both worlds. You teach the students how a hacker sees the world via the offensive challenges and you also give them a view of how to identify security issues in the code they write and test.
Clean and Clear Slides
Keeping the slides simple, using pictures and graphs where appropriate and minimising large chunks of text will makes the content easier to explain and absorb. Below is an example of a simple slide explaining the different parts of a set-cookie header. The downside of minimising the use of text is that the slides alone are not as useful. However you could record the workshops with a screen reader and /or add links to websites with more info at the bottom of the slide to help solve this problem.
Keep it Engaging
Asking lots of questions throughout the workshop is a good way of judging students skill level, how much they are taking in and will also keep them more engaged. The only downside is that it will slow down the delivery of the content so you will have to keep an eye out for how long you are spending asking questions.
Teaching with Real Examples
To make the workshops engaging and relevant there is nothing like real world security examples to spark interest! Instead of only demonstrating SQLi, XSS or IDOR’s with fake demo web applications like WebGoat, throw in a few example issues from within the company itself. This might not always be an option as the issues might already be fixed or the company might not allow you to disclose the issue to developers. So in that case you can show screenshots of the issue instead of a live example or in the worse case use examples from other companies that have recently disclosed a similar issue:
Scheduling and Keeping Track
Keeping track of things like who has attended which workshops and who need invites to which workshop sessions is a necessity if you want the training program to run smoothly. However it also helps to gather metrics to track progress against your success criteria (70–80% attendance rate) and is a must if you have compliance requirements. Tracking this data can be achieved in many ways but using Excel is probably the quickest and easiest to setup and manage, at least with a small number of students.
The spreadsheet ends up being your source of truth, so when it comes time to send out invites for the workshops you can filter out all the “done” rows which leaves you with a list of students to invite. The downsides to using Excel is that it’s hard to keep in sync when new employees join and leave the company or if you plan to run workshops regularly throughout the year.
When it comes time to send out the email invites for the workshops, it’s often good to give a high level overview of the training program and a brief summary of what will be covered.
Getting Feedback
Taking a survey, using a tool like Surveymonkey or Google Forms at the end of each training session is a very valuable exercise and will help you track your success criteria (>80% “would come again”). If you ask the right questions and make it quick enough to complete you will get some great feedback and insight into how well you have presented, how you can improve and about how effective it was run. Some questions to ask could be:
- What was your overall opinion of this workshop?
- Did you understand the content that was presented?
- How well was the content presented?
- Did you learn something new that will make you more security aware when designing, developing or testing software?
- Which topics did you enjoy / get the most out of / think are the most relevant to your day job?
- What did you think helped understand the concepts more, the defensive challenges, offensive challenges or a mix of both?
- What could have been improved to make the workshop better (e.g. more/less time, more exercises, more demos, too fast, not enough foundation knowledge, etc)?
- Any other feedback?
Most of the questions were multiple choice but the last few were free text but be warned that you will get some very honest feedback so be ready! Also it’s important to remember that everyone will be at different levels of knowledge and learn in different ways so it’s hard to please everyone!
“10 minute coffee break at the halfway point :)”
“Level 3 training room is the sleepiest room in the building!”
“I found we spent a lot of time setting up the env for the workshops.. If we could make this more streamlined then we have more time to work on the exercises.”
“Could spend less time explaining the breakdown of a URL or the structure of an HTTP request; that stuff is my bread and butter, I don’t need it explained again.”
“More exercises, more practical examples”
“This should be catered — Perhaps with a hamper to take home afterwards :P”
Ongoing Challenges
What happens when you have finished delivering the training? If they don’t keep these newly learnt skills fresh they will likely forget it in a few months! So what are some ways of keeping students thinking and using their skills throughout the year?
- Run a 2–3 hour workshop the last Friday of every month and do practical offensive and defensive security challenges with a scoreboard to keep it competitive.
- Send round a weekly security challenge via email where the first to solve it gets some points.
- Use the scoreboard mentioned above to promote earning points if students find valid issues in production code.
To improve your training program further you might decide to build more workshops that slowly get more advanced as the students skills improve. In addition you could develop a formal web security examination and/or certificate requirement which would further align with activities in the Education and Guidance section of the OWASP SAMM framework. Whatever approach you take there is no doubt that training is an important part of a company’s overall application security program.
Articles in this series
