How much is enough training?
I’m often asked by CISOs how much security training is enough for employees? How many hours and how many modules of security training should we be giving them?
These are not the right questions to ask. It’s the equivalent of asking how much should I be exercising. The answer depends entirely on what you are trying to achieve. Keeping with the exercise analogy, are you trying to lose weight, get your daily dose of activity, or get fit for a competition? For security training, after your employees finish what is it that you want them to do differently?
Let’s look at the four key steps to creating and running an effective security training program that get to actual results.
- Identify your end goals and desired behaviors
- Find the right solution to address that behavior
- Prove effectiveness at a small scale
- Use measurements to roll out the training to a larger audience
Identifying Security Behaviors
In order to begin to understand how much training is enough, security teams need to define their goals and what key security behaviors they want to influence. Let me explain what I mean by key security behavior. This is a defined, measurable set of actions that lead you to the results you want. Change efforts often fail because they either focus on the results but don’t identify the specific actions to get there, or they spend time and energy on a bunch of actions that are good ideas but aren’t the key few. Examples of key behaviors could be reducing the number of people executing malicious software or increasing the reporting rate of an organization.
Finding the right solution
With your behavioral outcomes in hand, your next task is to find the solutions that effectively start transforming employee behavior in the right direction. The solution may be training or it may be another approach altogether such as positive incentives, management support, more project time, etc. If you can’t show that a funny security video about URLs reduces the number of people who navigate to malicious sites, then you shouldn’t spend time getting as many people as possible to watch the video.
Once, when creating a tailgating behavior change campaign I discovered that employees who weren’t wearing badges knew all about the company policy requiring them to wear badges visibly. The reason they weren’t wearing them was because the badge-pull they were given as new hires had broken and they had no way of attaching the badge. As a result, they kept their ID’s in their wallets. No amount of training would have fixed that core problem.
Proving effectiveness using A/B testing
When you do determine that a training is needed to change a behavior, the next step is to ensure how effective that training is. In a past life, I ran a large scale security awareness programs at Salesforce where I used A/B testing methodology to test effectiveness. I’d first take a small pilot group (3% of the total target population) to administer the new training and test that group beforehand on the behavior I was interested in improving. For example, I’d send employees a phishing email with a malicious link to see how many people would navigate to the site and take action. I would also send that same test to a group of employees who were not part of the pilot group. The goal of this step was to get a baseline measurement. I would then administer the training and run a similar test upon completion with graduates as well as employees who didn’t go through the training. I then compared our results to look at the effectiveness of the training as it related to reducing phishing clickthrough. In this case study, I found that the alumni of our simulation training were 50% less likely to get click on links and 82% more likely to report suspicious activity than non-participants.
Rolling out to a larger audience
Managers are interested in protecting their employees’ time and brain cycles and therefore rarely have interest in championing an ineffective security training for their org. Armed with data about the effectiveness of a training against a concrete behavior, it is significantly easier to get cross-organizational buy-in for employees to complete a training. The message between the security team and other business leaders could unfold like this:
“Malware execution is a risk to our company as seen by data from our Incident Response team. Our security team has a training that will improve our resilience to this by X% based on a pilot we just ran. I’d like your support in getting your organization through this training by the end of the quarter to reduce our susceptibility to this risk.”
Finally, if you continue to measure the impact of the training on a larger audience you will be able to provide senior leadership with measurements on the security improvement of each business vertical. This is an excellent opportunity to introduce elements of positive reinforcement for top performers and competition amongst leaders.
Do Less! Get More!
When security teams are able to answer what behavioral outcomes they want from a training, they may discover that there are employees who don’t need any training at all. There may be roles in the company who do not need to be proficient at a specific behavior. Additionally, if security teams measure the desired behavioral outcome of a training before administering it, they may find that there is a subset of employees who already exhibit the desired behavior (ie don’t click on links) and may not need to go through the training. They’ve already passed the final exam and can stand up to real world attacks or write secure code. Perfect! This can save you and them time and effort.
By taking this approach to training, you’ll find that both the security teams and employees’ time and intelligence will be spent on improving the security posture of the organization instead of just “getting through it”.
