Thinking About Security and OKRs
I love OKRs.
I love their clarity.
I love their focus.
I love that, if correctly done, departments and individuals set their own goals and hold themselves accountable. Goodbye, Manager. Hello, Self-motivation.
I’ve seen OKRs work for engineering and for product. I’ve seen them work for sales and for marketing.
Making them work for security, however, proved difficult.
At first glance, OKRs for security seem easy.
A quick definition of OKRs: An “objective” is a statement about the world, and a “key result” is how you know that it’s true.
So for example, the marketing department might have an objective of “Marketing Efforts Support the Sales Team,” and you will know this is true (i.e. a key result will be) when 10 marketing-qualified leads are generated in the next quarter.
If engineering wants to have a “Highly Performant Web Application,” they might measure page load time, or server response time, or time to first byte.
If product wants to “Delight Customers” they might might measure net promoter score, or daily active users, or average time on site.
Security objectives are obvious. They might be:
- User accounts are secured against unauthorized access
- The company office is secure against data leakage
- Our cloud is securely configured
That pesky word– secure –appears in each objective. How can you definitively measure the presence of absence of ‘secure’? What key result will say whether or not security is truly present?
Number of breaches? That number should be zero.
Number of vulnerabilities? That number should also be zero.
It is incredibly difficult to run a department if your ideal output is to have nothing happen.
Everybody has a Plan Until Someone Starts Phishing Your Users — Or — The Strange Thrill of Seeing Your Security Tested
Without saying too much, last year I learned that security through obscurity only last so long. At some point, those gates you put up to keep the barbarians out gets tested.
I will not deny the anxiety this caused: it felt awful to know that there were Bad Actors working to corrode the company that the team and I had worked so hard to build. Conversely, this provoked a strong sense of duty: it was on us to protect our customers and to provide a tool with best-in-class security.
And strangely, there was also the thrill of discovery.
For the first time, I had a clear measure of what had (and what had not) made us more secure.
And now I’ve realized the key to setting good security KRs is to focus on that clarity.
Developing a Sense of Touch with Baseline Metrics
Imagine you were learning to bake croissants. If you follow best practices, you’ll have something delicious and flaky. If you screw something up, you’ll end up with a greasy brick.
Security is not like that, though. If no one tries to inject sql, then no sql gets injected, regardless of the presence or absence of a vulnerability. You could screw things up, and purely by chance end up just fine.
This is disorienting. Learning depends on seeing the outputs of your actions.
Let’s take the objective from above, “User accounts are secured against unauthorized access”
How do you know if this is true? Unfortunately, you don’t. And honestly, you can’t. Not totally. Not in perpetuity. But you can observe user behavior. You can be intimately familiar with how they authenticate, and thus finely aware of changes in this behavior.
To do that, you must record metrics like…
- Login Attempts
- Successful Logins
- Failed Logins
- Password Resets
- 2FA Sent
- 2FA Entered
- 2FA Prompts Abandoned
- User attributes including IP and browser
Now, imagine that somewhere on the internet, a Bad Actor buys a set of stolen email addresses and passwords. Now, imagine that this Bad Actor attempts to use those email addresses and passwords to log into your website (aka credential stuffing).
You would see the above numbers spike. Hopefully, you would have tooling in place to identify which credentials are being repeatedly stuffed into the login. Likewise, you should be able to notice treads in these logins (repeating IPs, traffic always coming from data centers, and so on).
You know the statement, “User accounts are secured against unauthorized access” is true because your team is able to identify and mitigate an attack. That is your key result.
