6 buckets of prodsec

Collin Greene
Draft · 2 min read

A product security team exists to protect people and their data.

We do this by making our companies products as secure as possible. Software security is hard though. As a species we are not capable of writing vuln-free software. Hell, we cannot even determine how secure or not it is with much certainty.

Given that, our goal should be to maximally reduce risk. I do this by tracking the outcomes of vulns.

There are 6 buckets a security bug can fall into on its journey through life

  • Prevented — best outcome. Never turned into code.
  • Found automatically — Found via static analysis or other tools. “cheap” time cost.
  • Found manually — Good even if it took more time. A large set of bugs can only be found this way.
  • Found externally — usually via bug bounty. Put users at real risk, expensive time cost but 100x better than other outcomes.
  • Never found —Most bugs probably end up here.
  • Exploited — the worst.

Prevented > autofound > humanfound > externally found > unfound > exploited.

Whenever we shift a vuln from leftware on this spectrum, we are winning.

  • Doing a security review of a sketchy part of the codebase? You just moved n bugs from never found -> found manually. Success!
  • Writing a static analysis rule based on a bug bounty report? Found externally -> Found automatically. Success!
  • Handled an incident where bugs were exploited, learned the lessons from it and turned it into rules, tools and secure frameworks? Success.

We aim to shift as many bugs as possible, as far to the left as possible. That’s it, that is the whole job.

Methods

Here are lots of methods to nudge future bugs towards a given outcome bucket

  • Prevented: safe by default languages and libraries, design reviews, training & documentation
  • Found automatically: static analysis, dynamic analysis, lint rules, tests, fuzzing.
  • Found manually: Security reviews and consulting, self-service tooling
  • Found externally: Bug bounty, external scanning services

Each method has a cost (time, $$$, friction to company), a return (# of bugs shifted and how far). Each company, situation, team has different trade-offs here.

Conclusion

We will still mess up sometimes and bugs will go unfound until they are exploited. It happens.

This is painful but security is not an activity where anyone can bat a thousand. What we can do is put our limited resources behind the methods that shift the most bugs, practicing outcome-based product security.

Yes this post is absolutely a rework of old ones (1, 2).