In my first few months at Facebook I found 24 security bugs which was a respectable haul. The following quarter we launched a bounty program which promptly found 71 bugs. The pragmatist in me hated, but could not ignore, that my time spent fielding bounty reports resulted in more security than spelunking through the codebase.

So began my professional relationship with bug bounties.

The following is my advice on how to prepare, launch and run a high quality bounty program. It comes from the many mistakes I made in launching and leading the Facebook and Uber programs. Despite some complaints I am a fan of bounty programs and ultimately see them as a cause for hope

Why I like bounty programs

Security is hard in a large part because it is unfalsifiable. I cannot say with conviction that I’ve found all the bugs in any given program. So I try to stack the odds in my favor by layering systems that surface bugs: design reviews, code reviews, type systems, program analysis, external security audits, blackbox scanning, etc. Bounty programs sit at the end of this process which means every bug found is one that slipped past everything thrown at it.

I witnessed my first real security effort at Microsoft in early 2000s. After slammer and code red Microsoft made the unprecedented move of investing a billion dollars into security. Turning this billion dollars into “security” manifested itself as vacuuming up all the security talent they could find, sticking us in a room together with the code and basically crossing their fingers that good bugs shook out.

In contrast bounty programs are cheaper, produce a roughly equivalent quantity (if not quality) of bugs and companies don’t have the headache of managing a horde of consultants.

An operating system is not the most amenable target but if done today I bet lots of those same bugs could be found via a bounty program (of which Microsoft has a few)

Having a large pool of people with different perspectives and knowledge attack your software brings a diversity of bugs that has surprised me. You get this in spades with a bounty program as each researcher has their own areas of expertise.

Bounty programs are a good fit with modern development practices. In the old days you planned n months of software development, you hit your deadlines, it went to QA, then it went in a box. Today everyone does agile, ships faster and QA teams are near extinct¹. Bounty programs fit this environment like a glove as fixes can be pushed quickly and act as informal QA.

That said a bounty program doesn’t replace anything you are or should be doing security-wise, it augments it. It is the icing on the Secure Development Lifecycle cake. A bounty program does not replace consultants, they are different tools to achieve different things. If you want a whole lot of bugs though bounty programs are a good bet however.

Bug-elimination research, like other user-interface research, is highly nonmathematical. The goal is to have users, in this case programmers, make as few mistakes as possible in achieving their desired effects. We don’t have any way to model this — to model human psychology — except by experiment. We can’t even recognize mistakes without a human’s help.

DJB

How to prepare

You are onboard and are eager to launch. Awesome. Here is what I would do.

Bug bounty skillset

The people on your team signed up to be security engineers and their job just became part customer service. The skills required are:

Its alive!

You are up and running and the bugs are falling like rain. It is chaos just keeping up but its good chaos. The first few months are all about just keeping up.

The hope is that over time bugs in your software will become harder to write and you will be compelled to raise payout amounts for your program.

The suck

Things will go wrong. It’s ok. It happens.

Keep it spicy

Run a solid, fair, competent bounty program and researchers will keep coming back

Here are some things I’ve tried and how they fared

Cause for hope

Bounty programs improve security and that is swell. More powerfully I see them as the way to train the next generation. I was profoundly lucky to have wonderful mentors who answered millions of my questions on irc or at 2600 meetings. Even with such benefits when I wanted to Actually Hack Stuff I had to go do it, and take on the commensurate risk.

Coming up today I would spend all my time on ctfs and bug bounties. Getting to hack real things with no risk of prosecution and a shot at some money for my efforts is a great thing. Bounty programs are a safe and sanctioned way to get into security. If you run a bounty program look for teachable moments in responding to a researcher who may not fully understand something.

We as a security community are a small tribe and bounty programs are the practical ladder by which much of the next generation will get into security professionally.

In a sea of snakeoil and grand claims a robust bounty program is a statement of confidence in your security —openly inviting scrutiny is admirable.

You will find bugs. You will become less of a faceless corporation to the security community. Tracts of your codebase will be illuminated as needing security attention. You will be surprised by the things you missed and become a better security engineer for it.

I do not speak for employers past or present. I thank Ryan, Katie, Erik, Zac and Nathalie for helping me on this article.

Footnotes

  1. A problems of its own in my opinion.
  2. The story here was someone submitted a claimed XSS, admitted it doesn’t fire, starts a reddit thread saying Uber bug bounty is a scam with a big donate here link. Takes it all down out of shame a few hours later when the report is made public.