Part 1: A pragmatic guide to building your bug bounty program

Getting set up, and maintaining your program

A bug bounty program is a great way to improve the security of your product with relatively little investment. This involves allowing the security community to try to exploit your product in a sanctioned way, effectively crowdsourcing part of your application security and penetration testing. In return, you offer a reward for significant findings, motivating 1000s of security researchers across the world to find and disclose vulnerabilities.

As with anything you open up to the world (particularly if it offers rewards), you’ll end up with some good spirited people who will help make your product materially more secure, and some who want to exploit your program.

Having run our bug bounty program for two years, I’ve found that the key to success is having the right checks and balances in place to reduce noise and beg bounties, encouraging higher impact reports, and ultimately making it an efficient use of your limited time.

In this series of blog posts, I’ve provided a pragmatic approach to starting your bug bounty program, targeted particularly at startups. I’ve addressed the pitfalls you might experience, and offered suggestions on how to maximise the number of useful reports. This series covers:

• A guide on how to get your bug bounty program live
• How to operate and maintain your bug bounty program
• Deciding how much you should pay
• How to deal with beg bounties (not a typo!)
• Opening the door to higher quality bug bounty hunters

This post will focus on getting your bug bounty program up and running, and the important aspects of operating and maintaining it.

How to get your bug bounty program up and running

When you get your bug bounty program set up for the first time, you will want to make sure you do the following:

• Make sure bug bounty hunters are targeting a demo environment, or have strict controls in place to prevent them impacting real users. If you’re unable to use a demo environment, set up explicit targets in your production environment and ensure that you document what is and isn’t in scope
• Let your engineering team know that you’re setting this up (and that, no, they’re not eligible for rewards — no writing bugs for a bonus!). This is a good way to build security culture and prepare them for fixing any findings. It’s also a good idea to align on SLAs for security fixes if you haven’t already
• Talk with your legal team about what you’re doing. They can help build out the terms and services for your bug bounty program, and will be able to ensure you’re operating in a compliant way — particularly navigating between malicious and beneficial exploit attempts and ensuring you’re meeting your company’s compliance requirements. You will want to partner with them on building the terms for the program to help set the scope, and the explicit findings that are in and out of scope — feel free to build on top of ours
• Make sure your finance team knows about the program and what this might mean for them. They may need to make international payments to some bug bounty recipients and you will want to establish a budget with them. If you’re wondering how much to budget for — check out the “How much you should pay” section in Part 2. Additionally, in your bug bounty terms, you should remind recipients they are responsible for all taxes
• When paying out a bug bounty you’ll likely need to perform some AML checks on the person you are paying. This helps to ensure you’re not breaking any money transfer/payment laws or sanctions. HR, legal, and finance teams may have ideas on how to do this efficiently. You’ll also need to collect their bank details — finance teams usually prefer bank transfer over Paypal or similar.
• Set up a security.txt page. Google’s is a good example: https://www.google.com/.well-known/security.txt. You should host this at yourdomain.com/security.txt or similar
• Create a mailbox to receive reports, e.g. bugbounty@yourcompany.com. We also recommend setting up an automated reply with SLAs for responses and reminding people of the terms and conditions. In particular, adding a list of out-of-scope findings to your auto-reply will address beg bounties and low-value findings without manual input
• Build a page on your website to provide terms and instructions on how to participate in your bug bounty. We put ours in our security centre
• Ensure you have the resources to handle the findings and reports. Initially, you should expect to dedicate about 20–30% of 1 person’s time on the bug bounty program. This will involve figuring out which emails to respond to, working with your engineering team to coordinate fixes, responding to the bug bounty hunter to validate and remediate issues, and sorting out payment afterwards. As your bug bounty program matures, you’ll spend less time on this — we noticed we were much more efficient after about three to four weeks
• Make sure the most visible employees in your company (usually the CEO/founders) know about this program so they can help redirect queries accordingly. Despite documenting the correct path for vulnerability disclosure, we still see reports to a range of people at Airwallex
• Set up somewhere to track security vulnerabilities internally — we use Jira. Make sure you can use this ticket as the source of truth for all vulnerabilities found and ensure it’s easy to add the relevant software engineers. We have our bug bounty mailbox auto-forward emails to a second email address which creates Jira issues. This lets us have a purely internal ticket per report that we receive, and we’re able to track the status internally. Alternatively, you could communicate with bug bounty reporters directly through tools like Jira or Zendesk
• Track your bug bounty reports to avoid duplicates and provide metrics on how effective the program is. Our bug bounty reports are all stored in a Jira project — finding duplicates is easy, and we’re able to get good metrics about the number of impactful reports we receive and how much we are paying out

How to operate and maintain your bug bounty program

As with anything you do in security, once you build it you need to maintain it. Operating your bug bounty program takes resources and is particularly expensive initially as you spend time getting the program set up and learn what works for your team and company. Automation is a good choice here, however I’ve found it best to operate things manually for the first few weeks to better understand your program, then set up low-cost automation to handle common manual tasks.

As an example, one of the first things we automated was our auto-reply responses to submissions. This auto-reply lists our terms and out-of-scope areas to discourage low-effort reports, providing the caveat that if it’s out of scope, we typically won’t respond. These small pieces of automation save us from manually responding to submissions which are out of scope, reducing the total volume of reports requiring attention by about 50%.

You’ll also want to build an SOP for processing bug bounty reports internally, including the relevant teams. Having this documented helps other teams understand what’s required of them, and gives a consistent experience to bug bounty reporters. The diagram below shows our rough process for triaging, analysing, remediating, and rewarding bug bounties:

In your SOP — you should also decide whether you want to identify your security team individually when communicating with bug bounty hunters or use a generic email address. If the issue is legitimate and interesting, our team generally introduces themselves and learns a bit more about the person submitting the reports. This helps build relationships with people who are ultimately helping make your organisation more secure, getting more of their time and increasing the number of valuable reports we receive. For initial engagements or low value findings, we respond using our bugbounty@ mailbox instead.

Validating reports

When validating whether a reported issue is a legitimate vulnerability you will need to go deeper than just what’s in the report. The person submitting the report doesn’t have access to your engineering team and documentation so they’re not able to look at the code and mitigation to understand what the root cause is, and see where else in your code base the same issue might exist. When talking with your engineering teams about the vulnerability, ask them to scope the impact. Having them determine the blast radius of a bug — including where else the bug might exist and what an attacker could do above what has been reported — is a good way to educate them and reduce the likelihood similar bugs occur in the future. The increased accountability is a bonus too. Finally — ask the bug reporter to validate the fix too.

Vendors vs doing it yourself

Bug bounty program vendors, like HackerOne and BugCrowd, can be effective to take a lot of operational burden off your hands. Generally, the way they work is by handling the triage phase of reports and getting better report quality through targeted pentesting of your products, by reputable bug bounty hunters on their platform. You pay more for their services (effectively paying a premium on top of the bug bounty amounts), but you are paying to reduce workload on your own team which is likely your most scarce resource. You can also explore working with vendors to set explicit targets for your bug bounty program, such as number of findings and money awarded. We run our own bug bounty program, but it’s worth getting in contact with vendors to see if they’re a good fit for your team and security program.

Lastly, if you have a bug bounty hunter submit a legitimate and interesting report, make sure to note their name and contact details! They’ll likely pop up again in the future and you’ll want to encourage their engagement. For more details on this, look out for the “Opening the door to bug bounty hunters with better findings” section in part three.

This is the first entry in a three-part series on starting a bug bounty program. Continue reading Part 2: Budgets and payments, and dealing with beg bounties here.