How to Prepare for a Blockchain Security Audit
Most developers understand that blockchain software should be treated as a safety-critical system. The cause of this is the unique combination of transparency, lack of access restrictions, and high-value transactions that occur on a blockchain. Security audits have become the norm, mostly at the smart contract level, but also at the protocol and operational levels.
However, making the most out of an audit is not an easy task. During our audit activities at Oak Security, we have noticed that teams sometimes struggle to choose the right time for an audit and prepare adequately. In this article, we answer some common questions and try to help teams to get the most out of their audits.
When and how to book an audit?
The first question teams normally ask is when to start reaching out to an audit provider. We have noticed different approaches in this, ranging from contacting us in the early idea stage to the other extreme, in which teams come to us on Friday asking for an “emergency audit” because they are planning to launch on the following Monday.
The best moment to approach an audit firm is when you have a clear outline of what you will develop and an expectation of when you think it might be ready. This is usually during the specification phase of a project. At this stage, we can already help with advice on the secure development process, tooling, and testing procedures.
Another reason to approach us early is that blockchain security expertise is hard to find and there are currently not enough experts to cover all projects, leading to waiting lists and pre-bookings at all top audit firms. We would love to be able to take on projects on short notice, but the reality is that we are usually booked up at least 6–8 weeks in advance.
On the other hand, reaching out early also introduces a few problems. It is hard to estimate the scope of an audit and a precise start date for the audit early on. Therefore, we might give you a price range at first and then narrow this down as you progress through development. Similarly, start dates are not set in stone. There is no point in auditing a moving target. If we agree on a date and the development team hits a delay, we can usually accommodate changes as long as everyone involved is realistic and we are informed in a timely manner.
Requesting a quote for the audit
We have provided a short checklist to helps teams send the right information to us when requesting a quote:
Starting the audit
When the moment of starting the audit has finally arrived, the auditors require the following:
- Full documentation and specifications. We do assess documentation in our audit reports, along with code quality and readability. In addition to this, our auditors need to make sure the code implements the intended model correctly. We can only verify that the implementation matches a specification if we have access to the specification. This is particularly true in the case of economic models based on mathematics. We can check your formulas for dangerous edge cases, but we cannot be sure each behavior is according to the team’s intention if the model is not documented.
- A frozen version of the source code, ideally in the form of access to a version control repository on GitHub or similar. A commit hash that will be used for the audit report. If you are still applying changes, you are probably not ready for the audit.
- A build and deployment system that works out of the box. The auditors can lose a lot of time if the deployment scripts for a local setup are not clear, or if dependencies are broken. Before submitting for an audit, we recommend trying to set the system up from scratch on a clean system.
- Runnable tests Testing should have been completed and reproducible. Our audit reports assess test coverage and try to measure it objectively. The reason for this is simple: Some bugs are incredibly easy to detect in testing, but hard to detect in an audit, and vice versa.
- A contact person that can answer questions in a reasonable timeframe. We will usually set up some form of communication channel that the auditors can use to ask questions or provide early feedback. Make sure someone on your side is monitoring the channel and can provide answers quickly.
We have prepared the following pre-audit checklist:
During the audit
We use a process based on the methodology pioneered by our Ethereum-specific partners Solidified. This process distinguishes itself from others by making use of a number of auditors working in parallel, independently, at first, applying their own way of working within our macro process. This means that our security audits all follow a basic process, but within this process, we get exposure to different tools and methodologies. Auditors may ask questions to the team at all times to clarify the intended behavior of the contracts. However, they are not allowed to share any security findings amongst themselves until later in the process.
We will describe the more technical aspects of auditing in a future article, but at some point, auditors will come together, discuss their findings, reach consensus, and produce an initial audit report.
That initial audit report lists issues and recommendations classified by severity. The project team now has some time (up to three weeks in our standard service engagement) to fix issues or acknowledge them with a reply. The auditors review these fixes and mark them in the next iteration of the report as resolved. There may be several rounds required but eventually, all issues should be marked as either resolved or acknowledged before a final report is published.
We allow the team to provide an official team reply to each issue, which is added to the audit report.
At the end of the whole process, the final version of the audit report is published on our GitHub repository. Not all audit firms make their report public, but transparency towards the end-user is one of our core values and we only ever compromise on this condition in very specific cases, in which disclosure would be decremental to security. However, it is rarely the case that security by obscurity is the best solution and we are very strict about making audit results public.
Please reach out
We hope this article helps project teams understand the requirements for an audit better and helps developers to make the most of their audits. If you have concerns about your security and/or are in the early stages of building on blockchain technology, please contact us for more information!
