An Analysis of the Security of Generated Code
GitHub Copilot aids developers with basic programming tasks. Useful examples are finishing code, eliminating syntax errors, linting, writing unit tests, generating functions, and reducing code complexity. However, Copilot often provides suggestions focused more on code function, not code security. To generate secure code, Copilot needs to be instructed how to do so. Copilot is a great tool to generate code but should not be considered secure by default. In other words, Copilot tends to assist with syntax over semantics.
For example, Copilot will not implement a specific design pattern without being told explicitly what pattern to follow either through engineering a proper prompt, or though the current context of the application.
Refactoring code also still requires extra effort. Tools such as ChatGPT and other similar “chatbot” style AI solutions make refactoring code simpler by using prompts such as “Refactor the following code to use X rather than Y” followed by a snippet of code. With Copilot, code blocks must be manually deleted and/or modified line-by-line. Therefore, lines of code must be evaluated individually. However, further developments may make refactoring with Copilot easier.
To produce secure code, an experienced developer must direct Copilot and verify the generated code to produce a result that utilizes best practices for application security.
- Write clear, concise, and precise prompts: Vague or overly complex prompts are less likely to generate the desired code. Tell Copilot what needs to be done, and how it should be done.
For example, let’s look at a prompt that is considered vague.
While the generated code does take in a string and hash it, SHA-512 is probably not the best hashing algorithm to use in this instance. A better option would be to use Argon2id, or scrypt if not available. Bcrypt is also a viable choice for legacy systems. Writing a more precise prompt specifying the hashing algorithm will help Copilot to rewrite the function to fit specific needs. Or instead of asking Copilot to “rewrite the above function,” remove the generated code and modify the prompt to use a specific algorithm.
Remember Copilot should be used to assist in syntax, not semantics — and does not prevent insecurity by design. For example, if Copilot is asked to run a command on a local machine, the generated code will do exactly that.
2. Consider the type of data being stored, transmitted, or processed: Data classification and how data is being validated needs to be considered. Various types of PII (Personally Identifiable Information) may be leaked by blindly trusting generated code. For example, are usernames and passwords being logged? What type of information is being transmitted unintentionally? Validation is also key; is the generated code providing validation that matches with the use case of the data being processed?
3. Design with security in mind: When using code generation or developing software in general, consider security from the very beginning. When engineering prompts, include appropriate validation, filtering, and data protections. This can cut down on later bug fixing, refactoring, and modification due to insecure generated code. A “secure by design” mentality should be used.
4. Copilot is not a replacement for other SDLC activity: To ensure secure code, other security measures should be performed. Including SAST (Static Application Security Testing), DAST (Dynamic Application Security Testing), and SCA (Software Composition Analysis).
5. Copilot should assist the developer, not be the developer: Copilot is a great tool for generating code; but code generation does not replace application security knowledge. The developer should have a basic understanding of how to perform a task. Copilot can be used to assist in generating code for implementation. Be mindful that Copilot is a tool useful for assistance with syntax not semantics.
6. When in doubt, ask questions: If you are unsure of the security of the suggested code, ask questions. Perform due diligence by performing your own research about latest best practices and guidelines.
While GitHub Copilot can increase productivity for developers and aid in day-to-day work, it is still imperative to verify generated code. Like any form of input, validation is required. Always design with security in mind to use GitHub Copilot to its fullest potential without sacrificing the integrity of an application’s security.
Authors: Anthony Cleary (Product and Application Security Engineer) & Stephan Heimburger (Cybersecurity Architect)
Reviewers: Randall Batridge (Senior Sales Engineer) & Josh Gaddy (VP, Director, Developer Advocacy)
