Is It Time to Get an AI Assist on Your Coding?

Short answer: That depends. Long answer: Let’s dig in!

AI tools for generating code have reached a tipping point in quality and affordability. They have the potential to act as a force multiplier for software engineers across all industries and skill levels, and it looks like 2023 will see widespread adoption. Early experiments using Github CoPilot have shown developers productivity increasing 55%, according to research conducted by Microsoft, Github, and MIT’s Sloan Business School.

Here are some questions an engineering organization needs to consider when thinking about coding with an AI boost.

Is code generated by AI good enough to use?

This is a trick question. And a critical question to consider.

AI coding tools are quickly changing the game. At the very least they help reduce the grunt work, and with the right approach these tools can dramatically speed up software development. But today, an engineer needs to use judgement and experience to evaluate whether the code generated by an AI tool is good enough to use as-is.

The quality of these tools is improving rapidly and will at some point consistently outperform human engineers’ ability to write code. But even then, a human’s judgement will be necessary to review that code. And the faster you go, the more code is generated at a time, the more danger there is for a mistake to be missed by the human.

Most development teams have a “two pairs of eyes” rule for human generated code, whether that comes from pair programming or code reviews. For a long time, we will need to take at least this same level of care with code produced by AI. For every line of code it will be important for someone to ask the question “Is this code good enough to use?”

Can you generate code without sharing confidential or secret information?

Technically, the answer to this question at the moment is “No.” But that may be okay.

In order for tools like CoPilot or Code Whisperer to work, they send your code to a service so that it can analyze and respond. Part of this risk is the same as in using Github or Sonar Cloud — Your code will be transmitted over the internet and stored on a third party’s servers.

In addition to the possibility of your code itself leaking, AI code generation tools add the risk of leaking your code or trade secrets if your code is used to train the tool’s models. We have seen this happen in these early days of ChatGPT with some Fortune 500 companies’ code becoming part of ChatGPT’s responses. Currently, the bigger code generation tools allow you to opt-out of such training, or have blanket policies against it.

Each service includes different levels of protection for your data throughout the process. You will need to review these protections and make a decision to trust the company offering the service as well as the safeties they have put in place.

Is it legal and ethical to use code generated by AI?

I am neither a lawyer nor an ethicist. But I can highlight some questions you will want to answer before attempting to answer the bigger question.

To determine whether it is legal to use AI generated code, we need to consider copyright and ownership. Broadly, these issues are still being considered by the courts. They are also issues that will change as new tools, models, and approaches mature. There are three important questions, here, to consider:

• Was the material used to train the AI copyrighted and/or properly licensed?
• Is the code generated by the AI copyrighted?
• Is the license and ownership of the generated code compatible with the code base it is being used in?

For many of these questions, you will need to trust the AI tool creators when you consider using generated code in your codebase. Choosing a trustworthy provider will become ever more important as more products arrive on the market.

Providers may soon begin to offer commercial assurance with their services, warranting that their product and its output do not violate a third party’s intellectual property rights in much the same way Red Hat does for their products. Without this, or similar indemnification, it is important to think through the risks.

Beyond the basic questions of legality, we will also need to wrestle with the ethics of AI code generation. Today we can, and should, carefully review any AI generated code before we incorporate it into our code bases. But soon AI will be writing complex code that stretches (or surpasses) our comprehension. At that point what should we consider before we incorporate that code into an applications?

Bonus question: Will engineers trained by AI be able to determine whether code generated by AI is good enough to use?

A newly popular use case for AI code generation tools is to assist engineers learning new technologies. An engineer familiar with Javascript who wants to learn C# may use an AI tool in their IDE to solve a problem in C# that they could easily code in Javascript—then study the C# code as a learning exercise.

From an immediate perspective, this is a great use of technology. It’s like having a private tutor who never gets impatient with your questions.

Long term, using AI to train humans has the potential to create a closed feedback loop where engineers learn to prefer code generated by AI. This may or may not be a bad thing. The perennial question of “what is good code?” will become an even more important topic as AI code generation gains popularity.

The Path Forward

Good engineers develop a tool belt of digital literacy skills throughout our careers, learning to filter through research, articles, Google results, Stack Overflow questions, and many sources of information with a critical eye. Those same skills, with some important sharpening, will be important for us to filter the current output of AI. Not only does this include code and content generated by our own prompts, it also includes ghost-written articles beginning to flood search results.

Most of us have seen someone make the mistake of cutting and pasting code from an online source without a full understanding of it. Using AI tools will present the same dangers, with a bit more camouflage. We have had to develop the ability to judge the validity and reliability of each source of information. We will need to hone these skills, and develop new ones, to accurately judge the validity and reliability of AI generated content.