Best Practices: A guide to coding with AI
I’ve spent the past year building dev tools to help developers code and collaborate with AI. In that time I’ve developed a list of best practices that I’ve found helpful when working with AI code generators. This article is for anyone looking to get started using AI code generators that wants to know what to expect and how they can get the best responses. I don’t discuss any specific tools here, these are best practices that apply generally when working with AI for code generation.
Useful Analogies
This is still the early days of this technology and it is far from perfect but it can still be very helpful. There’s two analogies I’ve found to be helpful when working with AI. The first is to think of it as a calculator for words. AI uses statistics to decide what word will come next based on the preceding words. It can get confused if the context or information it was given was incomplete. There have been many times when I’ve discovered that the quality of the AI’s responses dropped significantly because there was a mistake in my prompt. When writing your prompts be as clear as you can and include as much detail as you can. Too much detail is better than too little. Another useful analogy is to think of the AI as you normally would a paired programmer. We don’t always expect a human to get the answer right on the first try. You need to break the problem down into steps first, work through it and iterate until you’ve found your solution. The same is true when working with AI.
Use References
Refer to your pre-existing code as much as possible. Even if you’re starting on a completely new feature it helps to give the context of the environment you are coding in and your preferred coding style. Including all the relevant references when generating code will get you much much better results.
Use clear terminology
Whenever you can use clear, industry standard terminology when writing your prompts. The AIs understanding of the world comes from its training data. The more you can tap into its training data when generating the better your results will be. If you aren’t sure which terms to use, ask the AI.
Break things down
This has been said a lot but it bears repeating. You can get far better results by asking the AI to break the problem down step by step before giving the final answer. Some AI code generators will include instructions like this for you automatically when you submit a prompt. If I’m working on a complicated request I will often discuss the approach first to make sure the AI is on the right track before moving ahead with generating code.
Focus the AI’s attention
The ideal prompt contains all the information the AI needs to complete the task and nothing else. If you are writing prompts by hand this is easy to do, but if you are generating the prompts or using an agentic system it can be more difficult. A common mistake I’ve seen when working with agentic systems is for the AI to pick the wrong file for editing because it has very similar functionality in it to the correct file. You can avoid these mistakes by taking smaller steps to get to the end result. For example, let's say I wanted to update the character controls in a game so that the character moves forward faster. If you are unfamiliar with the codebase, immediately prompting to update the character controls to go faster is likely to result in a poor outcome. Maybe the logic that controls the speed is completely separate from the controller logic. Maybe there are different files for handling input from different devices such as keyboard for desktop or touch on a mobile device. A better approach would be to start by asking where you can find the logic that controls the characters movement. You can ask follow up prompts until you’ve found the right spot, and then ask the AI to make the updates once you fully understand the scope of the changes.
Iterate
It’s not uncommon to realize that our initial approach or assumptions were incorrect after we’ve written some code. This problem still applies when coding with AI, but the AI can help here as well. If you’re getting an error, share it with the AI. If you’re not getting the results you expected, explain what you are getting vs what you were expecting. The AI is often very good at improving upon its initial attempt when asked to try again. Once you’ve got something that works you can revise further to improve the code’s performance or maintainability.
Instruct the AI to give full responses
It’s not uncommon to get back partial responses when asking for code updates. It can help to instruct the AI to give complete responses. This is the prompt I use.
“Please provide the full response without omitting any parts. If there are parts of the code within your response that haven’t changed please include them in your response. You will be penalized for shortening your response by removing sections of the code that haven’t changed.”
Instruct and warn
When giving instructions you can often get better results by explaining not only what you want but also what you don’t want. Try to use the sort of language in the example given above for getting full responses. Avoid just repeating yourself, e.g. “add comments to the code, don’t add code without comments”. Instead use clear definitive language, e.g. “The code must include comments. You will be penalized if there are no comments that explain changes in the code.”
Know when to use AI and when you’re better off doing it yourself
It can take some time to get good at this. If you have some previous experience with the code you are working on it is not uncommon for it to be a lot quicker to update the code yourself rather than explaining to the AI what you want in a prompt. To give a really basic example let’s say you want to update a URL. Doing a copy and paste or a find and replace will be a lot faster than having the AI make the change. There are many changes, big and small, when it’s just easier to update the code directly. Tasks that are self contained and can be easily explained in a prompt are really good for the AI. Things that need to be crafted like setting up a layout are easier to do by hand. After working with the AI for a while you will get a feel for the sorts of tasks it’s able to handle and the ones you’re better off doing yourself.
Not all coding environments are equal
An LLMs proficiency in any given programming language varies depending on how well that language has been represented in its training data. Typically Python and JavaScript work best. Java can also be well supported. If the language you are using doesn’t appear very often in the LLMs training data it may struggle to output quality responses. The same is true for libraries. It can be common to get poor results and hallucinations when working with libraries that the LLM is aware of but has not been trained on.
Not all AI models are equal
There are many different AI models out there that can write code. Typically the bigger they are the more capable they are. Very large commercial models like those from OpenAI and Anthropic tend to top the leaderboards. You can check the EvalPlus Leaderboard to see how different models compare in their coding abilities.
Review and format your prompts
When putting together complex prompts you’ll save yourself time in the long run if you review them first. Parsing through 100 lines of code trying to figure out what went wrong is a lot harder than double checking your prompt. If you are generating prompts dynamically this is especially true. I’ve frequently seen response quality drop off due to a bug causing a problem with a dynamically generated prompt. It can be common for AIs to pay more attention to the beginning and end of a prompt. Try to put your most important instructions in these locations. Using markdown can also help direct the AI’s attention and give additional context. I’ve included a link to OpenAI’s official prompting guides at the end of the article in which they discuss different formatting techniques.
Writing code that is AI friendly
If you’re using an agentic coding assistant it will probably have a RAG system behind it. This allows the AI to be familiar with your codebase. These systems benefit greatly from having small code files that are self contained and easy to understand. Smaller files are also easier for the AI to update. By keeping the files small and well organized it makes it easier for the AI to find exactly what it needs and focus only on that, which improves the quality of its responses. Commenting complex or unusual blocks of code and workarounds can provide very helpful context for the AI and yourself. The same is true for logging. Having the AI output debugging logs can help you identify issues more easily as well as the AI if it’s able to see the output. This also encourages the AI to think through the problem from the perspective of how it will be executed.
IP Issues
Be aware of IP issues surrounding the use of AI. If you are working for a company or a client they may not want the code you write being used to train a public model. Many AI assistants use your interactions and generated code to retrain their models. If you are working on your own secret sauce you might want to be guarded about what you expose as training data. Some companies do not allow the usage of AI code generators because they are concerned it could be a copyright infringement. The court cases that will decide if copyrighted data can be used to train an AI are just beginning. You can follow the progress of the GitHub Copilot Intellectual Property Litigation case at https://githubcopilotlitigation.com/. There are alternative open source models like star coder 2 that are trained on training sets like the stack that allow open source developers to opt out if they want to, but they aren’t currently as capable as the commercial models from companies like OpenAI.
A note on tipping and emotional appeals
For those that aren’t aware there have been many reports that you can improve AI response results by offering to tip the AI or telling it to take a deep breath before it generates its response. There is some data to back this, see here. The data I’ve seen on this suggests that the responses are longer, not that they are higher quality. Whilst there could be some merit to the argument that the AI is mimicking human responses and humans respond better when they’re being tipped, I haven’t noticed much of a difference when using these techniques. Personally I would suggest being clear, direct and concise in order to get the best response.
Useful Links
Find out more about my work at oppydev.ai
