Computers write code. For real?

Developers love to code! And so do we. But there’s always something you hate in the job you love, isn’t it? Imagine, you have this great idea of a new feature for your app — your customers will really enjoy it, it’s a fantastic improvement! You start to code, and then… Wait! How do you really call this API? What was that optional parameter that is actually required in some cases? Let’s try a web search, look through the docs, read some blog posts, check out a Q&A site… FEW HOURS LATER: ah, that’s it! Found the solution. If only there was someone or something who could give a quick hint on how to use that API…

Wait a minute! Recently, GitHub announced the general availability of its Copilot tool, an AI pair programmer as they call it. Could it be helpful? We decided to give it a try. And while playing with GitHub Copilot, we also discovered that AWS announced a new service — Amazon CodeWhisperer, an ML-powered coding companion. Wow, that is going to be interesting! As you know, we are an all-in AWS company, so we definitely wanted to try this AWS offering too. So, buckle up — let the computers write some code for us!

Computers write code!

Disclaimer
I Love My Local Farmer is a fictional company inspired by customer interactions with AWS Solutions Architects. Any stories told in this blog are not related to a specific customer. Similarities with any real companies, people, or situations are purely coincidental. Stories in this blog represent the views of the authors and are not endorsed by AWS.

How it works

Both tools work in a similar way. They are powered by machine learning models trained on massive amounts of existing code, including code in public open-source repositories. Additionally, CodeWhisperer seems to use some training data from the Amazon code as well, as stated on the FAQ page.

It’s like code completion on steroids! You can write some initial lines of code or just describe your intent in a comment (in English), and then hit Enter. The context at your current cursor position is analyzed (including any existing code in the file, the comments, imports or dependencies). Finally, you get a code suggestion for the next line, or a complex multi-line statement, or even a complete function. Sometimes you get multiple suggestions so you can browse through them and decide which one is in-line with your thoughts.

At the time of writing, CodeWhisperer supports Python, Java, and JavaScript. We also wanted to try it with C#, so we are hoping C# will be supported any time soon. Copilot supports more languages, including C#. However, Java is not listed as a language Copilot was optimized for. We will find out if it is a big deal. Our goal is to get help from an AI coding companion in both Java and C# — the two programming languages we mostly use at I Love My Local Farmer. In this blog post we describe our experience with Java.

Installation

Both tools have to be integrated into your IDE. Copilot has to be installed as a dedicated IDE extension and supports Visual Studio Code, JetBrains IDEs, and Visual Studio. CodeWhisperer is included as part of AWS IDE Toolkit, so if you are already using the Toolkit, there is no need to install another extension — just update the Toolkit to the latest version. CodeWhisperer integrates with multiple IDEs as well — including JetBrains and Visual Studio Code. There’s also support for AWS Cloud9, and even the AWS Lambda console. In our Java teams, we use Visual Studio Code and JetBrains IntelliJ IDEA, so we only tried Copilot and CodeWhisperer there.

For Copilot, you also need a GitHub account. The billing and some configuration options are handled through that account. For CodeWhisperer, since the service is in preview at the time of writing, you don’t need any specific account. We had to request the preview access though by filling out the form as explained in the service FAQ. We got a preview license key in several days after submitting the form.

Implementing an AWS Lambda function handler

In one of our previous blog posts we described how we started going serverless with one of our Java apps. Can we make the life of our developers easier for the next migrations, if the teams will use CodeWhisperer or Copilot for writing serverless code?

We’ll try to write a Lambda function handler that integrates with the API Gateway and implements a single API method — uploading a file to an S3 bucket. The file itself will be provided in the query string parameter as an URL. This API could be used by our Delivery App for uploading file attachments.

Amazon CodeWhisperer

Let us start with a blank Lambda project for Java. Since the context is important for getting helpful suggestions, let’s import some types first:

import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestHandler;
import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyRequestEvent;
import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyResponseEvent;

Okay, now we have a pretty obvious context. We indeed intend to write a Lambda function handler that uses the API Gateway integration.

Let’s try to express that in a comment:

// AWS Lambda function handler that returns HTTP 200

We are now excited to see what CodeWhisperer will suggest us.

Well, it seems that CodeWhisperer got it right — it offers a class that implements a Lambda handler using the API Gateway integration. A bit disappointing though we did not get the whole implementation at-once.

By accepting suggestions (by hitting the Tab key) and hitting Enter a couple of times, we ended up with this:

Pretty impressive. Yes, this is a very basic implementation, but we didn’t actually type a single keyword.

But what if we don’t provide the context at all? Can the AI model guess what we need if we only describe our intent in a comment?

Starting with a new blank Java file again, this time the only content will be this comment:

// AWS Lambda function handler with API Gateway integration

CodeWhisperer suggests us a couple of imports, line by line:

Hm, these imports look exactly the same as those we used in the previous example. It seems that CodeWhisperer tried to suggest us the best packages for our job — and succeeded.

Now, we want to implement the actual business logic in our Lambda function. First, we need to download the file from the specified URL. Let us start with such comment in the class:

// Function to download a file from the specified URL

CodeWhisperer suggests a couple of possible implementations. Some of them are not really useful:

Others don’t even compile:

But finally there is a suggestion that looks interesting:

Noticed the file name? It might be pretty obvious what was the source of the training data for this particular suggestion. With a bit of tweaking, we can use this code for our business logic.

We also noticed that the comments in the suggested code don’t really add up on value. Every software engineer knows: comments should answer the question “Why is this?”, not “What is this?”.

The final step is to upload the file to an S3 bucket. Let’s try our luck.

// Function to upload a file to an existing S3 bucket

And here is the best guess by CodeWhisperer:

Again, with some fine-tuning, we can use this code in our Lambda function. We wired-up both helper methods in the handler function, and we’re done.

GitHub Copilot

Starting from scratch and inviting Copilot to be our pair-programmer. We don’t really want both tools to compete, because it might be not fair in our case. GitHub states that Java is not the language Copilot was optimized for. Furthermore, Amazon might have more training data for the AWS API we want to use. But still, it is interesting to see what Copilot will suggest us.

With the same imports and intention comments, we get the following suggestion from Copilot for our Lambda function:

Impressive as well! Worth noting that this suggestion appeared at-once, not line by line. On-spot! Yes, the code is pretty basic, but again — we have a working template for a Lambda handler just in one second.

But what if we completely remove the context and only keep the comment, as we did with CodeWhisperer? Can Copilot understand our intent? Well, this time it doesn’t really work out. There are no alternative suggestions and no imports guesses. The Context type is not imported as well. Probably we need to provide more context here.

Implementing the file download function now.

// Function to download a file from the specified URL

Copilot provides two suggestions. One of those proposes a buffered stream reading, but implemented manually:

The second one treats the input as a text file and downloads the file into memory, returning a String containing the file data. But what if the file is binary? And what if the file is larger than our Lambda function memory size? Attention to details is needed when deciding which suggestion to accept.

Again, the last step is to upload the file to an S3 bucket. Without providing any additional context, we ask Copilot to suggest an implementation.

The first suggestion uses a deprecated constructor but is fairly acceptable:

The second suggestion proposes to use a third party utility for uploading the file, but it is not obvious where to find it — there are no hints about packages or namespaces.

One more suggestion — even using multi-part upload and some really comprehensive logic. Unfortunately, this code won’t compile. A possible reason for that might be that the suggestion is based on an old version of the API which changed in the meanwhile. In this code, there is an attempt to instantiate an interface which is indeed not possible. Furthermore, a couple of methods are used which are missing in the current version of the AWS S3 SDK for Java.

With a bit of SDK documentation reading, we managed to fine-tune the multi-part upload code so that it works with the current SDK version. Wiring up both methods in the Lambda function handler — and done!

Conclusion

After trying both tools and looking at the code we got, let us be honest with ourselves and answer these important questions:

• Can we use the generated code as-is?
• Is the code production-ready?
• Is the code clean, safe, readable, and fast?
• Does the code follow the best practices of the programming language?

Depending on your level of experience, you might answer these questions differently.

We came to a conclusion that the AI-driven coding companions are good enough in giving you an idea of how to implement some functionality. You definitely can spend less time searching across the Web for good code samples — you basically get those code samples directly in the source file you’re working with, and even optimized for your current context. You can implement comprehensive logic faster, especially if you are new to an API or a library.

However, the coding companions — at least at this time — are still not capable to replace an experienced software engineer who knows the best practices, the Do’s and Don’ts, the pitfalls, and who can think out of the box. The AI tools do not ask clarifying questions, they do not know the Big Picture of the project you work on (and thus are missing a lot of context), they do not suggest edits to existing code, and they learn from code whose quality might be questionable.

Still, it is a great opportunity to try the cutting edge technology right now and let the computers help us write programs for the computers. Just don’t rely blindly on the suggestions you get and make sure you consider them with a grain of salt, which is indeed also the case when using code samples from the Internet. With more and more training data, especially based on high quality code, the AI-driven code companions might become efficient pair-programmers. Looking forward to that!