Let’s try TestGPT the test cases generator: codium AI

We have seen ChatGPT and other Generative AI tools which are generating code for the logic we requested and also the test cases for those.

Codium AI is an AI-powered platform that helps development teams automate code reviews and identify potential issues in their code and also writes test cases for the code that AI understands. Let’s explore whether this tool is genuinely helpful.

To begin, let’s consider a straightforward program that calculates the sum of all numbers in an array.

export const sum = (arr: number[]) => {
  return arr.reduce((a, b) => a + b, 0);
};

I already have Codium already installed as a VSCode extension which you can download from the link provided. Link to download.

Codium can provide suggestions or test cases for every function it understands, and it will display a string that says “Generate tests for the below function”. See the figure below:

Let’s click on that and see what it shows. The analysis is shown in three parts Test Suite, Code analysis and Code Suggestions.

Let’s first see the code analysis. If we read it we can understand that it is trying to explain what the function objective is, what is the input for that function and how the function actually works. This pretty much looks good. Let’s move on to the next one.

Here is the snapshot of the code suggestions provided by Codium. The first suggestion is to add type checking of the input array of numbers, which is an excellent suggestion. Let’s move to the next one.

The second suggestion says to use for loop instead of reduce, which can be faster for large arrays. This is also a good suggestion, and we can make the change if we anticipate processing large arrays.

The third suggestion is to use BigInt for large numbers. If we are working with large numbers, this is also a valid suggestion, and it is good that Codium brought it to our attention.

Let’s now move on to the Test Suite. Codium provided a total of six test cases, including five happy paths and one edge case. This is sufficient for this function, and the Test Suite appears to be well-designed. It is worth everything that this was achieved with no input or suggestions from me.

For one of the test cases, I suggested making the array slightly longer and adding decimal values to it. This was a helpful suggestion, and it demonstrates how Codium can be customized to suit the specific needs.

It is not surprising that Codium was able to deliver exactly what we requested. May be we can even specify the exact length of the array, and Codium can generate output based on that specification. I encourage you to give it a try and see for yourself.

We can provide custom configurations to Codium and re-generate the tests according to our specifications. For example, we can specify the test case style to be ‘Given When Then’ or ‘Arrange Act Assert,’ and we can reference existing tests that we wrote manually. Additionally, we can provide sample data inputs and indicate whether the tool should auto-mock or not. We can also specify how many test cases to generate. This is a powerful feature because Codium can understand the way we style our test cases and recommend the same pattern, so we do not have to make any modifications.

I copied all the tests that were generated above and ran the test command to see if they work. Guess what? They work pretty fine. All 6 test cases passed. Kudos to Codium 👏.

In my opinion, this is good but make sure you do a review on the tests written and then consider them adding to your code. Do not blindly add them as they might sometime write something which could not be valid.

I would like to take a complex example in the next post and we can see how this AI work.

Thanks for reading this far.

Happy Learning !! 😃