AI for Engineering Teams. AI for API testing
Our AI journey goes on, and over the past few months, we’ve been experimenting with using AI for API testing and have built an AI testing tool that provides notable time savings for our SDET and SDE engineers.
For a deeper dive into AI-powered API testing tool and its impact, check 6x Faster API Testing with AI-Powered API Testing Tool publication. Below, I’ve just described the core concepts of how it leverages AI to generate the tests.
It is also important to note that you can use a similar approach for test generation directly from ChatGPT or Copilot Chat. We’ve just found that wrapping all AI interaction steps, AI prompts, and other automation into a tool is much more convenient and easy to use for our engineers.
As a result, our data revealed that with this tool a mid-level SDET engineer could accomplish tasks six times faster than without it.
Step 1. Using AI to generate test scenarios
The Swagger (or OpenAPI Specification) is a standard for describing and documenting RESTful APIs, making it easier for developers and testers to understand and interact with API endpoints.
We decided to see if AI can help us formulate test scenarios based on Swagger API method definitions. For instance, consider the task of creating tests for the official Swagger Demo API endpoint.
Below is a sample prompt that can be crafted manually or using automation to ask AI to help you with this task to develop ideas for test scenarios to test the API method that adds a new pet to a store (addPet method).
The provided prompt includes Swagger API definitions and AI instructions. As a result, AI produced a very comprehensive list of test scenarios to cover API functionality — a good combination of positive, negative, and edge cases.
Step 2. Using AI to write the tests
With our test scenarios, we could finally harness AI to create the final automated tests. These tests can then be executed programmatically.
As a result, you will get the tests generated for you.
Final words
The prompts above are very basic and are designed primarily for demonstration or educational purposes. They don’t cover many important best practices and aspects when writing API auto-tests.
For the tool, we used the foundation above to come up with more sophisticated and advanced queries, providing production-grade quality and the ability to adapt to varying coding styles, techniques, and other nuances inherent to different projects
As for the business impact, according to our calculations, the additional cost per API endpoint averages $0.064, amounting to $3.2 for 50 endpoints. When weighed against the expenses of SDET labor, AI/LLM emerges as a highly cost-efficient companion. Consequently, businesses can benefit substantially by incorporating AI into their API testing processes.
