Introduction to API Testing Checkpoints
Checkpoints in automation tests are used to validate whether our test cases succeeded or failed. Some tools might define it as “assertion”, but the goal is the same, to compare the expected outcome with the actual outcome.
In our modern world, API (Application Programming Interface) has been used by many tech-companies to build their system architecture. API, in basic terms, is defined as a set of methods that enables transmission between various software components. There are many types of APIs, but in this post we will discuss more on REST API and gRPC service.
What is REST?
One of the most popular web-based approach in API implementation is using REST (Representational State Transfer). Client-side sends a request and the server-side answers with a response. The server-side data are represented in simple formats, such as JSON and XML. JSON is a simple data interchange format. At its heart, JSON follows a relatively simple data structure, therefore we can represent all kinds of structured data in myriad ways.
JSON Data #1
{
"name": "Jon Snow",
"address": "Winterfell, The North, Westeros",
"occupation": "Lord Commander of the Night's Watch"
}JSON Data #2
{
"first_name": "Jon",
"last_name": "Snow",
"address": {
"street_address": "100 North Winterfell St.",
"city": "Winterfell",
"state": "The North",
"country": "Westeros"
}
}
What is gRPC?
gRPC is a RPC (Remote Procedure Call) platform developed by Google. Client-side can directly call a method on the server-side as if it was a local object. By default, gRPC utilizes Protocol Buffers (Protobuf) for data serialization. Through Protobuf, gRPC produces a simpler and compact response for the corresponding client to process. The data format are converted automatically from their Protobuf representation into our application’s programming language of choice. It also provides a schema for the data to be transferred. Due to this reason, our internal services might communicate easier since they already know the expected values. Below is an example of Protobuf schema definition of “Person” data type.
message Person {
required int32 id = 1;
required string name = 2;
optional string email = 3;
}How it’s done here
Working at DANA, I have had the chance to test projects that use RPC calls (including gRPC service) and REST API. Both test frameworks are based on TestNG. Aside from their pros and cons, any testing process would not be complete without defining an assertion process. When we run our test scenarios, how do we decide if the application has given a presentable outcome? How do we validate whether the API’s output is already expected? This is the importance of defining checkpoints/assertions in our testing process. Checkpoints are defined in the final step of our test script development. It is used to determine whether our test case is successful or failed. We can also examine further why it fails, e.g: is it a bug in the API, or is it expected and we need to change the checkpoint? Without defining it, our automation test would not have a clear goal.
Working with a more complex system, to verify if the API response has returned a success state is not enough. There are more data that we can examine from the output. Below are a view methods that I have used in my testing process.
JSON-Schema Assertion
In every data transaction between our systems, it is important to enforce a structural component to ensure its consistency. We already saw that gRPC service handles this scenario, therefore it is also important to define one for our JSON payload as well. This is where JSON-schema assertion comes in.
JSON-schema is the contract for our response payload that defines the expected data types and format of each field.
{"type":"object","properties":{"first_name":{"type":"string"},"last_name":{"type":"string"},"address":{"type":"object","properties":{"street_address":{"type":"string"},"city":{"type":"string"},"state":{"type":"string"},"country":{"type":"string"}},"required":["street_address","city","state","country"]}},"required":["first_name","last_name","address"]}If we validate JSON Data #1 using this JSON schema, it will fail since it does not have the required properties.
Database Assertion
Oftentimes, API responses only return “OK” with status code 200, or other results that are not clear enough.
{
"status": "OK",
"place_id": "123456abcdef"
}How do we extract a more valuable checkpoint out of it? Can we utilize the information further? This is another assertion point that is included in my testing process: Database assertion.
Let us look at an example Map API collection which contains the above APIs. The APIs are able to do simple CRUD (Create, Retrieve, Update, Delete) operations. Whenever an action is performed by the client, such as adding a new place via AddPlace API, the corresponding Create operation must be invoked on the backend and the place must be stored in database.
To validate the stored data, we can utilize CSV reader to put our expected values and JDBC to connect and retrieve the actual values from our database. We can use place_id from the response as the search key into our database.
With the example above, our test case would pass since the expected result is equals to the actual result. If, for some reason, our test case failed in the assertion process, then we would need to check whether our expected values are incorrect or our API is not working as it is supposed to.
To sum up,
Test assertion is an essential part of our test case development. There are more data that we can observe and verify aside from the API’s response. Asserting the JSON-schema affirms a clear data structure for our REST API response payload to follow. We can also do database assertion by comparing the expected data using CSV Reader with the actual result in the related database.
