EXPEDIA GROUP TECHNOLOGY — SOFTWARE

Automation and Performance Testing Using Karate

Test APIs for correctness and performance in BDD style with minimal Java code

You can use Karate to define API functional tests without the bother of writing glue code, then reuse the same tests for performance testing.

Photo by Jason Briscoe on Unsplash

Karate is an open-source behavioural-driven development (BDD) based testing framework from Intuit. It supports Cucumber/Gherkin like technical syntax to describe the fine details of making REST calls and evaluating their output.

Let’s look at how to define API tests using Karate, then extend our approach to handle performance testing.

Defining Karate tests

Start by including the necessary dependencies in your pom.xml:

<dependencies>
   <dependency>
      <groupId>com.intuit.karate</groupId>
      <artifactId>karate-apache</artifactId>
      <version>${karate.version}</version>
      <scope>test</scope>
   </dependency>
   <dependency>
      <groupId>com.intuit.karate</groupId>
      <artifactId>karate-junit4</artifactId>
      <version>${karate.version}</version>
      <scope>test</scope>
   </dependency>
</dependencies>

Karate uses a config file named karate-config.js to configure execution environment, base URLs, timeouts, and such. This file is also where you put code that should be evaluated only once during a test execution, using the karate.callSingle method.

Sample Karate-config file

We create default authorization tokens with token-generator.js, which internally uses authorization.feature file to obtain tokens.

Now, let’s see how we can automate the users/{user_id}/info API using karate:

In this feature file, we:

• Read JSON data based on environment using the env variable.
• Configure a header using the configure headers keyword.
• Call the feature file file user-token.feature to generate userToken on the fly.
• Generate some default tokens using token-generator.js as shown in karate-config.js above. Those tokens will be generated only once, and they will be available in all the scenarios of our feature files. Anything set in Background will be available in all the scenarios.

We are using keywords like url to define base URL, path to define path variables, method to define HTTP method type, and status to define the expected HTTP status code. Upon execution, the response will be available in the response object. We are using ‘match’ keyword to do validations on the response. The above test is data driven with data provided in the Examples section. We can also use tags like @smoke, @regression, and @performance to run scenarios based on these tags.

Performance testing

Here comes the fun part. We can reuse the same feature files in a performance suite without any modification to the original feature files.

We will use Gatling (and Scala) only for defining the load model, and then reuse the feature files that we have already written for feature testing. First, we add the Gatling dependency and gatling-maven plugin.

<dependency>
    <groupId>com.intuit.karate</groupId>
    <artifactId>karate-gatling</artifactId>
    <version>${karate.version}</version>
    <scope>test</scope>
</dependency>
<plugin>
    <groupId>io.gatling</groupId>
    <artifactId>gatling-maven-plugin</artifactId>
    <version>${gatling.plugin.version}</version>
    <configuration>
        <simulationsFolder>src/test/java</simulationsFolder>
        <runMultipleSimulations>true</runMultipleSimulations>
        <includes><!--To run simulation files where load is  
                configured -->   
            <include>features.users.UserInfoSimulation</include>                       
           <include>features.employee.EmpKarateSimulation</include>
        </includes>
    </configuration>
</plugin>

Next, we write a load configuration file in Scala to provide some basic configuration like the number of users to simulate, total amount of time to run the tests, etc. You can put assertions on desired SLAs. Gatling will use this configuration file to run your Karate tests for performance testing.

Sample Load Configuration File (UserInfoSimulation.scala)

As shown above, we defined the load in a simulation class, UserInfoSimulation, for our user-info.feature file. Here, we defined the concurrent users (load) as 15 and it will execute scenarios defined in user-info.feature for 5000 seconds. Then we use SLA assertions provided by Gatling on average and 95th percentile response time.

You can use the below command to start your tests:

mvn clean test-compile gatling:test -Dkarate.env=perf

Once the tests have run, a detailed report will be generated:

References

• Sample project: https://github.com/brij001/Karate/
• https://github.com/intuit/karate/tree/master/karate-gatling
• https://gatling.io/docs/current/

Learn more about technology at Expedia Group™