Automation-First Approach Using the Karate API Testing Framework
Adopting an automation-first mindset is the first step to reduce manual and repetitive work. Thinking this way enables us to move faster, and more efficiently. It unburdens us from mundane, repetitive work, allowing us to focus on solving problems and creating value in the Software Development Life Cycle.
So the first thing is to look for a tool that helps us write automated tests faster and is easy to maintain.
Karate is an open-source API test automation tool. API tests are written using Behaviour Driven Development (BDD) Gherkin syntax. Unlike other BDD frameworks like Cucumber, Specflow or JBehave, Karate has all the step definitions written for us so we don’t have to worry about writing them. This enables even non-developers to easily write API tests for the services. Let’s check some of the main features of the framework and how it makes API test development easier.
Setting up a Karate project
Assuming you have Java and Maven installed on your machine, setting up Karate is very easy with the Karate Maven archetype plugin.
- Open a command-line interface (CLI)
- Enter the following command :
mvn archetype:generate "-DarchetypeGroupId=com.intuit.karate" "-DarchetypeArtifactId=karate-archetype" "-DarchetypeVersion=<version>" "-DgroupId=karatetests" "-DartifactId=karatetests" "-Dpackage=karatetests" "-Dversion=1.0.0-SNAPSHOT" "-DinteractiveMode=false"
- A project folder will be generated with name
karatetests
. - Launch IntelliJ or Eclipse (or any other IDE)
- Go to
File > Open > karatetests
If you have an existing Java Maven Project, you could add the following dependencies to the project:
Create Some Api Tests
- Under
src/test/java
, create a feature file calledtest.feature
- Enter the following:
Feature: Demo Karate Tests Scenario: Verify that test server is up and running Given URL http://localhost:8000/api/json?pretty=true When method get Then status 200 And match response.id == 2
- Execute test
The test should be successful!
Here, as you can see, you don’t even need to write any Java code as Karate sits on top of cucumber and it inherits all the cucumber features. You can create your API tests in BDD syntax without the overhead of writing any code to implement your scenarios.
What’s so Cool about Karate?
No compilation and programming knowledge needed!
Karate provides its own DSL (Domain Specific Language), which uses a Gherkin-like language enabling one to write tests without programming knowledge and write tests in .feature
files. Here is slightly more advanced one:
Feature: sample karate test script Background: * url 'https://...' Scenario: get all customers and then get the first customer by id Given path 'customers' When method get Then status 200 * def first = response[0] Given path 'customers', first.name When method get Then status 200
The Given/When/Then format for scenarios makes it easy to read and understand. Also, one does not need to know much about the internal Java implementation; making it easier to write tests.
Using Custom User-defined Functions
Most of the code is written in Karate. But there are cases where you need to perform custom actions like generating test data like a random string, a date, numbers, or tokens. In these cases, Karate offers flexibility to write custom functions using Java or JavaScript and one can directly call them in the feature files.
Calling any Java code is easy. Given a custom, user-defined Java class:
This is how it can be called from a test-script:
Feature: Availability Request Background: * configure lowerCaseResponseHeaders = true * def testData = Java.type('com.trivago.eb.util.TestDataGenerator') Scenario: I perform successfully an availability request to a customer endpoint * def startDate = karate.get('startDate') != undefined ? startDate : testData.futureDate(14, 'yyyy-MM-dd') * def endDate = karate.get('endDate') != undefined ? endDate : testData.futureDate(15, 'yyyy-MM-dd') Given url endpoint And path '/customer_availibility' And form field start_date = startDate And form field end_date = endDate When method post Then status 200
Better Support for Debugging Tests
As Karate does not require an IDE to write tests, and the tests are written in .feature
files, debugging test failures can be tricky; but recently there is support for this using the VS Code Karate plugin.
GraphQL Testing Support
The framework provides amazing GraphQL testing support, making it easy to perform requests and verify complex and dynamic JSON responses. Let’s see an example…
Feature: Test GraphQL endpoint Background: * url 'https://test.server/graphql' Scenario: Simple GraphQL request Given text query = """ { customer(id: 101) { id name } } """ And request { query: '#(query)' } When method post Then status 200 * match $.data.advertiser.id == 100 * match $.data.advertiser.name == 'Automator' * match $.data.advertiser.supplierId == 100
In line 4, we are setting up the URL to our GraphQL server. Then we are calling the GraphQL query in the Given
statement. The following line shows how to build a request with an And
statement and finally how the JSON is validated in the * match
statement.
Apart from these, there are many other cool features Karate provides. You can feed data from a JSON array and perform Dynamic Data-Driven testing at runtime. Matching the full payload in a single line, as well as updating the JSON payload is very easy. As mentioned above, Karate is easy to set up using Maven. It’s therefore also very easy to integrate into your CI pipeline. One can run tests regularly and receive fast feedback on newly added check-ins. You may have a look at all the features over here.
What’s Next?
We have been writing automated tests using Karate for quite some time now and are happy with the results. We will focus on improving our automation coverage next and find more areas where we can reduce the effort of manual testing. Building and maintaining automated tests is a lot of work. To get a good return on investment, we will be using these measures to evaluate our testing:
- Finding real problems: Your Automated tests should find enough real problems.
- Avoiding false alarms: Many times a test will show a failure because of an intended change, and you will have to go back and modify the test.
So if your team is looking for a testing tool for your project to write API tests faster and without much programming knowledge, Karate might be worth a try.
Originally published at https://tech.trivago.com on November 14, 2019.