Karate Framework vs Rest Assured
If you want to have a stable application you need to spend a notable amount of time on testing and making sure that every business feature meets the defined requirements. Sometimes the whole testing process is one of the time-consuming parts of a software delivery cycle. It is also known that automation reduces time spent on repeated actions and eventually helps to reduce LTTC (lead time to change). That’s why we always should strive to use automation for such kinds of processes. Automation tools allow us to save time, provide means to reduce the time spent on repetitive manual tasks, shift quality left, and much more.
Do we really need API testing?
Since the applications are moved to the cloud, the importance of the APIs has been critical. API testing is a must with the enhancement of the microservice architecture and being mediator platforms. Many features of the applications that we use daily rely on hundreds of different synergistic APIs on the microservice architectures, if one of the services fails then we will have limited operational capacity of the related service.
What are the 4 most important criteria for finding the optimum API testing tools?
- Ease-of-use and Capabilities
- Ability to optimize your workflows
- Automation
- Management & Maintenance
The ideal API testing tool needs to be easy to use for a beginner user to pick up and start using with minimal training. It should include capabilities according to your needs. For example, if you are going to execute data-driven tests, the framework should have support for it.
Among many other qualities, your API testing framework must provide the ability to optimize your business flow. It should comprise intelligent mechanisms that simplify things you do daily. Think of test creation, monitoring, working with test data, and integrating your test solution into your requirements of the business.
It’s valuable for your API testing tool to provide a mechanism to run your tests in automation. As we can imagine, manually testing APIs is not realistic, reliable, or simple. There are usually several endpoints in the microservice architectures and it is important to automate them. Thus we can programmatically trigger our test suites as a part of the CI / CD process.
In this article, I will compare two of the most popular API test automation tools, Karate Framework and Rest Assured, based on my experience.
Karate was created by Intuit in April of 2017 and it gained popularity pretty fast. Developed and maintained by Peter Thomas who created a new company called KarateLabs and moved the Karate framework support to the new company. Thanks to him test automation was made simple. It currently gets about 5.6k stars on GitHub. That’s quite impressive for a tool that is around for 5 years. The latest released version is 1.1.0.
Karate Automation Testing Framework was based on Cucumber JVM but as of version 0.9.0, not anymore(see why) but retains the syntax.
Karate is the only open-source tool to combine API test-automation, mocks, performance-testing and even UI automation into a single, unified framework. The BDD syntax popularized by Cucumber is language-neutral, and easy for non-programmers. Assertions and HTML reports are built-in, and you can run tests in parallel to achieve high speed of execution.
There are several articles and videos about how to create a project in Karate or how to implement DSL (Domain Specific Language) methods in Karate so you can easily learn how to do it. Here, I want to talk about how easy it is to develop tests in this framework and what you should be aware of. If you have already read my last article about this framework and you have a large number of tests, you should be careful in choosing this framework. It can enchant you easily with powerful skills but when it comes to performance, you should check if the memory error appears in the latest version. I have been having a memory problem with the karate framework in versions before v1.1.0.RC1.
So let’s dig a little deeper
A karate code looks like the one below;
Tests are written in Gherkin format which is also used for BDD, but Karate is not true BDD. There are several API request methods which are PUT, POST, DELETE, GET. In another language or framework, you need to declare those methods with a library or you need to develop the methods, while in karate making a request is very simple. Just write the type of request that you need after the “method” keyword.
It is a keyword-driven framework so you don’t need to declare those methods which allow you to only focus on writing your tests. There are several benefits of this like:
Data-Driven tests are so easy:
call vs. callonce
If you use the call keyword in Background, your call method will be compiled for each scenario in that feature file. If you use the callonce keyword, your method is going to be compiled once and cached for each scenario instead of executing that method (or feature) again and again. Let’s have a look at it;
I have observed that callonce is making the test a bit slower because it caches the response of the request and reads from the cache for each scenario so if your response is large, it is going to make your tests slower. Depending on your scenario, you may prefer making that request every time for each case with the call keyword instead of caching with callonce.
REST Assured is developed and maintained by Johan Haleby. Johan started the project when he was working at Jayway back in December of 2010. The project is now sponsored by Parkster. The latest version is 4.5.0.
If you only need to check requests and responses, this library will simplify things. In REST Assured, you don’t have to create boilerplate code to test and validate complex REST responses. You can also use modeling your request and responses since Rest Assured framework is based on java.
Rest Assured framework has a BDD or Gherkin support that helps to enhance the readability of the code and clean coding. REST Assured assists in easy integration of classic testing frameworks such as JUnit and TestNG.
A rest-assured code looks like the one below:
Data-driven tests are easy but you need to implement dataProvider from TestNG or equivalent library:
Comparing Test Execution Times
I have observed that Rest Assured is a bit slower than Karate when it comes to executing tests. It can cost you time when you have large amounts of test packages. I have made a small example to show this and used the same libraries in both project and framework’s libraries have almost the same lines of code. Rest assured has 46k lines of code and Karate has 50k. I used the same API to test and API was stable.
In my opinion, Karate Automation Test Framework makes more sense, especially for the teams that create and maintain web services and where acceptance testing is not the highest priority. Karate is not for the teams that are willing to perform BDD even if it looks like that.
References
In general, I share my experiences and the work that we think is cool in the Getir Core Automation Team. I am thrilled to be a part of the Getir technology team and very excited about the work we will do in the future. If you would like to be a part of this team, please do not hesitate to apply to one of our open positions at career.getir.com. 😊
Thanks for reading and feel free to reach out :)
Cheers🍻
