API testing using Postman and Newman
The beginning of the Coding Phase
GSoC coding phase is the one that beholds tons of learning and exciting work for a participant. June 1, 2020 — June 7, 2020, was the first week of this journey.
The Community Bonding period involved a lot of brainstorming over ideas, features and timeline. Surprisingly, the work I stated in my proposal is completely different from what I am doing. So the last month involved finalizing lots of things before we could proceed for the Coding Phase.
In the first week, I worked on the basic setup of frontend and backend repositories, the addition of User Models and the registration API. In this blog, I will be describing testing using Postman & Newman — something which I tried out new this week.
Have a look at the docs if you are planning to implement in your projects too:
Why API tests
API tests ensure software testing at the user-interface level, check the functionality, reliability, performance, and security of the programming interfaces. It also helps in error handling and handling edge-cases to prevent server errors. Tests also allow making requests which are not permissible through the UI. The addition of a new feature brings in the possibilities of breaking other features, which can be handled using tests — so a developer can figure out what or how the tests are failing. As obvious, it proves more effective than manual testing.
Postman and Newman
Newman is a command-line integration for testing Postman Collections. In my GSoC project, I am using Newman as a tool for testing. It’s easily integrable for CI (Continuous Integration) servers and build systems.
The reason for selecting Postman for documentation was it’s easy UI to create detailed API docs quickly. Postman contains a full-featured testing sandbox that lets you write and execute JavaScript-based tests for your API. Newman provides a wide range of options to manipulate the received response and apply assertions. Postman also allows us to test in different environments. It is easily integrable with build and CI systems. In a project which is open source, you expect people to keep contributing. Tests and builds ensure proper integration of every addition or change or bug-fix.
You may look at the backend repo to see the integration of Newman:
Implementation
Test scripts use dynamic variables to carry out test assertions on Response data. As I proceed with my project I will be adding test scripts for requests, folders and collections. All tests get executed after the response is received. The test scripts and pre-request scripts are executed in the following order:
- Collection scripts
- Folder scripts
- Request scripts
First, all the pre-request scripts will be executed before sending the request, then the request is sent, the response received and then the test scripts are executed.
Look at the docs to learn implementation of the scripts —
This is how the test scripts snippet in the Postman collection for my project looks after exporting as JSON. This file is autogenerated when you export your collection:
Running the collection for a single iteration gives such an output:
We can also run the collection multiple times. For example, this command runs the collection 10 times:
newman run <collection_name.json> -n 10This will give a summary of average response time and excitingly there is a visible variation. The output of running the same collection 10 times on my PC:
Iterative testing in any case will provide better results. It can enable us to find ways to optimize our processes or ensure testing over a variety of data or variables, thus saving time and covering maximum edge cases. This way we can identify failing examples in development.
I will be writing about my experiences every week. Trying out the implementation has been a fun process for me and I am looking forward to more extended learning during GSoC!!
