Integrate API Testing into CI
Integrate Postman into Travis
Postman provides a CLI, called Newman to let testers perform API testing out of Postman. For example, the following command is to execute the given test by the given environment setting.
$ newman run test.postman_collection.json -e env.postman_environment.json
Test and environment files both are exported from Postman. Although there is no command to initialize the test which means Postman is indispensable, the Newman CLI still makes API testing in CI be more straightforward.
Here I use Laravel as the API development framework and Travis as the CI tool. A few steps can integrate them.
- Place the test file exported from Postman under code base.
$ tree tests/postman
2. Install Newman in CI environment.
- npm install newman
3. Start the server to make APIs can be fetched (This can be done easily in Laravel since it provides a built-in server).
- php artisan serve &
4. Executes the Newman command to run the test.
script: — newman run tests/postman/GraphQL.postman_collection.json
Not Supporting Code Coverage
Now we can see the tests’ statuses. It would be better if the code coverage report be generated as well after executing Newman. It can help developers to know about that whether any APIs are not tested yet. Newman provides an option to specify the report type. However, it just customs the style or format of the output.
Above shows the report in the format of phpunit (
-r phpunit) which uses one dot to represent one test step. We can see that no any information about coverage is provided. Although there is no such integration currently, Newman is still a good tool and can makes codes be reliable.
More details are in the github repository.