Achieving Continuous Testing in the Continuous Delivery Pipeline for Web-apps
“Continuous Testing” is the buzzword at the moment in the QA/Test Automation world. Let’s have a look at how this can be achieved in an efficient way.
What is Continous Testing?
Continuous testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate feedback on the business risks associated with a software release candidate. [ref: wikipedia]
Following Test pyramid explains an efficient strategy to achieve Continuous Testing:
As shown in the diagram above, code coverage via Unit Tests plays a major role on this. Maximizing coverage for your front-end/backend services via unit tests will give confidence at the foundation level itself.
Next step is to set-up CI pipelines, which will be triggered every time developer commits code to source control repository to enable code compliance with the quality guidelines set and also ensures immediate feedback to Developers.
Another advantage of CI Integration is as follows: once the developer creates a Pull Request(PR) in the Source control System, the reviewers of the PR can easily get information about the CI Status on the PR itself, which will reduce the feedback time required to approve the PR.
In the next stage of Continuous Testing model, we need to write integration tests and API end point tests[RestAssured is the commonly used library for validating api response, schema and other tests]. For web frontend, adding integration level cypress tests, which can mock specific pages and validate particular functionality, will also help to improve the coverage.
For UI Automation, considering its complexity and maintainability, try to pick specific set of test cases from your Master Test Cases, and automate it using Stable frameworks. To ensure Browser and OS compatibility these tests can be integrated with any cloud testing platforms and execute it in your CD Pipeline.
Adding Performance Tests scripts [Jmeter scripts will do the job] and Security Tests [OWASP ZAP is a good open source tool to use], will make Continuous Testing more powerful in the Continuous Delivery Pipeline.
Sample Implementation of such pipeline will be as shown below:
10 Key Points to consider while implementing UI/Functional Test Automation Frameworks:
- Use stable automation frameworks
- Use POM model to organize your tests
- Try to use robust element locators which are less likely to change and are fast.
- Organize your Test Data.
- Use BDD frameworks, which can help to easily convert your user stories/test scenarios into Automation Test Scripts and provides many other benefits.
- Lets not duplicate same tests as part of different test suites.
- Avoid long running tests and split it into separate tests for reliability
- CI/CD Support and other key features like parallel execution, failed tests auto-retry, reporting tests results etc.
- Implement traceability between your Manual Test Cases and Automated Test Cases. [Ex: Test_case_id can be used for mapping]
- Cloud Testing platform support is a major factor and needs to be considered in the starting of the “Test Automation framework” design itself.
