Shift left testing: A GWI success story
Shift-left testing speeds up software development, helps teams identify bugs earlier in the delivery process, and improves quality throughout the development lifecycle. But there is a lot of confusion surrounding the term, how it is defined and it requires the adoption of new development practices to work perfectly.
In this post, we’ll explain what shift-left testing is in simple terms. We’ll also explain the common blockers that stop shift-left testing from succeeding, and how we in GWI are defining shift left testing strategy and how test automation helped us put it into practise.
What is shift-left testing?
Before explaining the term lets review the traditional way of delivering software which is waterfall methodology.
For years this was the best approach before delivering our product. Each step is the outcome of the previous and the process is synchronous. The problem with this approach is that defects are detected rather late in the process making the quality a time consuming task for our whole delivery. Repetitive tasks means slower delivery and not with 💯 success in terms of zero regression policy.
With a shift-left approach, testing takes place earlier in the development lifecycle, and at all stages (when there is an opportunity to do so). This makes sure that quality is considered from an early point in the development process, reducing costs and risk.
In short the goal is to test earlier and with repetitive manner to prevent defects from coming out…
Why shift-left testing can fail
Testing is not enough
While shifting left can help us identify major and minor problems earlier on, it does not remove the need to regression test your system with every release!
You still need to test often to ensure that you’re catching bugs that may have slipped through the cracks.
Is not part of development process
A business can’t test earlier unless the SDLC is adapted to accommodate earlier testing. If you want to carry out functional UI testing earlier, you also need to develop and set the requirements of the UI earlier!
To do this, you have to change the development workflow so that it supports earlier testing, like Agile and DevOps.
Slow manual testing process
The larger the system, the more testing is required to make sure your are releasing safely.
Every update, release, customisation, and integration poses a new threat to the overall quality of the system. And the demand for faster, higher-quality software development means that manual testing doesn’t fit the bill.
Customers are expecting high-quality software with new features and functionality, but businesses cannot keep pace.
Testing Pyramid
Testing pyramid is just a framework that defines the level of testing in early stage and facilitate engineering teams to use it appropriately. It make sense to define the type of tests performed in early stage and their aspect.
A typical pyramid shows the above levels that we need to invest. The more you go up in the ladder the most effort you need to pay to maintain and at the same time the time to get feedback is greater. That does not mean we should not have for example functional UI testing but we should depend less on them than integration or unit testing.
Let’s review the pyramid we decided to architecture in a microfrontend based application.
You will see a blend of functional and non functional testing that applied together as early as possible in development process. We provide also the automation tools we used for each level of testing.
Once the decision has been made to move to shift-left testing, it is normal to experience challenges. While it means your capacity for testing more and earlier is within reach, the wrong tooling can wreak more havoc than bring good.
RTL — Maintainable tests for React Components
MSW — Mocking Service Worker Mock by intercepting requests on the network level. Seamlessly reuse the same mock definition for testing, development, and debugging.
Cypress — Maintainable tests for Web
Axe — Accessibility testing engine for websites and other HTML-based user interfaces
k6 — Load testing tool
Sentry — Error tracking and monitoring
Snyk — Static code analysis platform for security
OWASP ZAP— Security scanner for DAST
Stryker — Test your tests aka mutation testing
But these tools and frameworks are just a part of the overall testing strategy. How did we decide which tool to use? Let’s review some simple steps below:
Step 1: Understand your project requirements thoroughly
Maintaining the quality of an application by delivering a bug-free product is crucial for the success of any project. Automated testing can help improve the quality of a project and increase scope & depth of the tests. For this, get a deep understanding of your project requirements such as project type, scope of the project, and existing team’s strength on code language before you start the process. There is no such tool that is good or bad but the ROI of any tool relies on the need, i.e. what precisely does one need to automate and what amount of test cases needs automation?
Step 2: Consider your existing test automation tool as a benchmark
Our QA team considered Cypress as a benchmark to evaluate and determine the best automation tool for their project for the other layers of testing pyramid.
Step 3: Identify the key criteria suitable for a project
There are many key points one should consider before deciding the best-fit automation tool for a project. Here is the list of few key criteria that was used to evaluate the best framework in our case.
- Ease of Development and Maintenance : Development & maintenance of test scripts should be as simple as possible to decrease the human and time resource utilisation.
- Ease of Test Execution for Non-Technical user: The test suite execution should be simple for any project member to run them easily as and when required. Also, it should be easy for QA testers who have very little or no technical knowledge.
- Intuitive Test Report: Test reports build confidence and so the reports need to be intuitive and simple for the management team to understand easily and the engineering team to use.
- Cross Browser Testing: Support to Cross browser testing is a must when there are multiple end-users and no particular browser restriction.
- Support to Keyword & Data Driven Testing: Keyword driven testing acts as an extension to the data driven testing framework. When a project becomes complex, test framework needs to be extended.
- Performance and Feedback Loops: Performance for each tool is important as we need fast results and of course reliable.
- Strong Community and Support: The stronger the development community is the better support you will have.
The above are some of the steps we need to consider when adding automated solutions in our pipeline for the best possible quality outcomes.
Conclusion
In the end the tools and the way you will choose to implement the framework for shift-left testing is entirely up to you to decide. We just provided some insights for better outcome in the process and how we in GWI defined an ideal shift-left testing approach.
Testing left is the future. In a nutshell, the traditional shift-left testing process is about earlier testing and using test automation. It creates better and more efficient results and higher-quality software. Finding defects early in the process will reduce the project’s cost, especially if it relies on test automation.
The ultimate focus is customer-centric. It’s all about taking care of customer requirements and improving the customer experience. The shift-left testing approach brought massive transformation to the industry and the QA’s role.
