How shift left testing can save you bugs, time, and money
In the world of software development, bugs are completely normal and are part of our daily lives. All engineering teams strive to have as few errors as possible because they have a negative impact on users, and fixing them costs the company time and money.
Preventing bugs is a complicated task, but we can reduce the number of errors in our code by following best practices such as writing tickets with very detailed descriptions that cover all possible use cases, using clean code when developing features, or writing robust tests, among other things.
Whose responsibility is it to ensure there are no bugs?
We could say that the responsibility lies solely with QA (Quality Assurance) engineers (if there are any), but besides being false, it’s unfair because we would be delegating all the responsibility to them.
In some teams, there is the perception of a QA engineer as a role that ensures no ticket introduces bugs into production. I’ve heard developers say, “I write the code and then the QA tests it.” This shouldn’t be the case.
The quality of the code is a shared responsibility among the entire team.
When should I start thinking about the bugs that the code might generate?
Most bugs are introduced during the development phase, and this happens for different reasons: what if the developer has started working on a ticket with an incorrect task description? What if the acceptance criteria are vague and ambiguous or not even defined?
We must think about bugs from the very moment we create and refine a task. Thinking about them during development is too late.
Let’s suppose we have the following software development lifecycle:
- Requirements
- Design
- Implementation
- Review and QA
- Deploy to production
If you notice, QA is in the last phase before reaching production, which means that QA feedback comes very late, and the developer will take that feedback into account in the last phase before deploying to production. This can lead to an endless conversation between the developer and QA when reviewing the functionality, and a single task can have multiple back-and-forth exchanges (like a ping-pong match) simply because QA wasn’t involved even before starting the development.
Fixing a bug incurs a cost that significantly grows as it progresses through the development stages. It’s much easier and cheaper to fix a bug in the earlier stages of development than once it’s deployed in production.
Therefore, not using shift left testing has several implications:
- By not focusing on testing from the first stage, implementation time increases
- With increased implementation time, feedback on the task arrives very late
- If feedback arrives late, the effort and cost of changing the code are higher
So, what is shift left testing?
Shift left testing means focusing on testing from the very beginning when we start thinking about how to approach a task. We must consider all use cases, both happy paths and corner cases, to ensure that task acceptance criteria are as detailed as possible.
Benefits of shift left testing
- Early bug detection: by moving testing activities earlier in the development process, bugs and issues can be identified and addressed sooner, reducing the cost and effort required for fixing them later in the development lifecycle.
- Improved product quality: with testing integrated earlier in the software development cycle, the quality of the software improves as issues are identified and resolved in earlier stages.
- Faster feedback loop: developers receive quicker feedback on their code, allowing them to make necessary adjustments promptly, leading to faster iteration and development cycles.
- Cost reduction and faster time-to-market: identifying and fixing issues earlier in the development process reduces the overall cost of software development by avoiding expensive rework and late-stage bug fixing.
- Increased collaboration: shift left testing encourages collaboration between developers, testers, and other stakeholders from the outset of the project, fostering a culture of shared responsibility and collaboration.
- Customer satisfaction: by delivering higher quality software with fewer defects, shift left testing ultimately leads to increased customer satisfaction.
How to implement shift left testing?
Include testers in the earliest stage possible
Including testers in the requirements gathering phase is crucial for successfully implementing shift left, as they provide a detailed perspective on how developments should be tested even before developers start writing code.
If developers have very detailed acceptance criteria and know exactly how the code should behave and what will be tested, the number of bugs and rework will be drastically reduced.
However, this also implies an evolution of the tester’s role, typically focused on capturing bugs once development has concluded; to the role of QA itself, oriented towards preventing bugs throughout the entire SDLC. This transition is not always easy, and QAs may need some mentorship when adapting to these new functions in their daily routine.
Add very detailed acceptance criteria to your tasks
Acceptance criteria are a contract that, if we have done a good job of researching the problem beforehand, should not change during development.
Making changes to acceptance criteria after development is very expensive, as it forces the programmer to rewrite already completed code due to incorrect requirements.
Make sure all the team understands the acceptance criteria
Developers must test their code before sending it to the review stage
Multiple bugs and conversations between developers and QA can be avoided by testing the code correctly. Unit testing is an extremely powerful tool that can be used to reduce review time and the number of bugs in the code. Manual testing is expensive and should be avoided.
Continuous feedback
Developers should work with QA and the Product Manager (if any) during the implementation stage so they can provide feedback to the developer as soon as possible. This way, we reduce the amount of rework and the number of late-stage bugs.
Real results
At JOOR, we have several teams within the engineering team, and I would like to show you some numbers on how shift left testing can save time, bugs, and money.
I won’t be showing the numbers for all teams and all quarters to simplify, but the rest of the teams have had similar results.
Metrics
- Cycle time: how long does a story take to be completed after being started
- Dev time: how long does a story remain in progress
- Review time: how long does a story take to be moved from in-review (code review and QA) to merged/deployed
Shape of work
Takeaways
In Q2, we didn’t work with shift left. The percentage of bugs we fixed was 10.3%. As you can see, the review time was over two days.
In Q3, we started using shift left testing, but you know how these things go. Establishing a new methodology in a team is not easy and takes time for everyone to get used to working with it and start seeing the benefits.
Despite this, we managed to reduce the cycle time by 23 hours and also the review time by 6 hours.
In the last quarter, Q4, we reduced the cycle time to 2 days and 11 hours, which is a reduction of 1 day and a half compared to 6 months ago. The review time was reduced to 14 hours, which is very low compared to the initial 2 days and 5 hours.
In addition to that, we were able to complete much more work while reducing the percentage of bugs.
As you can see, it doesn’t take more time to program using shift left because the development time is maintained, but the review time is greatly reduced, so we get more work done in less time and also without adding more bugs.
