QA from the trenches
Just a brief introduction about me, my name is Agustin Perez, I’ve been involving in multiple software projects for more than 11 years, some of them more complex than others. This article will be based on my experience in software quality assurance, managing testing and agile methodologies.
I have had the opportunity to participate in multiple types of projects: small, medium, and large; between 20 to 350+ engineers working together to deliver value to the customers, so that to be aligned in some concepts, let’s try to look into the past…
This article will explain how the software quality assurance(SQA), methodologies and participation were evolving from traditional to agile approaches. We will explore more details in terms of their issues, misinterpretations, disadvantages, and advantages. Moreover, I will give you some advices or tips to avoid misunderstandings regarding the involvement or contribution of quality assurance concepts and practices in the agile software development.
Waterfall
It is a linear approach of software development; this model emphasizes a sequential process/steps: requirements, analysis, design, coding, testing, and maintenance. This model was the first in the software development life cycle (SDLC) methodology to be used widely in software engineering, it is characterized to finish a stage 100% before the next one can begin.
- In waterfall, testing is a separate phase
- Development team and testing team work separately.
- No flexibility to react to internal or external changes
- Testing is carried out only after the completion of development — no variation
- Regression testing is performed only in the end.
- Testing levels cannot overlap.
In a traditional, waterfall approach more than 60% — 80% of the test project time can be spent planning the test effort and defining test cases, then at the end of the project, 20% — 40% of the effort is related to execution of the test cases. Generally, an additional 15% to 40% is required to fix defects and gaps in the product that did not pass the tests.
Advantages
- The approach is structured, design errors are captured before the software implementation.
- Works well for smaller projects where requirements are very well-defined.
- It is easier to measure progress by reference to defined milestones.
- The total cost of the project can be accurately estimated after the requirements were defined.
Disadvantages
- Requirement changes were not supported during the SDLC.
- A project can often take longer to deliver due to misunderstandings in the early phases.
- No working software is produced until the last phases during the life cycle.
- Adjusting scope during the process could cancel a project.
V-Model
V-model is a highly disciplined SDLC model in which there is a testing phase parallel to each development phase. V Model application is almost the same as the waterfall model, as both the models are of sequential type.
Advantages
- This is a highly disciplined model and phases are completed one at a time.
- Proactive defect tracking, they are found at early stage.
- Easy to manage due to the rigidity of the model. Each phase has specific deliverables and a review process.
Disadvantages
- Testing in the V model starts only after implementation is completed.
- Very rigid and least flexible.
- Poor model for long and ongoing projects
- Once an application is in the testing stage, it is difficult to go back and change a functionality
With these classical and traditional ways, software quality assurance (SQA) has failed due to the complexity and uncertainty of the projects, the participation in the SDLC as a silo affected the SQA tremendously furthermore the QC members are located into the trenches; So how can we resolve this situation?
In the new digital transformation process, the vintage approaches of testing do not seem to work as well as they used to, new elements such as continuous integration/deployment, agile methodologies, DevOps, shorter delivery times or automation push the SQA to new boundaries and open multiple areas, where QC members must be involved to leave the trenches successfully.
Do you know about agile testing quadrants? Are you aware of shift left testing?
“Agile Testing Quadrants” is a visual tool for understanding or categorized different SQA tests, derived from Brain Marick scope, with this tool QC members are able to discover, build, plan, and execute the testing needed.
The tool provides a glance to differentiate between business and technology facing tests, and those that “support the team” or “critique the product” so that the whole team can understand the testing process successfully.
“Tests on the left-hand side are those that guide development, the ones that are written before coding happens or concurrently as coding proceeds. The tests on the right-hand side are those that critique (evaluate) the product after coding is complete.“*
Book — Agile Testing condensed*
It is important to emphasize that the quadrant numbers do not imply any order, to summarize the quadrants are merely a taxonomy to help and support teams plan their testing approaches and ensure they have all the resources that they need to accomplish them.
Shift Left Testing
As I mentioned above, the typical quality model just focusing in the last part of the SDLC, generally during testing and maintenance phases to accomplish quality goals, on the other hand talking about shift left testing, quality must be through all the processes involved: scoping, requirements, design, code, client interaction, customer review, testing, acceptance, production, maintenance, and so on.
Nowadays terms such a BDD, TDD, ATDD, CI/CD, APIOps or DevOps becoming extremely popular, with these testing methods or practices the evolution regarding testing activities are “a bit” different from the past, shift left testing is an approach related to software development where testing is performed earlier in the lifecycle, one of the aims of shift left testing is preventing or detecting errors as soon as possible in order to mitigate expensive defects or blockers through the SDLC start to end.
This approach involves developers, customers, or any stakeholder of the project, according to this the quality becomes a key factor into the pods, spreading the quality culture across the organization.
Advantages
- Agile testing as a whole team earlier in the cycle, this saves time and money.
- Reduced human error rate thought all the SDLC (Scoping to release)
- Monitoring performance over time
- Quality increase continuously
- Automated testing is increased
Disadvantages
- Without agile testing mindset, the objectives were not accomplished as a team, siloed issues (BA, SM, PO, QC, etc.)
- Lack of leadership, culture, or support during the implementation.
With the implementation of shift left, shift right testing appeared as well, this approach is related to the idea of performing actions later in the development process for example in pre-release and post-release phases
Techniques involved with shift right testing can include:
- Canary releases.
- Chaos testing
- A/B testing.
- CX-based testing
- Fault injection testing.
- Production monitoring
- Destructive testing.
- User acceptance testing (UAT)
As you can see, we have shift-left testing focusing into the process, moving testing earlier in the SDLC and the shift-right testing as well, concentrating on applying testing practices after our application has been deployed. With these approaches combined the “continuous testing” practice emerges in the agile projects, this is a practice where we are trying to drive rapid feedback to development as changes come in, identifying that an issue is detected, let development team know, get it fixed, promoted and quality increased. Automated test cases are a key element here, however, take into account in which level you are located: unit, API, UI or exploratory testing; Be aware, to implement continuous testing it is mandatory that, a holistic quality view is generated through the entire SDLC and the processes defined(BA, DEV, QA, others).
It is clear that we have advantages and disadvantages for each approach or methodology, our responsibility as QC members is to define or to create the best strategy or process, however software quality assurance is not an independent area, we should be aligned and integrated with the governance.
According to the panorama previously illustrated, there is not a silver bullet to resolve all the issues that could occur in any project or methodology followed, for that reason, I will give you some tips or key factors to leave the trench safely.
Firstly, software quality assurance must be a priority not only associated with QC members, all the different stakeholders must collaborate in this aspect though the entire SDLC, it is required not only in products, but also in processes, the agile methodology aims to implement QA at each stage of the project’s lifecycle so that in the process you can test frequently, early, and continuously.
Secondly and aligned with the first point, it is important to participate covering the SDLC, in SCRUM for example the ceremonies give you the opportunity to improve multiple aspects: refinement, planning, estimation, etcetera; in these SCRUM events, we are able to provide feedback or comments associated with other “arts” like business analyst, product management, or development if it is the case.
Thirdly the QA process must be defined properly and according to the needs of the project, the scope could cover functional, non-functional or automation, in this point the leadership and management are key element, in order to accomplish quality goals, otherwise the results will be a project plenty of gaps, misunderstandings, failed point of actions and probably a tired team that won’t be committed.
Last but not least, unfortunately it is common behavior that quality assurance area becomes in the bottleneck of the projects, during these conditions the pressure and rush situations lead to wrong approaches; please create numbers metrics and KPIs are useful, perform RCA to identify pain points, decrease the lack of confidence in your regression or automated test suites… and please remember this phrase before take any decision, “You can’t produce a baby in one month by getting nine women pregnant.”
Probably, if you follow these suggestions, you will survive in or out of the trenches.
If you are here, thank you so much for your time, hopefully it was an enjoyable and applicable reading.
