Verification and Validation In Sprints
In agile development, both verification and validation happen as close to simultaneously as possible. Largely, this is due to the necessity to keep refining and updating the user stories, requiring small V&V loops for continuous feedback.
Prior to any further exploration on this however, we must disambiguate the full definitions of the terms ‘Verification’ and ‘Validation’.
A test of a system to prove that it meets all its specified requirements at a particular stage of its development.
An activity that ensures that an end product stakeholder’s true needs and expectations are met.
However — acknowledging the fact that verification and validation activities happen practically in tandem, it is vital to find as to how teams could enable the process to occur quick, whilst retaining a high level of quality — this is imperative to the Agile process.
Automation first approach
Every story that gets played in a sprint should be verified and validated automatically, wherever possible. A git push should run both verification and validation tests, and provide fast feedback.
So, what helps automation?
- Good analysis sessions
- Automated code review
- Service virtualization
- Synthetic data generation
- Thin slice regression for end-to-end automated tests
Testability of the software component
When the testability of the software component is high, achieving the following criteria is made simpler:
- Every component undergoing a test has a single, well defined responsibility.
- The degree to which we can automate our test at various layers of the architecture is high.
- The degree to which we can use diverse technology and diverse test method in parallel is high.
For a quality engineer to design a good test automation framework, i.e. there is good layering and the tests don’t just run on the UI layer, a very good understanding of the system architecture is important. Validation starts at the lowest component level.
A strategy for a non-production environment is always useful. The environments should be easily provisioned, torn down and spun whenever needed. In a route to live pipeline, teams usually have Dev, Test, Staging and Production environments.
For each non-production environment it is worth considering:
- Let’s state that the dev environment is owned by a team. This team should have the flexibility to spin up as many environments as is necessitated. Most of the validation and verification tests on the services owned by the team should be run here, and it should make best use of the service virtualization (mocks and stubs) in the dev environments.
- While a dev environment might be needed to run all automated tests for verification and validation, an integrated test environment might be needed to validate scenarios which cannot be automated, or perhaps required in order to run end-to-end regression tests. The aforementioned environment should aid in supporting our quality-with-speed approach.
- In order to collectively create an effective environment strategy, some things for teams to consider include a strategy around how test data is generated and used, so as to limit the use of live data.
- Good DevOps processes supported by the right non production environment set-up would help make faster and safer verification and validation activities within sprints.
Techniques and Practices
Ultimately, the goal is to achieve faster delivery in sprints, whilst retaining a good amount of quality.
Software development and testing techniques and practices have a lot to contribute to that. Teams could consider —
- Using the TDD/BDD development process
- Risk based testing - for test scenarios that cannot be verified or validated automatically, as it uses risks to prioritise and emphasise on an appropriate test’s scenarios.
- Exploratory testing (this feeds into the maturity of our automated test suite).
- Model based testing, validating whether the runtime behavior of the software under test is contradictory to the prediction made by the model.
In conclusion, for enabling quality-with-speed in our verification and validation processes, we need to remember that:
- Automating everything is not possible, and should not be primarily focused on.
- Creating editing and maintaining manual tests scripts is not only time consuming, but cannot give us the confidence wanted to release software in an agile process.
Confidence in the quality of software comes with faster feedback. When even a minute change is made to the source code, the verification and validation tests should happen soon enough to enable continuous integration and deployment.