Quality Engineering Redefined
In the business world, QA is just a process at the end to certify a release, but not necessarily to ensure customer needs are intact and/or developer productivity is improved. Conquering this mentality can only be done with a team that has a growth mindset. My journey at Varo corroborates this and redefines Quality Engineering.
What is software quality?
Quality in the true sense is a #CustomerFirst mindset. If you drive all the software engineering life cycles around this mindset, you achieve a quality product.
Good software performance is the customer experience you want to provide to make customers happy, and it’s achieved through engineering excellence.
We started with the task of building a bank from scratch. How does that translate for ‘Quality Assurance,’ aka ‘Testing’ teams, within the organization? Writing huge test plans, test cases, plan to automate but not exactly sure how? Make a huge performance test plan?
This traditional approach to QA didn’t lead us to where we are today with Varo Bank.
Role of mindset
We started with cultivating the mindset that Quality or performance is not owned by just one team. It needs to be owned by everyone working for the customers.
Of Course, setting the vision and building a team of Software Development Engineers in Test, aka SDET, proved instrumental to our success. Even changing the name of the functional group to ‘Quality Engineering’ was a key part of the mindset shift.
Defining the role of QE
In the conquest to find successful QE, aka SDET, we were required to define the role aptly. We were not looking for people with a mindset of just testing or coding as a role. Our need was to find candidates who like to bridge the gap between different teams and roles. Ideally, they would pull from each of the following roles:
- Developer, to understand the code implementation and breaking points
- DevOps Engineer, to understand the deployment process and contribute to CI-CD
- Product person, to recommend customer needs under different scenarios
- Designer, to recommend customer usability needs
- Customer, (The most important one) to represent a customer-first attitude
High road
Our mighty team of a handful of SDETs at various levels was tasked to create the entire efficient automated engine of test execution for a bank from scratch.
Embedding SDETs within development teams with a new microservices-based architecture segregated in three main layers (client, orchestration, and microservices) allowed us to develop a test strategy targeting the right concern at each layer.
‘Shift Left’ is synonymous with arresting the issue early in the Dev lifecycle, but unless you ‘Focus right base,’ the strategy would still not be effective. That means focusing to test what is developed in each layer.
And being part of how each layer is getting built provides enormous insights to an SDET on how to ‘Focus right base.’
The microservices layer targeted the whole business logic, so we built our tests at this layer heavy but are built to be efficient.
The orchestration layer targeted communication between the client and microservices layer with a defined schema, so our tests targeted headless client tests from this layer down, which is still very efficient to run and maintain.
Now to the client layer, which most people assume is the only layer in which the tests should be run.
We chose to stick with the intention of this layer too and built a thin-set covering whole features following the 80/20 rule. That rule simply means 20% of features or tests actually cover 80% of customer usage.
How do we measure success?
‘Success’ is a relative term, but to measure it you need to be very clear about what you would like to achieve. Our goal was to be a ‘Trusted Quality partner in delivering the highest business value, quickly, predictably, and consistently.’
So we started with measuring where we are in terms of org, process, tools, and trust. Then we defined where we would like to go on these same terms.
Defining operational excellence and clear execution goals helped immensely in the journey.
What is the real measure of success? As the old adage goes, “The proof is in the pudding.” Our small but mighty team was able to churn through millions of tests running seamlessly in the CI pipeline and schedulers to successfully launch a customer-centric national bank. We did all that with no major disruption and we consistently adhered to engineering excellence principles. Check out the Varo Bank web app here.
Scale with QE
To accommodate scale, do we need to keep expanding the organization? As ‘automation’ is not just magic, it takes a workforce to maintain it. It’s very expensive, isn’t it? Not really!
It might be expensive, but if planned well it will pay for itself.
While building the base tests itself, we tried to adhere to DRY principles, i.e. do not repeat the same tests over and over again. Building thousands of test cases and hundreds of test suites gives no pride to our QE. On the contrary, the more tests, the more effort it takes to maintain, which makes it inefficient. So keeping it lean from the start and understanding each new feature addition can be a modification in tests rather than an addition, which helps us achieve the goal.
With the QE mindset embedded within the team, the role is not just for a single person in the squad. It’s a shared role within the squad.
This mindset allows us to be nimble but fast and provide scope to scale.
Conclusion
Is the goal accomplished in the journey? Not Really!
To incorporate the scale, it’s important to go to a touchless delivery model and realize the true level of engineering excellence. But at the same time, we have most surely arrived at our first fast track destination from point 0 to 1.
The Journey to V2.0 with touchless deployments is developing and will be the essence to capture in another blog.
Hence, I’d like to conclude with W.A. Foster’s quote, “Quality is Never an accident. It is always a result of high intention, sincere effort, intelligent direction, and skillful execution. It represents the wise choice of many alternatives.”
