About Quality Engineering at SafetyCulture
We recently hosted our first Quality Engineering Meetup at SafetyCulture, where I had the opportunity to talk to other leaders and engineers in the community about how they approach building quality software. I’m always amazed to hear how different every organisation’s approach is; even their definition of quality differs significantly to reflect their culture, business objectives and organisational maturity. Sometimes there are similarities in the vision, but all of us are on a different part of the journey there.
In this blog post, I’ll talk about what quality means for SafetyCulture and what our quality engineers (QEs) are doing to help us progress through our journey.
What is Quality at SafetyCulture?
At SafetyCulture, we believe in a holistic definition of quality.
“Quality means shipped software which surpasses our customers’ expectations, delivered in a dependable and consistent manner, in an environment where our teams can move fast with confidence.”
Quality at speed that scales can only be achieved if everyone involved in the delivery lifecycle thinks about quality and is held accountable for the quality of their deliverables; everyone makes design choices with quality in mind.
The staple of our quality approach is a combination of a process where everyone involved knows what they need to do to ensure a quality outcome, well-designed automation at the appropriate levels, and customer-experience-focused targeted manual testing guided by data and risks.
Instead of providing a service to find bugs, our quality engineers focus on preventing issues in the first place by working along the whole delivery lifecycle, uncovering risks and advocating for testable designs in the early stages, making sure that edge cases are identified and that there is a common understanding of what “Done” looks like.
We help teams set ambitious quality goals such as zero bugs older than four weeks, surfacing the right metrics to the right stakeholders, improving test coverage, and providing an easy-to-use testing toolset.
We captured our guiding principles in the SC Quality Engineering Manifesto with detailed explanations of why and how we’re working towards them.
These principles are intentionally kept high-level, as due to the different maturity levels (differences in product area, customer base, team tenure and process maturity) the specific activities required in each team are different.
With that said, one of our main goals is to continuously improve the quality maturity of our teams and drive alignment.
Whole Team Approach
In the past, quality engineers or testers have been the “gatekeepers of quality”. Having a certain person or role being responsible for quality is inefficient and creates a bottleneck. Preferably, everyone in a product team thinks about the quality implications of their decisions and everyone is involved in quality-related and testing activities — the quality mindset has to be adopted by the entire team. Whether it’s thinking about the performance impacts of a code change, or the risk areas of making a design change / UI redesign, everybody needs to have a quality-driven thought process.
The essence of the whole-team approach lies in the quality engineers, developers, engineering managers, product managers and designers working together in every step of the development process.
The whole team should analyse these areas together to produce better quality software. This approach requires constant collaboration. Initially, QEs are involved in the majority of design discussions and acceptance criteria reviews. As the team’s maturity is increasing, the team becomes more proficient in performing said activities without the QEs so they can focus more on the continuous improvement and customer advocacy part of the role. Also, understanding the different, sometimes peculiar use cases of our customers, uncovering more and more edge cases, and testing under different circumstances (low connectivity, budget devices, etc.) are all very time-consuming activities.
This transition from individual responsibility to collective ownership is often the hardest part of the cultural shift that teams face when adopting an advanced quality-engineering approach.
Activities Performed by Quality Engineers
The QE team’s role in the organisation is two-fold:
- We are enablers, helping teams implement advanced practices that result in high-quality outcomes at speed
- We support our feature teams in their regular delivery work
Our quality engineers are working across 3 different levels:
- Product area (feature team)
- Platform level initiatives
- Engineering-wide initiatives to help us build world-class practices
These activities ensure that learnings and skills are shared to drive consistency across product areas and to make sure problems are not recurring.
Quality Engineers’ Responsibilities in the Feature Teams
Quality engineers are accountable for the below activities in their product areas (based on the maturity and capacity of the team, QEs either conduct, pair, or train/coach the activities).
- Work with the product team to understand and document current quality practices and the team’s quality maturity; create an improvement roadmap to fill in potential gaps together with key stakeholders and the whole team. Follow through with implementing the roadmap via regular quality retrospectives
- Define a team-specific quality strategy to support the teams’ business goals and OKRs and that is aligned with the SafetyCulture Quality Principles
- Document and visualise automated test coverage and collaborate with developers to achieve/continuously improve required coverage
- Measure and communicate the quality status of the product area, including internal/external qualitative and quantitative metrics such as:
- Customer advocate insights
- Defects, Customer Problem Reports, incidents, alerts and incident improvement action dashboards
- Leading quality-orientated projects
- Role modelling, training and coaching technical and non-technical testing practices
- Supporting feature delivery as a domain Subject Matter Expert (SME)
- Assisting in defining Acceptance criteria
- Risk assessment
- Assist in defining test plans
- Pair testing sessions
- Assist in defining automation plans
- Organising and coordinating ad-hoc sessions like dogfooding, bug bash, exploratory testing and community events
- Timely and continuous feedback on where the quality bar is met or (more importantly) not met, holding the team accountable
- Contribution to reported issue investigations (as product area SMEs)
- Test device management
Engineering-wide Initiatives
The Quality Engineering department roadmap is divided into multiple tracks, capturing work grouped by themes and owned and driven by our quality engineers. These tracks are aimed to help us build world-class quality practices, and also give opportunities for our QEs to grow and develop their leadership skills.
Better guidance — Quality Coaching Track
The goal of this track is to increase developer confidence in conducting the expected quality-related activities and to improve quality-consciousness (culture) across the company.
- Helping quality engineers to become more efficient coaches themselves, and coaching product teams on testing practices, designing for testability and test planning.
- Creating, documenting and sharing recommended quality best practices in the QE knowledge hub and creating training materials (including training videos and courses using our own learning platform, EdApp for this).
Automation Coverage Track
The goal of this track is to increase confidence in our releases and to reduce manual testing overhead so capacity can be focused on more value-add activities like exploratory, UX, and non-functional testing. Visualising current automation coverage levels on different layers in each of the features owned by a team; creating a formal test ownership model and clear guidance on how to create, troubleshoot and update tests; continue to research and develop new, more efficient test frameworks.
Planning Track
Make sure all product teams have an efficient way of planning quality activities and set ourselves up to find problems early
- Roll out test planning format and process (risk analysis, quality activities part of the capacity planning, AC review)
- Bake in test planning as part of the feature planning process
Mobile Testing Track
The purpose of this track is to describe the current state of mobile testing at SafetyCulture and create a single testing strategy that brings full alignment and therefore efficiency and consistency across the different teams involved in the app development, focusing on automated test coverage and testing processes.
Defects Track
High-quality defects, effortless and automated analysis to understand trends, and clear guidance on resolution. Establishing an aligned defect life cycle across all teams, improving the collection of defect meta-data and generating and publishing triage packages at the organisation and team level.
Data-driven customer experience improvement
This initiative’s goal is to help our company deliver a delightful experience to our customers whether it’s on web or mobile. User Experience issues do not usually block our customers from performing a task, but if we don’t start improving them, customers will start to have friction and a less pleasant experience, thus overall lower engagement.
Raising the bar from “the experience is good enough and our customer can do their jobs” to “our customers love the experiences, they are more productive and happier when performing their day-to-day tasks”.
We will come up with best practice recommendations on how to prevent, detect, track, and resolve issues that will have a negative impact on users.
Final Thoughts
It is important to note that the above is a snapshot of the current state with a few examples. We’re continuously working on evolving our approach as our business grows, our environment is changing and as we learn from our successes — and yes, from our mistakes as well.
It is impossible to capture every aspect of the quality journey in one blog post. This time we started with a high-level overview, and we’ll return with more details.
