Roles and Significance of Test Quality Assurance (TQA)
“What does the TQA team do? Is this team also part of KBTG? I’ve never heard of them before. What do they work on? I’m curious to find out.”
It’s a question we often get when people inquire about what team we’re in. So today, I would like to take this opportunity to open the doors to TQA and let everyone get to know us better.
Test Quality Assurance
TQA, or Test Quality Assurance, is a team under SQM (Software Quality Management). Currently, we have 8 members led by Kay Natkritta, our Head of TQA. Despite our small team size, our workload is anything but 😂. Our responsibilities include overseeing various processes related to testing to ensure standardized quality. We diligently check the quality of processes during testing through various controls such as Quality Gate, Quality Index, and many others. Rest assured systems that have undergone testing by SQM before production are of unquestionable quality.
The main responsibilities of the TQA team can be divided into three key areas:
- Standard and Process
- Governance and Quality
- Metrics and Measurement
Let’s delve into the details of each topic together.
Standard and Process
The TQA team is responsible for establishing a Standard Framework to design the testing management framework known as TMF (Test Management Framework). We define it based on current working practices and align it with the globally recognized ISTQB (International Software Testing Qualifications Board) standards. Every employee in the SQM department, regardless of their area, must pass the ISTQB Foundation certification exam. This ensures that everyone comprehends the standardized testing processes and contributes to building a solid foundation for software testing.
The major aspects outlined in the Framework are related to the testing process, encompassing KBTG Test Level and Objectives, Roles & Responsibilities for each Test Level, Sub-Phase Activities, Entry & Exit Criteria, Test Monitoring, Tracking, Reporting, Defect Management, Test Data & Environment, Test Metrics and Measurement, all the way to Test DI Tailoring Guideline + R&R.
After announcing the adoption of the TMF, the TQA team has actively sought feedback from everyone using it. This feedback is crucial for continuous improvement, ensuring that the framework meets requirements and provides maximum benefits.
For this year’s Test Templates, due to the antiquated nature of the old templates and to ensure alignment with the announced TMF, we have undertaken a significant overhaul of the Deliverable Item (DI) Test Template. This involved collaborative workshops with the Head of Test Management from each area, resulting in the creation of four distinct templates as follows:
- Master Test Plan (MTP)
- Test Summary Report (TSR)
- Performance Test Plan (PTP)
- Performance Test-Test Summary Report (PT-TSR)
The TQA team also conducts training sessions and activities to build the capabilities of employees in the SQM department, including:
SQM Training
The TQA team has developed Learning Paths for Tech Kamp, New Joiner, Conversion, and Internship, all the way to regular employees based on their job positions. Each individual is guided on the fundamental online courses they should take in a sequential order. This year, our TQA team planned the creation of SQM Training by collaborating with the Heads of Test in each area. The result is a total of 17 courses; however, due to the nature of our training that relies on SQM employees volunteering as instructors, we have opted to conduct only 6 courses this year. The courses are available in both online formats via MS Teams and On-Site Workshops held in the Academy Room at the KBTG Building, Muang Thong Thani.
Tech Kamp
The TQA team collaborates with the Test Automation Team to plan the Tech Kamp program (similar to a Bootcamp designed for newcomers with no prior experience). Each year, there is coordination to determine what topics participants should learn. We provide a list of online courses that each should complete before joining the program with the TQA team. This ensures that participants have a foundational understanding of SDLC and testing practices, enabling them to apply their knowledge to real projects assigned in the future.
Lesson Learnt & Knowledge Sharing
We organize Lesson Learnt & Knowledge Sharing sessions every month to share our knowledge with employees in the SQM department. The topics are derived from feedback gathered in each session, identifying interesting and popular subjects. We reach out to speakers from within the SQM department, external experts, and sometimes invite KBank employees to share their insights. The format is more like a talk show than a traditional lecture, with engaging activities such as themed costume contests or various games. This creates a relaxed atmosphere on Friday afternoons, allowing everyone to unwind while gaining valuable information.
Every time we organize the Tech Kamp program, SQM Training, and Lesson Learnt & Knowledge Sharing sessions, we compile the videos and instructional materials and place them on the KBTG Knowledge Management (KM) Portal. This ensures that everyone within the organization can access and study the content.
In addition to the SQM Library, we also have initiatives in various other areas:
- Self-Service
- QC Utilities
- Service Request
Alongside the SQM Library, we organize SQM All Hands every six months, where everyone in the SQM department comes together for a fun-filled gathering. This year was especially remarkable with the SQM Reconnect event held in Pattaya. Our TQA team was in charge, from coordinating with the hotel to designing all the activities. No external companies were hired for this purpose.
Should SQM employees encounter any issues, we have a TQA Line@, which serves as a Chat Bot to respond to their questions.
Governance and Quality
The TQA team is responsible for controlling various qualities within the SQM department. This involves overseeing the project’s quality, commonly referred to as Quality Health Check, as well as governing various projects under the testing purview of the SQM department. This encompasses both internal and external aspects.
We serve as the Quality Gate, responsible for controlling the overall picture of testing. This is divided into three main gates, namely:
1. Entry Criteria
Check Readiness: Evaluate whether the project is prepared for testing. This involves utilizing an Entry Criteria Checklist for the inspection process.
2. Healthiness Index
After a project enters testing within the SQM department, we conduct monitoring to assess the project’s health. Currently, we have established metrics for measuring what we call the ‘Healthiness Index.’ These metrics are displayed on the Grafana Dashboard.
3. Exit Criteria
It involves verifying that a project undergoing testing is ready to exit and move into production. This is done using an Exit Criteria Checklist, checking factors such as the absence of remaining Defect Severity 1 and Severity 2 issues, ensuring the system functions properly before deployment.
For all three gates, we identify measurable metrics to present as a Quality Index, indicating the project’s quality at each gate. If we can effectively control the quality across all three gates, it strengthens our Test Framework.
When a project enters testing within the SQM department, we conduct a Quality Health Check to assess the project’s quality. This is categorized into three states:
- Critical a project that requires close supervision, having issues that impact budget, timeline, and quality.
- Watch List a project with the potential to impact budget, timeline, and quality.
- Normal a project that hasn’t encountered any issues yet. (On Time, On Budget, With Quality)
We have implemented QHC on the Web (Quality Health Check) for Test Managers to register incoming projects and provide the current status of each project.
Finally, the TQA team conducts a session called SQM Governance every Wednesday, where all three gates are monitored in this session. If any project is marked red or orange, it will be highlighted for further examination. If there are issues that require assistance from the MC Sponsor, the TQA team organizes a monthly session called SQM Leader Monthly Meeting with MC Sponsors. This session serves to discuss highlights from SQM Governance, seek advice, and address any needed solutions. It’s important to note that this governance process focuses on internal aspects only. For external governance, we collaborate with other parties, including Operational Risks (OpRisk) and Internal Audit (IA) from KBank. Additionally, within KBTG, there is IT Governance working alongside the TQA team to govern quality.
Metrics and Measurement
The TQA team compiles data from various sources for different projects to create standardized performance metrics. This involves extracting information from the PPM Tool used by DM and PM, the QHC on Web utilized by Test Managers, and ALM-QC, which stores Test Execution results and manages defects during testing. Additionally, we synchronize defects from JIRA, a tool used by developers in Agile development. This synchronization ensures seamless collaboration between developers and the Test Team, allowing us to comprehensively collect and analyze results accurately.
Furthermore, we have gathered data to conduct Data Analytics that reflects the overall project landscape. Various metrics have been defined according to international standards, allowing us to implement ongoing Quality Improvement initiatives.
As the TQA team is responsible for controlling and ensuring the quality of projects under SQM testing before they are deployed into production, we have established a set of quality metrics known as the Quality Index. Currently, this index is displayed on the Grafana Dashboard, serving as a tangible indicator of the project’s quality. The Quality Index draws data from various test tools such as PPM, QHC on Web, and ALM-QC. It includes a Healthiness Index, which is assessed based on percentage values and color coding, indicating whether the project status is green, red, or yellow according to predefined criteria.
This comprehensive index allows us to evaluate the cause of issues, analyze the testing situation, and identify solutions promptly. It proves beneficial for future development by providing insights into areas that require improvement and enabling us to address issues efficiently and proactively.
In terms of reporting, the TQA team prepares a Test Progress Report to monitor, inspect, and report the results. This report includes various numerical values, such as the number of defects and the number of executed test cases, allowing stakeholders to track the testing progress and assess whether it aligns with the planned schedule. We have developed QC Utilities by scripting macros on Excel to extract data from ALM-QC and generate reports. This automation ensures that users can efficiently retrieve information and calculate various metrics within a matter of seconds.
In addition, we offer various QC Utilities to manage planning and data on Quality Center (ALM-QC). We have developed SQM Service Requests to facilitate service-related inquiries or report issues encountered during usage. This serves as a channel for users to seek prompt assistance from our backend support team.
Having read up to this point, how do you all feel? I hope everyone has gained a better understanding of our TQA team. Quality is something that requires continuous improvement and adherence to international standards. Our TQA team is dedicated to ensuring the delivery of software products to our customers with the utmost efficiency. Thank you for your attention.
For friends at KBTG who are interested in the content we teach or the documents mentioned in this article, you can find them on the KBTG Knowledge Management (KM) portal under Software Quality Management. Feel free to explore and learn more!
And I’ll see you in the next article. Goodbye for now! 👋
For those who enjoy this article, don’t forget to follow Medium: KBTG Life We have knowledge and great stories from KBTG folks ready to be served here first.
