Quality at Insider: An In-Depth Look at Our QA Process
At Insider, we take quality seriously. As a software development company, we recognize the critical role that QA plays in ensuring that our products meet the highest standards of quality.
We have a dedicated QA team that is responsible for overseeing the quality process. By implementing a robust QA process, we reduce the risk of costly and time-consuming issues, improve the user experience, and build customer trust and satisfaction. Our QA team works closely with the development team to identify potential issues and develop strategies for addressing them. This collaboration is key to ensuring that our products are of the highest quality.
In this article, we will explore our approach to QA and how it fits into our overall development process.
Hiring and Onboarding
When it comes to hiring QA engineers, we have high standards. We look for candidates who have experience with both manual and automated testing, as well as a strong understanding of the latest QA methodologies and tools. The process for hiring a QA these several stages:
- short phone interview with HR, where the candidate is asked to provide an overview of their background and experience. This is also an opportunity for us to introduce ourselves and provide a general overview of position and processes.
- completing an assessment, which involves a specific scenario and typically designed to assess the candidate’s ability to solve problems and identify defects in software.
- technical interview with Insider QA team members, which is an opportunity for the candidate to demonstrate their technical knowledge and expertise. It may involve questions about testing methodologies, tools, and processes, as well as more general questions about the candidate’s experience and background in QA.
- interview by the Head of QA & Security and the CTO of the company, which is an opportunity for the candidate to showcase their communication skills, as well as their understanding of Insider’s goals and vision.
Once a candidate has been hired, they go through an extensive onboarding process to ensure that they are fully prepared to join the QA team. This process includes:
- Training on the tools and processes used for manual testing, such as test case management and bug tracking
- Explanation of the different types of testing, including functional, regression, load and user acceptance testing
- Training on the tools and processes used for automation testing and continuous integration
And of course since Insider is a B2B company and our goal is to ensure that our customers are satisfied with the products and services we provide we also regularly conduct customer-centric training sessions such as:
- Introduction to how to identify and prioritize the needs and preferences of the end user
- Training on how to incorporate customer feedback into the development process
Why do we want our QA Engineers to contribute to both Manual and Automated testing processes?
At Insider, we believe that QA engineers should be involved in both manual and automated testing processes. This is because each type of testing has its own unique benefits and limitations.
First, manual testing allows QA Engineers to identify and document defects, usability issues, and complex user flows that may be difficult to automate. This type of testing also allows QA Engineers to verify that the software functions as expected from the perspective of the end user, under various conditions and scenarios. It also enables the detection of UI/UX issues that can make the usage of our products easier and more effective.
On the other hand, since we have complex products with numerous features in each of them automated testing can be more efficient and cost-effective in the long run, as it allows for the execution of repetitive tasks and the detection of issues on a larger scale (i.e., issues that were previously fixed but have resurfaced due to recent changes).
In other words, in our process automated testing complements the test process by providing additional coverage and identifying issues that may not be easily caught through manual testing due to the complexity or time required to test them.
By contributing to both manual and automated testing processes, QA Engineers can ensure that the software is thoroughly tested and that any issues are identified and addressed before the product is released. This ultimately helps to improve the quality and reliability of the product, which is essential for maintaining customer satisfaction and trust.
QA’s Role in our Scrum Teams: Ensuring Quality and Efficiency
In the world of agile software development, the sprint is a key concept. During each sprint (in our case it’s 2 weeks) our teams deliver all ready for release tasks and for every sprint there are goals that teams have to achieve.
One of the key stages of a sprint is the testing stage. This is the time when the team puts the product through its paces to ensure that it is functioning correctly and is free of defects.
During the testing stage, our QA members do
- exploratory testing, in which team members manually go through the product and test its various features and functions
- automated testing, in which our automation tests suite runs for every branch and checks product’s functionality if it’s effected by a new feature/fix or not. Automated testing not only involves checking the results, but also adding coverage for new features and fixing the logic of existing automation tests if necessary.
After all tests have been completed, a task moves on to the security and UAT stages. These stages involve additional checks and tests to ensure that the task meets the necessary security standards and has been thoroughly tested in a user acceptance environment.
Some of QA’s roles in the team during a sprint is:
- Participating in sprint planning, daily stand-up meetings, and other Scrum events.
- Identifying, documenting, and tracking defects and issues.
- Providing guidance and support to other team members on quality and testing related matters.
- Automating by adding new coverage or fixing existing cases for every task that is checked manually and reviewing other QA contributors’ pull-requests.
- Conducting regression tests as new features/fixes can have an unintentional impact on older features.
QA’s Role in the Deployment Process:
Here at Insider, we strive to keep our partners satisfied by deploying tasks as soon as they are ready. We understand that waiting for features or bug fixes for several days can be frustrating, so we have implemented a deployment schedule that occurs 5 days per week. Additionally, if a particular team or product requires additional deployments, we are willing to accommodate their needs as necessary. By quickly deploying tasks as soon as they are ready, we are able to provide our partners with the timely support and updates that they need.
Once a task has undergone testing and all checks have been successfully passed, it is ready for deployment. As a standard procedure, we have implemented the “qa/smoke” check for all pull requests in all repositories where changes to the code could potentially impact quality, user experience or even triggering test machines for next tasks. Although it may seem like a small and insignificant requirement at first glance, the “qa/smoke” check serves as a safeguard to help prevent any potential issues that may arise as a result of code changes.
In case a test only fails on the current branch, we send notifications to both the Jira task ticket and the QA’s Slack. This allows the QA team to focus on the cases that have failed specifically on their branch, rather than trying to troubleshoot issues that may be caused by flaky tests. (In a future article, I will go into more detail about our approach to dealing with flaky tests in our process)
If a QA engineer determines that the failed cases have or do not have any relation to the current task, they have the option to approve or decline them based on whether or not the changes in the code have had an impact. If QA finds any suspicious moments that require further investigation or checking, they decline the results. This allows us to identify and address any potential issues before the task is deployed. Once QA has approved the results, the task is considered ready for deployment from their perspective. This thorough process helps to ensure that we are only deploying tasks that have been thoroughly tested and are of the highest quality.
Every day between 2–8 pm, our Arcanine bot (written with Nodejs and Express Arcanine is a deployment management bot to handle the Jira Issues and GitHub pull requests by merging both sides together) collects approvals for tasks from both the responsible developer and the responsible QA. If either of these individuals will not be available the next day, the task should be declined. This ensures that we have the necessary resources in place to complete tasks efficiently and effectively. By coordinating with both the developer and QA team, we are able to ensure that all tasks are properly approved and assigned before moving forward with the deployment process.
Every day at 7 AM, our pre-production process automatically starts by triggering 5 machines with all merged tasks that were approved the day before and running different automation test suites. To ensure the smooth running of this process, we have both a weekly QA guardian who is responsible for overseeing the entire QA process, as well as a daily product QA guardian who is assigned to monitor automation tests and quality of their own product. These heroes play a crucial role in identifying any critical errors that may arise and alerting the team to take action — they may recommend reverting the changes or preparing a hotfix to address the affected area.
While the preproduction process runs automatically and the guardians follow up on it, the QAs responsible for each task also perform manual testing in that environment. Once both the automation and manual checks have been completed, the weekly guardian gives their approval and the deployment process to production begins.
Once all repositories have been deployed on production, the QA team performs their final live tests while live automation runs. This allows us to confirm that the tasks are functioning as intended in a live environment and to identify any potential issues that may have been missed during the testing process.
Conclusion
At Insider, our QA process is an integral part of our development process. From hiring and onboarding skilled QA engineers to involving them in both manual and automated testing processes, to testing microservice-based applications in AWS environments, we are committed to ensuring the quality of our products. In a future article, I will go into more detail about our e2e automation projects written in Python/Java and Selenium, with more than 50 contributors.
Are you interested in joining our QA team? We’re always on the lookout for talented QA engineers who are passionate about testing and finding bugs. If you think you have what it takes, we’d love to hear from you! Please contact us via LinkedIn or email.
