SQA: The Secret Sauce to Sendoso’s Success

Amna Malik
Nov 15 · 7 min read

Software Quality Assurance is a set of methods and activities to ensure that the developed software meets all requirements. It is a continuous process of verification and validation of all functionalities, which ensures the delivery of a high-quality product to the end-user. Each phase of the SQA life cycle determines the platform’s performance and usability. Also, it certifies all the technical and business requirements with an ability to scale.

Test Automation and its importance in Sendoso

An extremely dynamic platform like Sendoso requires an effective testing strategy that allows it to quickly scale and innovate itself to keep up with the challenging demands of consumers without any issues. To achieve this, Automation Testing becomes a key element in any test strategy.

Early Challenges

Some of the challenges in the start were as follows:

  • Manual execution of repetitive test scenarios was time-consuming for daily releases
  • Dependency on full-stack engineers to define testing scope due to limited involvement of QA in project planning
  • Delays in release deployment caused by inaccurate estimations of manual testing efforts
  • Consistent human errors while testing manually
  • Inconclusive and ambiguous reporting of bugs
  • Lack of documentation

Our Main Strategy

While manual testing is here to stay, we understand the importance of designing and following a structured strategy for test automation. Creating test strategies means that teams are better informed on how to go about the testing.

Single Source of Truth

TestRail, a test case management tool, was introduced for documentation, providing engineers with a single source of truth for all domain-relevant documentation. Anyone within the company can review product-related test cases using TestRail instead of navigating through multiple platforms. TestRail also assisted with the onboarding of new engineers.

Identifying the Right Tool

Initially, each team started with a separate automation tool but quickly identified that using different tools implied different capabilities. To overcome this, SQA engineers researched a variety of tools including Selenium, Cypress, WebDriver.io, Puppeteer and also explored numerous frameworks like Jasmine and Cucumber. After carefully analyzing the requirements and limitations of Sendoso’s platform, the SQA engineers identified Puppeteer as the Automation tool that best manages all expectations.

Sendoso Platform has many integrations which require API callbacks and multi-domain redirections. Puppeteer is a Node library that provides a high-level API to control Chromium or Chrome over the Devtools protocol. It uses a hybrid framework, i.e., Behavior-Driven Development (BDD) and Data-Driven automation framework with Cucumber and JEST with Javascript as the scripting language. Puppeteer is a fast, consistent, and reliable automation tool.

Deciding What To Automate

Through organized documentation that all the relevant engineers review, we were able to identify and plan efficiently which tests needed to be part of the automated smoke suite. This especially made it easier to manage the daily release testing process. Moreover, it became clear what:

  • can be automated
  • cannot be automated
  • should not be automated

This helped us in automating the major features that needed to be tested recurringly to ensure long-term sustainability. Major features can be defined as all those functionalities that allow the user to successfully navigate through most of the platform. This way any feature enhancement or new functionality request affecting the rest of the platform can be integrated in a risk-free manner.

We understand the limitations that automation has and try to bridge that gap through carefully planned manual testing. Functionalities that are not part of the major flow of the platform or suffer no impact from any enhancement throughout the application should not be part of the smoke test suite. Similarly, email verification that requires third-party navigation and hence, cannot be recurrently tested.

Setting up the Right Environment

When performing automation testing, a stable environment is needed that is not rapidly changing and has the same capability as the production environment. This is useful when identifying any performance-related issues on our platform.

Scripting and PR Reviews

To enhance and scale our Automation Test Suite, each Github PR is reviewed by SQA engineers as well as full-stack engineers. We attempt to follow the standard review practices with valuable feedback. This allows us to refactor and incorporate any changes in functionality to improve the reliability of the automation test suite consistently. The testing scope also becomes clear for every engineer by having SQA Engineers involved in the Pull Request process. SQAs help Software Engineers learn about how the larger product works.

Execution on CircleCI (Continuous Integration)

Once the PRs are reviewed, approved, and merged, we can use Continuous Integration (CI) to ensure that the automation suite is executed with the daily release deployment. We cannot execute an exhaustive test script on each deployment as it takes a lot of time to test each functionality in detail; instead we add part of the script that is called smoke suite with the release process. Repetitive manual effort is no longer required.

Bug Reporting

Bug reporting has improved massively as well. Through our automated test execution on CI, the issues are quickly identified and reported as bugs with all the relevant information added automatically. We follow a standardized bug reporting template and assign it to the relevant engineer on JIRA. The engineer reviews the bug and can track all the required information with a complete and precise automated test report. If the engineer understands the issue and fixes it, will mark it as ‘ready to test.’ Once the bug is fixed, the SQA engineer verifies the fix before deploying it to any shared environment. After this, the smoke suite will execute again and ensure that the issue is resolved and has not affected any other part of the platform. Finally, the engineer marks the bug as ‘resolved or fixed.’

Hackathon

Engineers at Sendoso always look to conquer challenging problems with creative and practical approaches. For this purpose, Sendoso organized a Hackathon event that provided every engineer with a forum to express their creativity that ignites innovation. It provided the engineers with a rigorous but extremely valuable experience and a steep learning curve to enhance their skills and get extra valuable exposure.

QA engineers participated in Hackathon as well, with the aim of fast-tracking automation testing into the project life-cycle. They researched different tools and tried various challenging scenarios to better understand the testing capacity and reliability of the tool. This played a vital role in redefining our testing strategy.

Contributions of Automation Testing at Sendoso

  1. A drop in recurring issues
  2. Increased productivity translated from platform stability
  3. Achieved faster and reliable testing
  4. Boosted ROI on profitability, enhanced customer experience and satisfaction
  5. Huge drop in the manual testing effort
  6. Rapid test execution and project delivery
  7. Improved confidence in the platform’s capabilities
  8. Received swift feedbacks on failures, in line with the “fail fast, fail early” principle
The chart shows the number of issues based on the Created field

The test plans are developed alongside regular roadmap feature development then executed automatically by continuous software integration tools. At Sendoso, automation testing promotes lean QA team size and enables them to focus on more sensitive features.

Way Forward

Sendoso actively carries out continuous research to identify better strategies and test tools. This pursuit also reinforced the fact that clear and accurate reporting is key. Engineers firmly believe in comprehensive reporting as it assists in drastically improving processes. We believe in our ability to create a robust and efficient test plan to help detect bugs early on in the software development life cycle.

Acknowledgments

Shoutout to Afshan Shakoor, for her prodigious support related to designing, implementing and improving the company’s quality processes.

Sendoso Engineering

The technology behind the world’s first Sending Platform