Automation Framework using Playwright
Quality Assurance is an essential part of each software application, as the objective of Quality Assurance is to ensure that all the functional aspects of the application are tested and defects are identified.
Why Test Automation?
- When an organization aims to accelerate release cycles more frequently with an improved quality of the software solutions, integrating an automation testing framework and strategies into the software delivery pipeline is the right approach.
- Automated tests undeniably protect the quality of software projects, and end-to-end tests are an important part of it.
- Though manual testing is suitable for any Exploratory, Usability and Ad hoc testing, it has a higher risk of missing out on the pre-decided QA/ release deadline which is not the case with an automated test suite. Automated regression also has an impact on processing time (ultimately on product release deadlines) as automation is significantly faster than a manual approach.
What is Helpshift Dashboard?
- The Helpshift Dashboard is a user interface that provides at-a-glance views of Helpshift’s key features for the agents to view, and customize such as the issues page, FAQs page, analytics page, and settings to create and customize bots and other functionalities. Hence, it is comparatively complex when it comes to manual testing of the dashboard.
Why Playwright?
- Playwright is a relatively new open source cross-browser automation framework for end-to-end testing, developed and maintained by Microsoft.
- Playwright is highly useful for performing cross-browser testing on complex applications. It provides features such as,
✓ Any browser — Any platform — One API: Cross-browser, cross-platform, cross-language testing.
✓ Resilient — No flaky tests: Auto-wait, web-first assertions and tracing features eliminate flaky-ness.
✓ No trade-offs — No limits: Test scenarios that span multiple tabs, multiple origins and multiple users.
✓Full isolation — Fast execution: Playwright creates a browser context for each test. Browser context is equivalent to a brand-new browser profile. This delivers full test isolation with zero overhead. Creating a new browser context only takes a handful of milliseconds.
✓Log in once: Save the authentication state of the context and reuse it in all the tests. This bypasses repetitive log-in operations in each test, yet delivers full isolation of independent tests.
✓Powerful tooling: Such as Codegen, Playwright inspector and Trace viewer.
Framework Architecture
- We have built an automation framework using Playwright in Java language binding which aims for the end-to-end test automation for the dashboard.
- The automation framework uses POM, a common pattern that introduces abstractions over web app pages to simplify interactions in multiple tests. For example, we have created a DashboardIssuesPage helper class to encapsulate common operations on the Helpshift Issues page on the dashboard. Internally, the automation framework will use the page object to write the tests in the DashboardIssuesTests class file.
- A multi-threading is implemented, with the help of which we can either run the entire suite or only run feature-specific tests. (This will help in cases where we know the impact of a change is only on a particular feature, say issues page on the dashboard)
PS: Stay tuned for a multi-threading approach. :)
The architecture diagram illustrates the automation framework developed for Helpshift’s agent dashboard.
Allure Test Summary Report
- Allure Reports are flexible, lightweight, and more verbose, providing results at suit levels such as status graphs, severity, and the duration and test levels such as each step of the test execution and screenshots of the test failures, etc.
- Allure reports are integrated with the automation framework that stores test run data in CSVs and JSON files. Typically, Allure needs a host to serve all the HTML and JS files that it creates. Since we’re running our tests on Jenkins, someone viewing the test report need not set up allure on their machines. All they’ll need is access to Jenkins.
- Allure gives an amazing historic trend per build that can help us deal with test flakiness or to figure out when an issue has been failing on an environment. Screenshots of the reports are attached below.
Conclusion
A playwright is a great tool with which you can easily write end-to-end tests for an application. The API is simple and it is easy to quickly get started with the codegen feature provided by Playwright.
