Automation Functional Test Coverage For Frontend Applications

Synopsis:

In the ever-evolving world of software development, frontend applications have gained paramount importance due to their direct impact on user experience. Ensuring the reliability and functionality of these applications is critical for delivering a seamless user experience. This synopsis outlines the significance of automation in achieving comprehensive functional test coverage for frontend applications. We delve into the importance of comprehensive functional test coverage, highlighting how it ensures that all critical features and user interactions are thoroughly tested. In summary, this synopsis offers a glimpse into the vital role that automation plays in achieving functional test coverage for frontend applications. By embracing automation, development teams can improve software quality, accelerate development cycles, and ultimately deliver a better user experience.

This Blog will cover the below features:

• Overall Test Coverage for UI Applications
• Per Test Case Coverage for UI Applications

Core Idea:

“Chrome Coverage” typically denotes a functionality found within the Google Chrome web browser. This feature empowers developers to gauge and scrutinise code coverage for JavaScript on web pages through the Chrome Dev Tools Protocol (CDP). Code coverage serves as a metric to evaluate the extent to which your codebase is executed when a specific software segment is executed. Once the test has been carried out and concluded, you obtain the raw coverage data, often referred to as Manifest Code.

After acquiring the manifest code, we employ the v8toIstanbul Plugin to transform the raw coverage into Istanbul coverage, using source js and map files. Istanbul coverage includes code coverage information and supplementary details such as methods, statements, branches, and corresponding map information, which specifies the starting and ending lines. It encompasses a list of JavaScript classes considered, alongside recorded coverage particulars. This method leverages dedicated sessions to capture code coverage on a per-test basis and overall test coverage.

To capture per-test code coverage, we utilise hooks or annotations integrated into the test automation framework. These hooks/ annotations create CDP sessions for each test executed, capturing code coverage per test, converting raw coverage into formatted coverage, organising, and subsequently storing this information. This stored data becomes a valuable resource for generating recommendations regarding which tests should be executed when there are code changes.

What is V8/Chrome Coverage?

V8/Chrome Coverage typically refers to a feature provided by the V8 JavaScript engine used in the Google Chrome web browser and other Chromium-based browsers. It is a tool that helps you analyze and measure code coverage in your JavaScript applications. Code coverage is a metric that indicates how much of your code is executed during a particular test or while interacting with your application. When you enable code coverage in the developer tools of your browser, V8 (the JavaScript engine) instruments your JavaScript code. This means it adds extra code to track which lines of code are executed and which ones are not. You then interact with your web application or run your tests while the instrumented code is being executed. As your code runs, V8 keeps track of which lines are executed.

In the Chrome → Press the Control + Shift + P in Windows and Command + Shift + P in Mac Os. to Open the Run Command Option.

Search for Start Instrumenting coverage and reload page and select that option. Once it is selected it will display the coverage for each files with unused bytes information and also it will highlight the covered and non covered information.

How to Capture the Test Coverage using Selenium with Java?

Selenium DevTools access refers to the capability of using Selenium to interact with a web browser’s Developer Tools, which is a set of features for debugging and optimizing web applications. With Selenium’s DevTools support, developers and testers can inspect, analyze, and manipulate elements within a web page, allowing for more in-depth testing and debugging of web applications. This functionality is valuable for tasks such as performance profiling, network analysis, capturing coverage and automating browser-related debugging, making it an essential feature for those working on web development and quality assurance.

Minimum Required Selenium Version for Support

<dependency>
  <groupId>org.seleniumhq.selenium</groupId>
  <artifactId>selenium-java</artifactId>
  <version>4.11.0</version>
</dependency>

In the before Annotation call the setup function to initiate Profiler and start the dev tools for each test case.

This code demonstrates the usage of Selenium’s DevTools capabilities for web profiling and performance analysis. It starts by acquiring access to the DevTools using driver.getDevTools(). It then enables the Profiler domain and creates a session using the Dev Tools Protocol.

The subsequent lines show the activation of Precise Code Coverage profiling. This specific type of profiling helps measure how much of your JavaScript code is executed, providing insights into your code’s efficiency and performance.

Lastly, the code navigates to the Facebook homepage, where the profiling and coverage analysis can be applied to the JavaScript code running on that page. This usage of DevTools is valuable for identifying bottlenecks and optimising web applications for better performance and also it will help to increase the test coverage.

@Before
public void setup(){
  ChromeOptions chromeOptions = new ChromeOptions();
  // Launch the Chrome browser
  ChromeDriver driver = new ChromeDriver(chromeOptions);
  // Get the DevTools instance
  DevTools devTools = driver.getDevTools();
  // Enable the Profiler domain and Create Session using Dev Tools Protocol
  devTools.createSession();
  devTools.send(Profiler.enable());
  // Start the Precise Coverage
  devTools.send(Profiler.startPreciseCoverage(Optional.of(true), Optional.of(true),Optional.of(false)));
  // Navigate to the page containing the JavaScript code you want to profile
  driver.navigate().to("https://facebook.com/home");   
}

In this code, the DevTools Profiler is used to gather precise code coverage data from a web page.

1. Profiler.takePreciseCoverage() is called to capture code coverage information, and the results are stored in the coverageResponse variable.
2. After obtaining the coverage data, further processing can be performed to analyze the code execution and identify any unused or redundant JavaScript code.
3. It mentions the possibility of calling a Node.js script, suggesting that the coverage data can be exported or analyzed externally.
4. Subsequently, the code instructs DevTools to stop collecting precise code coverage data using Profiler.stopPreciseCoverage().

This sequence of actions is useful for profiling and optimising JavaScript code on a web page, enabling developers to identify and eliminate inefficiencies and improve the performance of web applications and also it will help to increase the test coverage.

In the after Annotation call the tear down function to get the coverage response for each test case.

@After
public void teardown(){
  // Retrieve the coverage data
  Profiler.TakePreciseCoverageResponse coverageResponse = devTools.send(Profiler.takePreciseCoverage());
  // Process the coverage data
  // Stop coverage collection
  devTools.send(Profiler.stopPreciseCoverage());
  // Close the browser
  driver.quit();
}

Once we get the coverage information, call the node js script to convert the Raw Coverage into an Instanbul Coverage Information.

How to Capture the Test Coverage using Playwright with Java?

Playwright CDP (Chrome DevTools Protocol) is a powerful feature provided by the Playwright automation framework that allows developers and testers to interact with Chromium-based browsers at a low level. It enables fine-grained control and inspection of web pages and browser behavior, making it a valuable tool for tasks like web testing, web scraping, and web automation.

Here are a few key points about Playwright CDP:

1. Low-Level Control: With Playwright CDP, you can send and receive messages directly to and from the browser. This level of control enables you to access advanced browser features, including network interception, manipulating the DOM, and emulating user interactions.
2. Network Monitoring: You can monitor network requests, intercept and modify them in real-time. This is particularly useful for tasks like capturing network traffic, mocking API responses, or ensuring specific network conditions during testing.
3. Performance Analysis: Playwright CDP allows you to analyze web page performance by accessing detailed information about rendering, resource loading times, and more. This is essential for optimizing web applications.
4. Access to DevTools Features: You can leverage various features provided by the Chrome DevTools, such as timeline recording, console messages, and memory snapshots, to diagnose and troubleshoot web page issues.
5. Automation and Testing: CDP can be used to automate complex user interactions, like filling out forms, navigating through web pages, and interacting with JavaScript-driven applications. This makes Playwright CDP a versatile tool for web testing and automated testing scenarios.
6. Integration with Playwright: Playwright CDP is seamlessly integrated into the Playwright framework, making it easy to use alongside Playwright’s higher-level automation APIs. This allows you to combine the power of CDP with Playwright’s simplified, cross-browser automation capabilities.

In summary, Playwright CDP provides a deep level of control and insight into Chromium-based browsers, making it an invaluable tool for web developers, testers, and automation engineers who need precise control and monitoring of browser interactions.

Minimum Required Playwright Version for Support

<dependency>
    <groupId>com.microsoft.playwright</groupId>
    <artifactId>playwright</artifactId>
    <version>1.38.0</version>
</dependency>

1. cdpSession = page.context().newCDPSession(page);: This line creates a new Chrome DevTools Protocol (CDP) session associated with the current page. CDP provides advanced debugging and profiling capabilities for Chromium-based browsers.
2. JsonObject params = new JsonObject();: Here, a JSON object named params is created to specify configuration options for the Profiler.
3. params.addProperty("callCount", true);: This sets the "callCount" property in the params object to true, which likely indicates a desire to count function calls in the profiling data.
4. params.addProperty("detailed", true);: This sets the "detailed" property in params to true, indicating a preference for collecting detailed profiling information.
5. params.addProperty("allowTriggeredUpdates", false);: The "allowTriggeredUpdates" property is set to false in params, suggesting a preference to disable triggered updates during profiling.
6. cdpSession.send("Profiler.enable", (JsonObject)null);: This sends a CDP command to enable the Profiler, indicating that profiling should be turned on in the browser.
7. cdpSession.send("Profiler.start", (JsonObject)null);: This CDP command is used to start profiling, which generally involves tracking and recording the execution of JavaScript code on the web page.
8. cdpSession.send("Profiler.startPreciseCoverage", params);: Finally, this line sends a CDP command to start collecting precise code coverage data based on the configuration specified in the params object. Code coverage data can be valuable for understanding how much of the JavaScript code is executed during page interactions.

In summary, this code sets up a Chromium browser, navigates to Facebook, and enables the Profiler with precise code coverage using Playwright and the Chrome DevTools Protocol. It is useful for capturing detailed profiling information and code coverage metrics during web automation or testing scenarios.

In the before Method Annotation call the setup function to initiate the cdp session and start the profiler and coverage for each test case.

private Browser browser;
private BrowserContext context;
private static CDPSession cdpSession;
private Page page;
 
@BeforeMethod
public void setup(){   
  Playwright.create().chromium().launch(new BrowserType.LaunchOptions().setDevtools(true).setHeadless(false));
  page = browser.newPage();
  page.navigate("https://facebook.com/");
  //This line creates a new Chrome DevTools Protocol (CDP) session associated with the current page. CDP provides advanced debugging and profiling            capabilities for Chromium-based browsers
  cdpSession = page.context().newCDPSession(page);
  //Here, a JSON object named params is created to specify configuration options for the Profiler.
  JsonObject params = new JsonObject();
  //This sets the "callCount" property in the params object to true, which likely indicates a desire to count function calls in the profiling data.
  params.addProperty("callCount", true);
  //This sets the "detailed" property in params to true, indicating a preference for collecting detailed profiling information.
  params.addProperty("detailed", true);
  //The "allowTriggeredUpdates" property is set to false in params, suggesting a preference to disable triggered updates during profiling.
  params.addProperty("allowTriggeredUpdates", false);
  cdpSession = page.context().newCDPSession(page);
  cdpSession.send("Profiler.enable", (JsonObject)null);
  cdpSession.send("Profiler.start", (JsonObject)null);
  cdpSession.send("Profiler.startPreciseCoverage", params);
}

1. JsonObject coverage = cdpSession.send("Profiler.takePreciseCoverage", (JsonObject)null);: This line sends a CDP command to the cdpSession (which is a CDP session associated with the web page) to capture precise code coverage data. The result of this command is stored in a JsonObject variable named coverage. This JSON object likely contains information about which parts of the JavaScript code were executed on the web page.
2. Gson gson = new Gson();: Here, an instance of the Gson library is created. Gson is commonly used for converting Java objects to JSON and vice versa, making it suitable for working with JSON data.
3. String coverageData = gson.toJson(coverage).toString();: This line uses Gson to convert the coverage JSON object into a JSON-formatted string. The resulting string, coverageData, contains the code coverage data in a JSON representation.

In the after Annotation call the tear down function to get the coverage response for each test case.

@AfterMethod
public void teardown(){
  JsonObject coverage = cdpSession.send("Profiler.takePreciseCoverage", (JsonObject)null);
  Gson gson = new Gson();
  String coverageData = gson.toJson(coverage).toString();
}

How the Profiler.takePreciseCoverage Output looks like it will return JSON Object where it contains array of Script Coverage and timestamp.

Script Coverage contains Runtime Script Id, Javascript Url or name and Functional Coverage

Functional Coverage Contains the Function Name, Functional Coverage Range and Block Coverage Boolean Flag.

Coverage Range contains the Start Offset — JavaScript script source offset for the range start , EndOffset — JavaScript script source offset for the range end and Count it denotes number of times function is called.

Below is the documentation for Chrome DevTools Protocol for taking the Coverage from V8 Engine → https://developer.chrome.com/docs/devtools/coverage/#:~:text=The%20Coverage%20tab%20in%20Chrome,Analyzing%20code%20coverage.

Example Coverage Information from Profiler.takePreciseCoverage

In the code snippet provided, a key operation involves the utilization of the Chrome DevTools Protocol (CDP) to meticulously record code coverage details from a web page. Subsequently, this code converts this acquired code coverage data into a structured JSON format for further analysis and manipulation.

Expanding on the initial context, it is important to note that the “coverage” variable holds the result of the “Profiler.takePreciseCoverage” CDP command. This result comprises valuable information regarding which segments of the JavaScript code on the web page were executed. Such code coverage data can be instrumental in assessing the performance and efficiency of the web application.

To elaborate further, the “Gson” library is employed to facilitate the conversion process. Gson is a commonly used Java library designed to seamlessly transform Java objects into JSON format and vice versa. In this case, it is being used to transform the “coverage” JSON object into a JSON-formatted string.

Regarding the rephrased sentence:

“In the screenshot above, ‘envFlush’ is identified as a function located at the start offset of 0 and ending at offset 151. It is executed once (‘count of 1’) and is contained within the homepage of the web application. This information is pivotal in determining the total number of functions within each JavaScript file, as it allows for the summation of function counts across all JavaScript files. By aggregating these counts, one can calculate both the overall code coverage and the coverage specific to individual test cases.”

Conclusion:

In the rapidly evolving landscape of software development, frontend applications have emerged as pivotal components, given their direct influence on user experience. The quest for reliability and functionality in these applications is of utmost importance to provide users with a seamless and satisfying interaction. This overview underscores the critical role of automation in attaining comprehensive functional test coverage for frontend applications.

Comprehensive functional test coverage stands as the linchpin in assuring that all essential features and user interactions undergo thorough scrutiny. It is the cornerstone of delivering a frontend application that not only looks appealing but also functions impeccably. In essence, this synopsis sheds light on the indispensable role automation plays in achieving functional test coverage for frontend applications.

By embracing automation, development teams can reap a multitude of benefits. First and foremost, it enhances software quality by systematically evaluating the functionality of the frontend application, thus minimizing the risk of defects and issues that could disrupt the user experience

Ultimately, the adoption of automation is more than just a practical solution; it is a path towards enhancing user satisfaction. The convergence of comprehensive functional test coverage and automation empowers software developers to not only meet but exceed user expectations. In doing so, it ensures that frontend applications are not merely functional but delightful, offering a superior user experience.

Sample Source code can be found at https://github.com/Suresh81484/frontendCodeCoverage/tree/master