Courtesy Pixabay:

Primer on Front End Automation

Nidhi Sadanand
Jul 27, 2018 · 6 min read

Automation Testing is a specialized job that is done with QE engineers who are familiar with the frameworks involved. In my team this year we have an ambitious goal of having developers write these automation tests as part of the development process thereby making test coverage (both unit and automation) a metric of the code quality.

As developer on one of the FrontEnd teams in Walmart and a noob to the automation setup and development process, I am sharing my experience on how I went about setting up the automation code base and executing it. My team works on Walmart’s online Returns experience , our front end responsive web framework uses Electrode a framework open-sourced by Walmart for large scale front end development and the Mobile App experience using React Native. This post is centric around the frameworks that WalmartLabs uses and is primarily intended for a developer getting into automation test development.

First the Frameworks

Selenium — its a Java based server implements the WebDriver protocol (which is a Rest based protocol that receives commands to execute on a browser). The Server spawns out separate processes which launches a browser that has remote debugging enabled over which it issues commands to the browser.

Magellan — this is the test runner which runs the tests in parallel . Its similar to KarmaJS in that running tests is a separate concern than actually defining the test. Magellan works by finding files or tags and spawning up new worker processes that work on these tests. Each file is a new test case that will be executed within a worker. The worker environment is a sandbox for each test. It can work on these in parallel or in serial mode depending on the settings provided.

NightwatchJS — is the BDD framework that abstracts out Seleniums webdriver apis and provides a framework for defining the test. It supports a design pattern called a page object model through which an HTML page or Mobile page can be abstracted out as page-fragments with commands , sections elements and pages .

Appium — is the Selenium equivalent for mobile app testing. Nightwatch uses Appium to interact with the Mobile device. Appium implements Web Driver specifications making it compatible with any client that uses Web driver api.

Shifu — is a mocking framework developed as plugin with HapiJS. It essentially provides an easy way of detecting and configuring mocks for particular URI route. For local or dev end to end testing, you would want to start by mocking all of the webapp’s dependent downstream services. So when you start your automation setup, you also start a mock server which will handle all your downstream api calls and return a mock response. The cool thing about Shifu is that it has a concept of a session, which allows sending different responses for a route per session. You do this by invoking a variant for a session. Variants are different responses for the same route, tagged differently. The shifu-magellan-nightwatch npm module has out of the box integration with Shifu so that this can be easily plugged into your automation setup.

The Page Object Model

  • Commands — these are abstractions that will act on your elements
  • Pages — a page is composition of the elements and commands all of which are available to the page object.

The Actors in our Setup

Entities in an automation

Note on the actors

  1. The Mocked services are optional — you can just as well do automation testing on a live or pre-prod setup
  2. Each Automation tests runs in its own sandbox which starts a Webapp in that process space.
  3. The entire setup can be run in one local machine or can be distributed over the network. You could have tests executing from your local machine talking to remote selenium servers that load up webapps from yet another domain.

Setting it Up

21:39:35 [WARN] [Nightwatch Extra] getEl saw selector .new-order-item-select-container but result length was 26, with 26 of those :visible

Once you have your front end pages all good for automation readiness, you can start by defining a folder structure for keeping your automation tests.We keep this under project_root/test/automation .

Here is sample folder structure that we followed — this is just indicative. As long as the page finds its elements and commands — it should work fine.

--conf (for configuration files)
--lib (for pageobject models)
--mocks (for mocks if applicable)
--scripts (bash/csh scripts for command line)
--tests (where the actual tests are)

Here are the npm dependencies that are needed for this to work (configured under optionalDependencies.

"async": "0.9.2",
"chromedriver": "2.35.0",
"config": "1.30.0",
"dpro": "1.2.0",
"event-stream": "3.3.4",
"jsonlint": "1.6.2",
"nightwatch": "0.9.9",
"phantomjs": "^2.1.7",
"phantomjs-prebuilt": "2.1.16",
"ps-tree": "1.1.0",
"selenium-server": "3.9.1",
"testarmada-magellan": "10.1.1",
"testarmada-magellan-admiral-plugin": "^3.0.0",
"testarmada-magellan-local-executor": "2.0.0",
"testarmada-magellan-nightwatch-plugin": "7.0.0",
"testarmada-nightwatch-extra": "5.0.0",
"testarmada-renv": "4.1.0",
"webpack-dev-server": "1.16.2"

The two most important configuration files are

magellan.json — Describes how magellan should execute. Magellan provides lifecycle hooks from where we can add bootstrapping code to start the Webapp and mocks if necessary — this is defined in the setup_teardown.

magellan.json — snippet

nightwatch.json — Describes how the tests and selenium should interact.


Note on the Nightwatch.json

  1. src folders — indicates the directory where the tests lie
  2. output folders — is for the test run reports
  3. custom_commands_path — indicates the folder where the page object commands are — optional depending on if this was separated out or not
  4. custom_assertions_path — indicates the path for custom assertions if any — optional depending on if they are any custom assertions
  5. page_object_path — directory where the page objects are
  6. globals_path — page where any global variable file are
  7. selenium configuration — selenium server and driver configs

The Test!

Sample Test

Test Inheritance

Debugging Test Failures

Large Scale Testing using SauceLabs

In Conclusion

Walmart Global Tech Blog

We’re powering the next great retail disruption.