Tagging tests w/ Protractor and Mocha

For the last several months, I’ve been itching for an opportunity to do more work in the area of automated UI testing. I recently had the opportunity during my 9–5 to begin building out a suite of UI tests to be used for validation of a 3rd party Enterprise Application Suite that we frequently write integrations for on behalf of our clients. The long-term goal of the test suite is to allow developers to feel confident their new integrations do not break existing functionality in the application.

After trying out a few UI testing frameworks(Nightwatch, vanilla Selenium WebDriver), I eventually made the decision to build our testing suite on top of the Protractor framework. Although Protractor was initially built with the goal of testing Angular applications, it lends itself very well to testing applications authored using other Front End frameworks. There were several factors that led to my decision to use Protractor, but those are outside the scope of this post.

The Problem

Getting started with Protractor was quick and painless. However, as the number of tests in my suite began to grow, I found that I frequently had the desire to run only a small subset of my tests at any given time. Some of the use cases for this were:

  1. Being able to execute tests for one page/feature while it is being developed locally.
  2. The ability to exclude slow-running tests during local development.
  3. The ability to exclude tests for features that may be disabled for clients in the 3rd party application.

After doing some quick Googling, I discovered that Protractor provides the option for you to run a limited number of suites. This feature allows you to use globbing patterns to segment your tests into different named suites. You can then use the suite parameter on the command line to pass a comma-delimited list of suites you wish to run. However, I discovered that this feature imposed certain limitations:

  1. It limits you to only being able to identify tests based on the information you can infer from the file system. This limits how creative you can be with segmenting tests into different groupings.
  2. It requires updating the globbing patterns in the configuration file if you decide to change the folder structure or naming conventions in your project in the future.
  3. You cannot tag a single test case within a file (ex: slow). Instead, you have to refactor the test out to a separate file, which will likely require refactoring your beforeEach hook into a separate file if you want to limit repetition.
  4. It moves the metadata about a test or tests into a configuration file, instead of associating it directly with the test(s). This makes it less likely that a future developer on the project will discoverer this information on their own, without explicit documentation. Documentation can quickly go stale, so it would be nice if we had a means of declaratively associating this info within the tests themselves.

The Solution

Although I personally found the suites functionality to be limiting, the problem can still be solved thanks to the flexibility of Protractor!

Protractor provides bindings to 4 different BDD testing frameworks. In my particular case, I was already using the Mocha testing framework, which I found has support for tagging in the form of their grep option. Armed with this information, it only took a small bit of glue code to achieve tagging functionality that I was happy with.

The Implementation

At a high-level, the steps to implement tagging with Protractor/Mocha are:

  1. Decide on a convention/syntax for how tags should be expressed through-out the tests (ex: #tagname or @tagname).
  2. “Tag” any tests by adding the tag within the description of a describe or it call.
  3. In the Protractor config file, within mochaOpts, use the grep property to assign a regular expression (either as a string, or a literal) that matches any of the tags associated with tests that should be run (ex: #tag1|#tag2). This can be overridden at run time by passing a value for mochaOpts.grep on the command line.
  4. Run the tests in Protractor, and Mocha will now use the grep option to limit your tests to only those with tags defined in the grep option.

This is a great start, but there are a few problems with this basic implementation:

  1. It doesn’t enforce a common convention/syntax. Developers may have different preferences, which will inevitably lead to different team members/contributors using their own syntax and regular expressions. This will lead to an inconsistent/unpredictable tagging API.
  2. It’s awkward for a tagging system to use an option named mochaOpts.grep, instead of something more straightforward like tags, which is a bit more clear about it’s intentions for this use case.

Luckily, the Protractor configuration file is just a CommonJS module that runs in Node.js. This means that we can pass custom parameters when running Protractor from the command line, and then parse them and dynamically set configuration values.

The final implementation I’m using now looks something like this bare-bones example:

With this implementation in place, I can now simply pass the tags option to Protractor on the command line:

protractor --tags feature1,slow

Conclusion

With a very small bit of effort, it was possible to create a declarative tagging feature for our UI testing suite, simply by composing together the features of 2 extensible tools (Mocha’s grep, and Protractor’s module-based config file and bindings to Mocha).