How I Filtered Tests with Cypress.io

Learn how to filter tests by tag using the Cypress testing framework.

Photo by Wade Austin Ellis on Unsplash

A common practice in the field of test automation is to filter runs based on a set of criteria, whether it be to solely run UI tests or those for an API. Test filtering can be helpful in trimming down runs so that they are scoped properly to the work at hand.

Cypress does not offer the ability to filter test runs natively, an issue that has been ongoing since 2018.¹ Instead, engineers must write their own filtering methods to be used with Cypress.

I came across this issue while working for a start-up in Raleigh, North Carolina. The shop had hundreds of tests written in both Ruby and Python, but nothing written in JavaScript. Having a wealth of experience with Python and Pytest, I chose to emulate what I loved most about the framework, marks.

Pytest marks are similar to tags in that they allow for test scoping during full runs. Instead of running the entire suite, I could choose a mark (or more if I felt inclined) to run against. Pytest would automatically select all the tests with a specific mark and load them. All other tests would be deselected and therefore, not processed within the test run.

In this tutorial, I will teach you how to build your own filter for Cypress tests that will load based on a test tag (or tags).

Getting Started

With any new JavaScript project, we must initialize using NPM. Once we have a package.json, now we can begin loading Cypress.

npm install --save-dev cypress

Once Cypress has been installed, run it with NPM so it may finish the installation.

npx cypress open

Cypress is finished installing when a cypress directory has been created. You can additionally verify the proper installation by running npx cypress --version.

Building a Better Filter

A quick Google search of “cypress test tags” will return two great results:

1. “Using Tags to Filter your Cypress Tests” by Marie Drake²
2. “How to apply Tags to your Cypress Tests” by TestersDock³

The first article is what I based my solution on. The second article uses a GREP solution which I was less fond of. I actually ended up working for a company that used the GREP solution proposed by TestersDock with mixed results. We found that GREP would pull in tests that did not belong simply because a word was written in a test description. For example, a test containing the word “API” would be pulled into the API suite.

I ultimately chose to remove the GREP solution in favor of modifying Marie Drake’s original proposal.

I began by building a filterTests.js module into my support directory with a function, filterTests.

The beginning of our filterTests method.

Following Marie Drake’s outline, I added two parameters — definedTags and runTest. The definedTags parameter is an array of strings. runTest is a function that contains all tests within a module. From here, I create a Cypress environment variable named tags. I chose to use tags instead of Marie Drake’s “TEST_TAGS” as Cypress environment variables are lower case rather than all caps and snake case.

The addition of our environment variable, “tags”.

I can now open Cypress with a tags environment variable by running the following:

npx cypress open --env tags=api

We need a conditional to check whether tags are present in order to take action against our tags similar to Marie Drake’s implementation. The main difference being that instead of splitting on a comma, we would rather split on slash. Once found, call the runTest method to run the test.

Conditional code for if “tags” is true.

Running Tests without Tags

The filter above (and the filter that Marie Drake proposed) will work in all instances, save those where environment tags have not been provided. Attempting to run tests using npx cypress open will fail as we do not have a second conditional that runs tests by default. This is a fault as there will be plenty of occasions where the full suite will need to be run.

Simply adding an else clause with runTest() inside will do.

Finished Product

You can copy the gist below and paste it into a filterTests.js module within your support directory. Continue reading to learn how to use the filter within a test spec.

Our finished filtering method.

Filtering a Test

We can use our newly built filter by importing it into a test file, then adding a method call prior to the first describe block. This way, all tests within the spec file will be filtered.

An incomplete use of the filterTests method.

Since we are automating the UI using Cypress, we can add two tags to the filterTests method, “all” and “UI”. I have written out two tests demonstrating the filter in a gist below.

A properly filtered series of tests.

We can test our filtering method by launching Cypress UI with the following:

npx cypress open --env tags=api

Select elements.spec.js and run.

The result of using an “api” tag on a UI repository.

The result is that no tests have been found as the tag api does not match any of the definedTags in the filterTests method. In order to run tests within the spec file, we must include either “all” or “ui”.

A properly filtered run using the “ui” tag.

Running Tests using Cypress Module API

Tests may be filtered through the Cypress Module API by creating a JavaScript file containing Cypress run parameters. As an example, we will create a module which runs all tests within the Elements page module.

Once created, you may run the following in your terminal:

node .\run-elements-tests.js

This will kick off an automation run with the environment tag ‘elements’ which should not load any modules which do not contain the tag.

An unloaded test module.

Skipped tests will show zeroes for all areas as no tests loaded within the Cypress runner.

A loaded test module.

Loaded test modules will run as expected.

Project Directory

Your directory should resemble the following upon completion of this tutorial:

cypress
    |__ integration
        |__ elements.spec.js
    |__ plugins
        |__ index.js
    |__ support
        |__ filterTests.js
        |__ index.js
.gitignore
cypress.json
package-lock.json
package.json
README.md

Summary

Filtering tests with Cypress is easy once you build a custom method to do so. Using environment variables, we can pass test tags to Cypress, then filter out tests that are out of scope for our run. Should we choose not to run with tags, we can effectively run our repository without issue.

Resources

1. “Filter or Grep Specific Tests to Run (like Mocha Grep Pattern) · Issue #1865 · Cypress-Io/Cypress.” GitHub, hannah06, github.com/cypress-io/cypress/issues/1865.
2. Drake, Marie. “Using Tags to Filter Your Cypress Tests.” Marie Drake, Marie Drake, 15 Dec. 2019, www.mariedrake.com/post/using-tags-to-filter-your-cypress-tests.
3. “How to Apply Tags to Your Cypress Tests like Smoke, E2E.” TestersDock, 29 Oct. 2020, testersdock.com/cypress-test-tags/.

Jonathan Thompson is a Senior Quality Engineer at Pendo.io specializing in test automation. He currently resides in Raleigh, NC with his wife and a Goldendoodle named Winston. You can connect with him on LinkedIn, or follow him on either Twitter or Github.