Running Cypress tests parallel in GitLab Pipeline

Learn to run e2e tests faster to save both from time and resources.

Photo by Yavuz Koca on Instagram

Okay, now for some reason, you’ve finally chosen to write some cypress tests for your development environment. Then some time, you have realized that your tests are run forever in the pipeline.

Waiting for the e2e tests more than 20 minutes is not the best way to perform this stage of the pipeline and there is a better way which is the parallelization of the job.

RUN SEPARATE TEST FILES

Before diving into the parallelization, let’s have a look at how e2e tests are run.

This test:e2e command runs the all cypress tests that are in the cypress folder. But, if we give spec flag to this command, we can run specific files which is the main aim of the parallelization because each job will run some files, not all of them.

yarn cypress:run —help

For example, if the script in the package.json file is changed with:

Then specified test files can be run with:

yarn test:e2e cypress/integration/example-test-1.spec.js

Only one test file have been found

Also, multiple test files can be run with comma separated argument.

yarn test:e2e example-test-1.spec.js,example-test-2.spec.js,

The real question in here is that how Gitlab will understand to run which test files?

RUN MULTIPLE JOBS ON GITLAB

On .gitlab-ci.yml file, parallel keyword can be added to run multiple jobs for the given stage. In below, this value is set to 7 which means 7 different job will be run for the stage called e2e which runs the e2e tests.

7 different job will be run with this adjustment

If the script is run with this cypress/integration/example-test-1.spec.js argument, then 7 different job will run the same test file which is not the purpose of this article.

SEPARATING THE TEST FILES

Now, new script file will be written and whatever that script file logs to the screen, that test files will be run.

- yarn test:e2e "$(node scripts/cypress-parallel.js)"

In here, cypress-parallel.js file will be run and that script will determine which test files will be run.

NODE_TOTAL variable is the value of 7 because number of jobs is determined with this in .gitlab-ci.yml file.

NODE_INDEX is the index number of the jobs which can have from 1 to 7 for this example.

And TEST_FOLDER includes all of e2e tests. The next step is the get all file names in this folder because it will be needed to pass the file paths.

We define a method called walk which basically gets all the files in given directory.

// walk('./cypress/integration')
[
    'cypress/integration/example-test-1.spec.js', 
    'cypress/integration/example-test-2.spec.js',
    'cypress/integration/example-test-3.spec.js',
]

Finally, it is desired to separate test files into different jobs. To exemplify, if there is 7 jobs and 13 test files then first 6 jobs will run 2 test files and last job will run only 1. For that purpose, we can filter the test files with their index.

If the NODE_INDEX = 3, NODE_TOTAL = 5 and total number of test files is 27, then this method will return 2, 7, 12, 17, 22 and 27. test files.

CONCLUSION

As you might notice, this is not the best way to handle the parallelization. There are better ways to separate these files into the jobs but even with this adjustment, the required time to run the job is decreased to 1.27 minutes from 20 minutes which is quite enough at least for our development. Furthermore, if any test fails on job number 5 for example, only that job will be run again, not the whole tests.