Initial Review of Testing with Cypress

What is Cypress

Cypress is an end-to-end or system level testing framework written in JavaScript. It can be used to test applications with browser based user interfaces (UI). Cypress leverages Mocha’s describe and it functions along with Chai’s expect assertions that makes adoption a breeze for anyone familiar with those tools. For anyone coming from Jest there may be a bit more of a learning curve, but it should not take long before you are comfortable with the testing syntax.

Cypress, similar to other tools like Selenium, allows you to write tests that interact with a browser for tasks such as to filling in form fields, clicking on buttons, etc. Unlike Selenium, it tries to enforce a user centric approach to testing the app. What this means is that if a css selector is used to find and click on a button, but that button is currently hidden or behind another element such as a modal the test will fail since the element would not be available to the user. You can pass an option to force the click, but that is considered a bad practice and should be used sparingly.

Cypress is async by default, which means you will not need to write all those helper functions to force Selenium to wait for elements to appear or a request to come back. From my experience the right sizing of these wait times in Selenium based tools is a fairly common source of flaky tests and those are all but eliminated when using Cypress.

First Impressions

Setting up Cypress is straight forward and well documented. Once you verify and open Cypress it will generate the following directory structure for you:

cypress
├── fixtures
│ └── example.json
├── integration
│ ├── 1-getting-started
│ │ └── todo.spec.js
│ └── 2-advanced-examples
│ ├── actions.spec.js
│ ├── aliasing.spec.js
│ ├── assertions.spec.js
│ ├── connectors.spec.js
│ ├── cookies.spec.js
│ ├── cypress_api.spec.js
│ ├── files.spec.js
│ ├── local_storage.spec.js
│ ├── location.spec.js
│ ├── misc.spec.js
│ ├── navigation.spec.js
│ ├── network_requests.spec.js
│ ├── querying.spec.js
│ ├── spies_stubs_clocks.spec.js
│ ├── traversal.spec.js
│ ├── utilities.spec.js
│ ├── viewport.spec.js
│ ├── waiting.spec.js
│ └── window.spec.js
├── plugins
│ └── index.js
└── support
 ├── commands.js
 └── index.js

Though you are likely going to delete everything in the cypress/integrations
directory, it’s great to see so many examples provided.

The Cypress API is very intuitive and will not take long to adopt. For instance cy.visit(‘https://redsquirrel.com') goes to the provided URL, cy.get(‘.myselector’) locates the element with the provided selector. The get method allows chaining, making it easy to interact with the element you selected.

cy.get(‘.button’).click();

If you are dealing with a text input, you can use:

cy.get(‘input[type=text]’).type(‘the string you wish to enter’);

When it comes to the type method, there are a handful of provided convenience strings that make inputting special characters like right/left arrows or backspace easy.

cy.get(‘input[type=text]’]).type(‘{rightarrow}{backspace}’)

One miss here is the lack of {tab} for these convenience strings, though there is an open issue on their repo to add this down the road.

If you use the run command to execute your tests, Cypress will generate a
video for each test file it executes. This initially seemed like a nice to have feature, but after using it I can’t imagine not having this anymore. The created videos are perfect for attaching to your pull request and/or diagnosing failures that are taking place on CI.

The one area that is a bit more difficult in Cypress than a Selenium based tool like Capybara for Ruby, is that it does not have direct access to the application models (unless you are using node). This can easily be seen as a positive since it forces a separation of testing concerns from application development, but as you start adding features that rely on certain data to be there and then cleaned up after each test, being able leverage testing tools like Factories that you have likely already built for unit testing is in many cases not directly afforded to you in Cypress.

Lessons Learned

Cypress encourages using commands as the primary way to extract duplicate logic that takes place in your tests. The quintessential example of this is user authentication. In many tests the user must first log in to access a feature or to create new resources and instead of copy/pasting this into the beforeEach of every test file, you can create and register a command that does this for you.

// cypress/support/commands.js
Cypress.Commands.add(‘login’, (email, password) => { … });

The command would then be available off of the cy object similar to the get and visit commands.

cy.login();

The logic behind login can be sped up by using the cy.request method. The request command will allow you to make an http request and can be used to bypass the need to load a form the user would normally fill in and instead you can POST directly to the server.

This means you will likely end up with one or more tests that runs through the UI filling in the form as a user would, and subsequent features that need a logged in user will use the command.

When it comes to creating data Cypress recommends three non-exclusive strategies. The first is using fixtures and stubbing interactions with the server. I prefer this approach when testing edge cases like error scenarios such as 500 statuses or when interacting with third party apis. Stubbing the responses in most other scenarios effectively defeats the idea of end-to-end testing.

The second approach is to seed the db prior to test execution or as part of the test start up. So far I have mostly used seeds for creating several different user accounts that have different permissions levels. Using Cypress commands you can then cy.signInAsAdmin() or other role. You can also use seeds for testing pieces of your app that need to display large amounts of data like search results that may need pagination.

The third approach is to generate and destroy the data in the test run. This can be accomplished by using the UI or the cy.request object to post to the server, or even use the cy.exec tool to have execute a specific script that can generate the data. If you set up scripts you may want to design them in a way similar to migrations, where there is an up and a down function to remove items that may have been added.

Our next iteration

End-to-end tests are naturally slower than unit tests since they perform much of the work that is mocked out in unit testing. As we add more Cypress tests our build times are going to get longer and longer. One of the next items for us to work through is leveraging the parallelization tools Cypress offers to keep this time from getting out of control.

I would also like to continue looking for areas we can refactor data creation into tasks leveraging application tasks (rake or management commands) or special end points that can create and destroy objects.

Overall Impressions

I have been pleasantly surprised at how easy it is to get up and running with the testing tool and would recommend exploring Cypress to anyone starting a new project or adding initial end-to-end tests to their existing project. I would however stop short of pushing to migrate to Cypress if there is already an established end-to-end suite in use.