Responsive Testing & Cypress
A scalable, dynamic, and powerful solution for responsive testing with Cypress.
Cypress is an open-source test automation tool for web applications. Although it’s most often compared to Selenium, it’s completely different regarding its foundations and architecture. With its extensive and detailed documentation, plus its simple initial configuration, it makes us forget old headaches when it comes to automating tests. The one thing that sets Cypress apart from other testing tools is that it seems to have everything solved.
The issue of responsive testing is that it is introduced as an unexplored and unexploited area: the documentation examples are more like a demo than a real project.
Something else that caught my attention is that while surfing the web, I only found an article by Suman Kunwar (which I’ll come back to throughout this text) that is much more similar to what I had in mind.
Within this context, I make my contribution: a deeper view focused on applying responsive testing to larger projects.
A few aspects I took into account to analyze the issue
We cannot use the Cypress documentation approach because it suggests that we set the resolution we’ll use in each test (there can be N resolutions within a forEach per test).
I had envisioned several test suites, so it became clear that this approach wasn’t feasible.
If we don’t want to use the if that appears in the previous example, we must always use resolutions with value pairs, or the Cypress presets (which reduce our options).
Would it be practical to have each test executed in all the resolutions every time we run the suite(s)? What if we need to run a regression test in just one resolution? In this respect, I also decided that it would be best to have a default resolution to test with all the time, and then use a set of N resolutions to perform integration tests.
Moreover, userAgents, which are mentioned in the above-cited article but not in the documentation, come into play. As you can see in the article, there are two key aspects:
- The available options are ‘mobile’, ‘tablet’, and ‘desktop’. A resolution (iPhone5) is linked to a single userAgent. The questions are clear: Will we always want to test iPhone5 with that userAgent? Will we always use the resolution of a single device to run mobile tests? Will we have to edit those values every time we need to use another device? The issue lies in the fact that it is not scalable.
- The solution seems rather static: we define the type of device and have only one option. Picturing the natural development of projects, in which definitions often change, it would be difficult to make the solution fit into those changes.
So, how do we manage to have a scalable responsive solution?
For that purpose, we’ll use the Cypress API through a script.
As in everything, this is no more than an alternative. It is not intended to be the ultimate solution, but, after having tried it, I believe it turned into a strong starting point.
The aforementioned article understood that the right path is to have a default configuration in the cypress.json file that modifies some of its settings according to the parameters we establish. This is where we set ourselves apart and expand the solution. We want to have N resolutions (meaning N available devices to emulate) and N userAgents (in other words, N emulated operating systems). Moreover, we want to be able to set iOS apart from Android. We want to be able to provide test runs and regression tests even more specific: iPhone-X, iOS 13.3.1.
Commands
The script will allow us to set <device> and <osVersion> as parameters; and, if necessary, (for example, if we run the script on continuous integration), we can define the <record> parameter, which will, in turn, set the record key stored as an environment variable. It will also allow us to establish the <open> parameter so we can use the Cypress runner.
Hands on!
The script logic allows us to always get to execute the suites. As long as the <device> parameter exists in the devices.js file, the script will wait for us to set the <osVersion> corresponding to that kind of device (iOS & Android). For example, we can set “iPhoneX”.
Once the device has been defined, the script will search within the osVersions defined in the os.js file corresponding to the iOS devices. For example, we can set “13.3.1”.
If the version we entered doesn’t exist in the list, the script will search for the default version of that OS.
node cy-start.js — -d iphoneX -osV 13.3.1 -o
From the Cypress runner, we can see the following environment variables are available: device, osVersion, osType.
We can take advantage of those variables in order to use them for the suite names.
When running the example script, we’ll see that the tests were executed at the specified version and resolution.
Finally, in order to help users, we make the ways in which the script can be executed available, as well as the devices and versions.
Here’s the complete script.
Conclusion
Thanks to this approach, we are now capable of testing functionalities or run regression tests on specific devices and versions, being able to adapt to any necessary combination. It also allows us to reduce daily testing times using versions set by default. Moreover, the script logic allows us to add as many configurations and variations as necessary to improve our tests and adapt them to our testing needs.
