Pragmatic Front-End Testing Strategies (3)

This article is the last of a series of three articles.
Part 1: Necessity of Pragmatic Front-End Testing Strategies
Part 2: Testing Visual Components Using Storybook
Part 3: Testing Logic in State Management Using Cypress

In part two of this series, we’ve dealt with using Storybook to automate portions of the visual testing. To refresh our memory, let’s recapitulate our Todo application’s processing stages.

  1. Display the default UI when the application executed.
  2. Retrieve the “Todo List” from the API server, and save it in the Redux store.
  3. Display the Todo List on the UI according to the value saved in the store.
  4. The user clicks on the input box, types in “Take a Nap,” and hits enter.
  5. Add “Take a Nap” to the Todo List in Redux store.
  6. Update the UI according to the new store value.
  7. Transmit the new store state over the server to synchronize.

These stages can mainly be categorized into two based on the “state of the application”. First, the stages 1,3, and 6 display the current state of the application to the UI. In part two of this series, we discussed why complete automation of such stages is difficult, and how we can use Storybook to facilitate the process.

Now, the remaining 2,4,5, and 7 compose the manipulation stages of the application’s state. Such stages can, again, be categorized into manipulating the state with the user input (4,5) and synchronizing the state of the client and the state of the server (2,7) The part three of this series will explore the traditional testing methods for such stages, and how Cypress fares against the traditional methods.

(Every code I wrote for this series can be found in my Github Repository. While the article will include all of codes that I think are critical for understanding, you can check out the repository if you are curious about the entire code.)

Writing Modular Level Tests

React applications that use Redux often consist of action creators, reducers, container components, and the presentational components. Also, if the project calls for additional functionalities like local storage or network IO, middleware codes to implement said items are included as well. For the purpose of this article, Redux-Thunk will be used to deal with the asynchronous communication between the server, so codes that deal with such additional features can be handled within the action creator.

Each module has its own unique role within the application, and are usually well defined and separated into functions or classes, which makes writing modular level tests extremely easy. Writing modular level tests is also explained in great detail in Redux’s Official Tutorial, and even Enzyme, the most widely used testing library for React, recommends following this method.

However, as I hopefully made it clear in the first and the second parts of this series, splitting the tests into levels that are too small indubitably increase the number of mock objects, and makes it harder to test intermodular functionalities. Furthermore, smaller units are much more prone to breakage due to refactoring because of increased dependency on the implementation detail. For example, if you were to write modular level tests to validate “addTodo” feature, you would end up writing four individual tests as below.

(While this article makes use of Jest and Enzyme, does not explain detailed usage of each tool. Most testing frameworks using JavaScript have similar APIs, so I believe that people who have not used Jest should still be able to follow along. For information on using Enzyme, refer to the official API documentation.)

1. Container Component (Header)

Validates that the container component, that connects Header component with the store, successfully connects the action creator addTodo() function to the store and that it passes the addTodo prop to the child component.

import React from 'react';
import Header from '../components/Header';
import configureStore from 'redux-mock-store';
import thunk from 'redux-thunk';
import {shallow} from 'enzyme';
import {configure} from 'enzyme';
import Adapter from 'enzyme-adapter-react-16';
import {addTodo} from '../actions';

configure({adapter: new Adapter()});
jest.mock('../actions', () => ({
addTodo: jest.fn().mockReturnValue({type: 'ADD_TODO'})
}));
const mockStore = configureStore([thunk]);

it('should pass addTodo action to child component', () => {
const store = mockStore({});
const component = shallow(<Header store={store} />).first();
const todoText = 'Hava a Lunch;

component.prop('addTodo')(todoText);

expect(addTodo).toBeCalledWith(todoText);
});

2. Presentational Component (Header)

Validates that changing the value of input element in the Header component and pressing enter actually executes the addTodo() function that is given as a prop.

import React from 'react';
import {configure} from 'enzyme';
import Adapter from 'enzyme-adapter-react-16';
import {shallow} from 'enzyme';
import {Header} from '../components/Header';
configure({adapter: new Adapter()});
it('should dispatch addTodo when input text', () => {
const addTodo = jest.fn();
const wrapper = shallow(<Header addTodo={addTodo} />);
const todoText = 'Have a Lunch;
  const input = wrapper.find('input');
input.simulate('change', {target: {value: todoText}});
input.simulate('keydown', {keyCode: 13});
  expect(addTodo).toBeCalledWith(todoText);
});

3. (Asynchronous) Action Creator

Validates that running the asynchronous action creator addTodo() function that uses Redux-Thunk actually dispatches the ADD_TODO action and sends synchronization request to the server.

import axios from 'axios';
import {addTodo, ADD_TODO} from '../../src/actions';
it('should dispatch ADD_TODO action and update Server data', () => {
jest.spyOn(axios, 'put');
const todos = [{id: 1}];
const getState = () => ({todos});
const dispatch = jest.fn();
  const thunkAction = addTodo('Have Lunch');
thunkAction(dispatch, getState);
  expect(dispatch).toHaveBeenCalledWith({
type: ADD_TODO,
text: 'Have Lunch'
});
expect(axios.put).toHaveBeenCalledWith('/todos', todos);
});

4. Reducer

Validates that the todos reducer actually returns a new array by handling ADD_TODO action and adding to the original array of Todo list.

import {todos} from '../../src/reducers';
import {ADD_TODO} from '../actions';
it('should handle ADD_TODO', () => {
const prevState = [
{
id: 1,
text: 'Have Breakfast',
completed: true
}
];
  const action = {
type: ADD_TODO,
text: 'Have Lunch'
};
  expect(todos(prevState, action)).toEqual([
...prevState,
{
id: 2,
text: 'Have Lunch',
completed: false
}
]);
});

Tests written in such patterns make use of numerous mock objects in order to independently test each module. This increases the number of unnecessary codes, and makes it so that we cannot test for intermodular connectivity. For example, even if we change the name of addTodo function that the container component Header uses to pass a value to its child component, to appendTodo, tests regarding the Header (presentational) component does not fail. Furthermore, since we separated the todos reducer, if we decide to create a root reducer by using combineReducers, we will not be able to make certain that todos reducer was included.

For these reasons, I strongly recommend that you use a congregate of modules to write tests instead of writing tests based on each module. In order to maintain the consistency of vocabulary, I will refer to using a relatively large collection of modules as a testing unit as integration testing.

(Because the vocabularies used to describe different tests are not strictly defined, different people have different references as to what they mean. I recommend that you check out explanations by Martin Fowler on Unit Test and Integration Test.)

Writing Integration Tests

Before we can start writing integration tests, we, first, have to define the border of our testing units. For example, we could test action creators, reducers, and store at once, or we could also just test the store and container component together. Different groupings have different pros and cons, so it is up to the tester to decide it for oneself. In this article, I will group every module except for the main module that creates the store and the router, in order to make clear the differences between unit testing and integration testing.

Since we already tested how state changes in store affect the visual components in part 2, we’re already halfway done! Now, the remaining half has to do with writing test codes to manipulate the store according to the user input. To make it easier to compare integration testing with unit testing, I will again test “addTodo” feature. First, let’s take a look at the finished test code. (I have added annotations to help with understanding.)

(In the code below, I am using the react-testing-library to compose the integration test. Unlike Enzyme, it provides numerous useful APIs to deal with bigger units. I wholeheartedly recommend that you take your time to learn the library’s philosophies and usage by reading the official documentation.)

The code above looks incredibly long considering the fact that it only tests a single feature, but when you break down the code into segments — preparation (1), execution (2), and validation (3) — the intent becomes crystal clear. First, you configure the store’s state in the preparation stage (1–1), mock the server synchronization request (1–2), and render the component (1–3). During the execution stage, provide the input box displayed on the screen with user input, and hit enter (2). Finally, validate the store’s current state (3–1), and validate whether or not the server synchronization request has been made (3–2).

Let’s compare this to the previous unit test. For starters, now that we got rid of the unnecessary mockery, the load of the entire test code became lighter, and the intent of every piece of code became much clearer. Also, because we eliminated the need for dependencies in terms of implementation details, the test will not be affected even if we alter the component’s structure or props. Contrarily, if the props delivered by the parent is not equal to the props the child component is trying to use, the test fails, rendering the intermodular testing moot.

(Testing Implementation Details written by Kent C. Dodds, the author of react-testing-libraries, does an excellent job explaining the disadvantages of testing implementation details.)

Validating the State of the Application on the DOM

We have successfully tested our application, as of the current state of the application, for components that visually display the current state and components that retrieve user input to change the state of the application. Can we now confidently say that we have written every single test necessary for the application? Not quite. We actually have one more test we can automate: the DOM.

“Now hold on a minute. Didn’t someone say that it’s more efficient to visually validate the DOM with your own eyes given how difficult it is to automate tests regarding the DOM?” Well, I said that, and it’s only half true. The reason is that DOM is not just composed of visual elements. To be more specific, the parts that are difficult to automate, the visual elements like layout, color, font, and images, are combinations of the DOM tree and the style information (CSS). If we remove these visual elements of the DOM, text, the order of the DOM, states of certain DOM elements are still logically verifiable, and therefore, automatable.

For example, let’s say that there is a feature that paints our Todo list on the screen according to the state of store’s todos array. Initially using Storybook, I asked that developers validate all parts of the UI using their eyes. However, to be technical, parts of the DOM, including whether the Todo list’s DOM element has been displayed in the correct sequence, or whether the text value of the Todo element matches that of the corresponding element saved in the store, can be automated. If these portions of the testing are automated, we don’t have to worry about data, and can focus purely on the visual aspects when we are testing the application using Storybook.

Let’s get back to the code, to better illustrate my point. The following code is an example of validating the state of the Todo list displayed on the screen using the DOM.

The test code above validates the sequence, length, text, and check-state of the Todo elements. The most important tip to keep in mind when writing such tests is that you should always minimize the test’s dependency on the DOM structure. For example, codes involving “parent of” a certain DOM element or which “tag or class” was used are more deeply rooted with validating visual components. It is pivotal that, for stable testing, you are not noticeably affected by the changes in such properties when traversing the DOM, and you do not use selectors like tag selectors, child selectors, or class selectors.

The react-testing-library suggests that the tests be written based on information that is revealed to the users (mostly text,) and if that is not enough, to use data-testid property instead of classes. For this test, because we have to check that the items are in the correct order, and that we left nothing behind, it can be a little bit difficult test by using only the text. Therefore, I assigned data-testid properties with todo-item values for the purpose of this test.

However, I did use the completed class in order to validate the checked state of each item, and this is because the completed class deals not only with the visual aspect, but also takes on the role of representing the state of the item. While I still recommend that you refrain from using classes for DOM traversal, for cases like this, where you have to validate the state of a class, classes can be effective. If you are a fan of more strict distinction, you can use separate classes for visual presentations and states.

Integration Test (Jest) vs. E2E Test (Cypress)

Now, all tests have ran. I hope that these examples clearly illustrated the advantages integration testing have over unit testing. However, the problem is that I have yet to mention Cypress that I promised to discuss. Truthfully, Jest and react-testing-library are both extremely powerful tools, and when these tools are handled efficiently, you can write effective tests without using Cypress. Then, why would we even need Cypress?

First, Jest is limited by the fact that it runs not on the real browser, but on the virtual browser environment rendered by JSDom. To further illustrate, because we cannot use the browser’s rendering engine, we cannot retrieve pixel information from the actually rendered product, and it is difficult to test that the router is functioning properly because JSDom does not perfectly mock necessary objects like history and location. (7)The reason we had to manually inject the StaticRouter at every turn in the example is due to the fact that we cannot directly use the BrowserRouter. Cypress, on the other hand, runs directly on the browser, so can be executed without such restrictions.

Second, and the most important in my opinion, point is how Cypress makes debugging extremely easy. Jest’s interactive CLI environment is quite powerful in that it provides the users with useful information regarding failed tests. However, it does not allow us to see the problem on the UI. Trying to write or debug a front-end application without being able to see the UI itself can be as painful as programming with a blindfold over your eyes. Especially when you are trying to validate the application’s state using the DOM, as we had to do in the previous code, without access to the UI, we have no other option but to simply console.log() the heck out of the program or stare at the complicated HTML with our eyes watering.

However, luckily for us, Cypress executes on the browser, so we actually have access to what the product looks like in the UI when we are writing or debugging. There’s more! Because the states of the application at every point of the test are recorded on the command log, debugging with Cypress is just as easy as rewinding a video tape. Furthermore, because we also have access to the browser’s developer tools, we can debug in a much more interactive environment than console logging.

These are just few features Cypress offers, and there are much more functionalities like API to simulate the user input so that we don’t have to actually trigger an event in the DOM, which makes testing a lot easier. Also, Cypress provides an API to mock server data to enable developers to test server-data-interaction without the restriction of different libraries.

E2E Testing and Cypress

As I illustrated above, Cypress provides a more thorough testing environment compared to the original integration testing based on Jest. However, before we get into using Cypress, let’s first discuss the difference between a traditional E2E test and Cypress.

E2E testing, short for End-to-End testing, is a test that tests the entire system from the user’s perspective. Traditionally, E2E testing meant using the web browser to run tests on every single parts of the system, and Selenium WebDriver was the most widely used tool for such tests. However, Selenium WebDriver, while ground breaking, was difficult to configure and to write test codes, and was also extremely slow. For these reasons, it was often only used by organizations that specialized in QA.

However, the creators of Cypress built Cypress for a different reason from that of original E2E testing tools. Cypress was built to be used by front-end developers during the development cycle. Since prompt feedback is absolutely critical during the development cycle, Cypress, with structures directly integrated onto the browser, provided the speed that simply could not be matched by Selenium WebDriver. Also, Cypress recommends developers to mock the backend API rather than testing the entire system, and provides necessary mocking functionalities to go with it. By using the aforementioned command log feature, you can use Cypress to create your project without the help of other IDEs, and dare I say that Cypress is a simply a more progressive TDD development environment.

(To be completely honest, E2E test with mocked backend is more of an integration test than an E2E test. However, as I mentioned earlier, the field of testing is not set in stone, but is more flexibly defined. Furthermore, while Cypress is mainly advertised for integration testing with mocked backend, it can also be used for purely traditional E2E tests. Therefore, for the duration of this article, I will categorize it as an E2E testing tool)

Other key characteristics are thoroughly explained in the official documentation, and differences between Cypress and traditional E2E testing tools, as well as other tradeoffs are also explained in detail. I wholeheartedly recommend that you take a look at it.

Starting Cypress

Cypress can be installed easily using npm.

$ npm install cypress --save-dev

When the installation finishes, you can run the next command without any other configuration.

$ npx cypress open

When you run Cypress for the first time, a cypress folder will be created in the project folder, and included are numerous sample files for users who are new to Cypress. For this article, let’s delete all of the sample files, and let’s get to writing our first Cypress test.

First, let’s create a todo.spec.js file in the cypress/integration folder to start our simple test. Most of APIs in Cypress are based on Mocha and Chai, and provides APIs consisting of the intuitive BDD style so that novice users can easily become acclimated.

it("true is true", () => {
expect(true).to.equal(true);
});

When the file is created, you will be able to see that the file has automatically been included in the Cypress test runner. Clicking on the file will open the Chrome browser modified with the Cypress extension, and you will be able to visually validate the test results.

Writing Cypress Tests

Now let’s get to writing actual tests. Cypress tests are often ran by running a separate local server, and directly connecting to the corresponding URL. In this example, we are using not only the development server, but also the API server, so both must be up and running before we begin testing.

You can connect to the API server on port 8081 by typing node server onto the command line. Next, enter npm start on the command line to run the webpack-dev-server at port 3000, which will connect the API server as a Proxy according to the configuration.

Before we start with actual test codes, let’s add some configurations. If you define the baseURL in the config file, you can use relative directory without having to write out the full URL every time. In order to do so, add baseURL in the cypress.json file in the project root.

{
"baseUrl": "http://localhost:3000"
}

Now, let’s write a code that tests the feature that paints the Todo list on the screen according to the state of the store. In order to mock the server’s response, run cy.server() and cy.route() to define the desired URL and response. Then, to connect to a certain URL, use cy.visit().

One of the advantages in using Cypress is that you can view the process logs and the actual application at the same time. In the image above, command log that contains every command executed to run the test is displayed on the left, and the actual application is displayed on the right. By clicking on each element in the command log, you will be able to see the corresponding view of what the application looked like at that moment. Also, Cypress displays other useful information like which network requests were mocked, and when a particular network request occurred.

Testing the State of the DOM According to the Browser URL

As you have seen in the previous example, in Cypress, using a mock of the server data is much more convenient than actually creating a store in order to manipulate it. The same goes for the router, and instead of injecting a mock router just to manipulate the state of the router every time, you can just change the browser’s URL directly. Let’s improve the previous example by monitoring how the Todo list is filtered with different URLs.

Aside from grouping the common initializing codes with beforeEach() in order to eliminate redundant tasks, and defining an aggregate using describe(), not much has changed. As such, by changing the input of the cy.visit() function to change the target URL, we can easily validate different states of the DOM under different router states.

Adding Todo

Now, let’s test adding an additional Todo element. In order to validate the value of the sync request to the server, we will use object options found in cy.stub() and cy.route(). For detailed explanation on options of cy.route(), refer to the official API documentation.

I have annotated the code using the same numbering system that was used to annotate integration testing example. In the preparation stage (1), we can now directly mock the network requests without being bound by the axios library, and we can also connect directly to the server URL without having to render. In the execution stage (2), we no longer have to emit change and keydown events, and can simply write our inputs like an actual user is typing by using cy.type(). Lastly, during the validation stage (3), we are validating the state of the application using the state of the DOM, instead of directly validating the value of the store.

Writing Balanced E2E Tests

So far, we started from testing units, and continually broadened the scope of the test to integration tests and, eventually, E2E tests. As the testing scope widens, the necessity of unnecessary mocks decreases, and the test coverage goes up. While many mistakenly think that unit tests have simpler structures and are easier to write, E2E are often much simpler due to the lack of mocking codes. Furthermore, since E2E tests are barely affected by the state of the implementation detail, as long as the functionality itself remains the same, the test will still pass even if we change the inner codes entirely. Therefore, with well written E2E tests, we can be more trusting and bolder with our refactoring.

However, this does not mean that all tests should be written in E2E form, without mocks. Given certain situations, you may be required to verify the value of the inner store or to mock the module in charge of actual communication in order to control particular network requests (WebSockets, etc.) There are some cases where it is much more efficient and reasonable to test certain components of the application instead of testing the entire UI. Also, for scenarios that involve complex operations therefore requiring numerous complex inputs, unit testing is much more effective.

Luckily, Cypress also offers ways to conduct unit testing and integration testing. Actually, if you decide to import certain modules instead of using cy.visit(), you can write unit tests that are similar to tests written using Jest. I recommend that you take your time to read articles like Sliding Down the Testing Pyramid and Testing Redux Store in Cypress’s blog, for they include comprehensive discussions on such testing methods.

(While you can write unit tests using Cypress, it cannot be said that Cypress officially supports unit testing. This topic is still hotly debated on this Github issue)

Storybook and Cypress: The Grey Areas

Now, we are truly done. To recapitulate, the strategy I recommend is as follows. Use Storybook to test the components that are in charge of visually presenting the current state of the application, and use Cypress to test the components that manipulate current state of the application with user inputs, server data, and so on. When I put it like that, it may come across as I am drawing a hard line to completely distinguish the two, but there actually is a grey area that overlaps. Storybook can, to some degree, be used to validate user actions, and Cypress can be used to validate visual aspects of the application. While I personally believe that it is better to treat them separately, I will briefly elaborate for those who feel threatened by the idea of having to deal with two testing tools.

First, let’s look at Storybook. The Storybook’s official guide document includes interaction testing. As the article explains, using Storybook’s Specs addon, you can write tests using Jest and Mocha APIs in individual stories. Also, the Actions addon allows users to monitor which actions were triggered by the user input in the component, so it can be used to validate simple cases involving user inputs.

Secondly, since Cypress itself is a visual tool, you could test the visual components without the Storybook. However, because Cypress has yet to implement Storybook’s characteristic overview where every scenario is clearly presented, for comparing the products manually, it is much more difficult than using Storybook. It is true that you could efficiently use the screenshots to conduct regression tests, but since Cypress does not provide a feature that allows image comparison, you would either have to implement it yourself or use plugins like cypress-image-snapshot.

(Visual testing tools I mentioned in the second part of this series are compatible not only with Storybook, but also with Cypress. If you are interested, refer to the official documentations of Applitools and Percy.)

Closing Remarks

Throughout this series of three articles, I have shared my thoughts on front-end testing strategies. I have discussed numerous topics like qualities of a good test, automating visual tests, Storybook tests, unit testing, integration testing, E2E testing, and Cypress, and I sincerely hope that you have found the contents of this series useful.

In normal depiction of the testing pyramids, they often suggest that the tests be conducted in the order of “Unit > Integration > E2E.” The intent behind it is that you should mainly utilize unit testing with results of integration and E2E testing as supporting arguments. However, I prefer the complete opposite approach — mainly utilize the E2E test, while using unit and integration tests to support your cases. Also, they recommend that you ignore the visual component of testing when discussing automation, and simply use the Storybook to validate the results using your own eyes alone.

Front-end codes do not simply deal with data, but deals also with display that will be shown to the users. Such codes demand the responsibility of testing strategies that is different from regular testing methods. By mindlessly following the well-known convention of testing of trying to test purely from the data perspective with unit and integration testing, it could lead to slowing down the development process while lowering the quality of the code at the same time.

Amazing tools like Storybook and Cypress have opened the gates of a brave new world for strategic front-end testing. During the short period of time of having existed, they are greatly changing the way developers test the front-end codes, and I believe that there is still much more we can expect from them. While I acknowledge that the methods discussed in this guide may not be suitable for everyone, I sincerely hope that I have inspired developers to strive for better testing strategies and to attempt the new with different tools.