How to test 400 React components without breaking a sweat
Investing in your development experience will pay in dividends.
Your organization’s development experience can be detrimental to your ability to ship code to your clients.
There is a very delicate balance that needs to be found between testing your code and maintaining a good development experience.
Finding that balance required a paradigm shift in the way we do tests at Kenshoo, which allowed us to reduce our CI/CD process by 80% while maintaining excellent test coverage.
A few years ago, we decided to take the UI monorepo architecture at Kenshoo to the extreme.
Being able to release consistent and high-quality UI features spread across multiple products and some 20 teams proved very hard.
Initially, we had a components library that was used in these very big projects. Teams would constantly cause accidental visual regressions in unexpected areas.
The cost we paid was a set of manual regression tests that we had to run on every product every time we wanted to release something new.
We never felt fully safe pushing our code to our clients.
This led us to split our application into smaller independent modules and turn our products into lean projects that simply orchestrate the integration between these smaller modules.
The architecture
The architecture we came up with is made up of three layers:
Base layer — core components
At the base of our architecture, we have a set of core components such as buttons, inputs, checkboxes, and anything that has no business logic.
These core components are in a monorepo governed by Lerna, which allows us to release each component to npm with its own independent version.
Middle layer — Feature modules
The next layer is another monorepo project that contains modules with business logic. Each module is made up of core components. Examples of such modules are a form the user fills out to execute an action, a toast notification feature, or a grid filter.
While these modules use the core components, they don’t necessarily depend on them directly. By default, our core components are defined as a peerDependency and are not bundled up with the end result. To handle this dependency policy, we use Preconstruct, which is an opinionated build configuration that uses rollup. Preconstruct ensures that every peerDependency is automatically defined as “external”, and that every dependency used in a package is actually defined in the package.json. If a certain module needs a specific version of a core component, it may define it as a dependency. However, this is not something we do often.
Top layer — Applications
Our top layer consists of various applications called platforms. Each platform is made up of modules and has two responsibilities:
- Orchestrating the integration between the different modules.
- Defining the versions of the core components that the modules are going to use.
Introducing Storybook and Chromatic
To support a good development experience, the monorepos each have their own Storybook configuration.
If you’re unfamiliar with Storybook, I highly recommend you take a break and check it out!
Storybook is a sandbox for running components separately from their hosting application. It enabled us to develop modules independently, providing our developers with immediate feedback on their local machines on what they’re doing.
On top of our Storybook instances, we use Chromatic to ensure no visual regression made its way to production unnoticed.
Chromatic is a service that we run on every build of our projects that automatically compares a snapshot of every story in Storybook against the previously approved version of that story (the baseline).
This proved to be a game-changer for us. In a very easy and visual way, we were able to identify any unwanted visual change to our components and modules.
This worked so well for us that all our teams quickly adopted the new repositories.
After about a year, the modules' repository contained about 400 modules that were a composition of core components and business logic.
There’s just one caveat: To ensure this architecture works well, we have to make sure that the core components versions in our monorepo and main applications are identical. We rely on Dependabot to manage this concern.
Let’s talk about the problems
We quickly found ourselves with a Storybook instance that took several minutes to build and, what’s worse, we found ourselves with a very long testing task for the entire project.
Overall, it seemed like our project architecture was incapable of scaling to the number of features we were pushing to it.
It became critical for us to ensure that the development experience is great without compromising coverage and quality.
Revisiting Chromatic
When we looked for a solution to improve our project’s development experience, we decided to revisit Chromatic.
At its core, Chromatic aligns very well with putting the developers’ experience front and center, and we looked for ways to harness that back into our project.
Chromatic covered an important step in our automation, ensuring that things looked the way they should. It felt like a good foundation to build on top of.
Run only what you need
The first issue we needed to solve was how much time it took to run and update Storybook: When our developers were working on the project, they would run the Storybook command and wait for the build to finish before developing. This would take a long time because we would load all the stories and modules while using only a few at a time.
To solve this problem, we moved each story to its relevant package:
Using Plop, we created a small CLI tool that lets developers input the name of the packages they want to load into Storybook, and then generates a config file for Storybook to load only the stories under the selected packages.
While this improved our immediate concern of running Storybook locally, we still had to wait a considerable amount of time when issuing pull requests. Every pull request would build Storybook and trigger a Chromatic test on the result.
To solve this, we used Lerna to identify which packages were changed in our commit, and we used our CLI tool to generate a Storybook configuration with only the relevant stories.
This script looks something like this:
To close the loop, we now need to tell Chromatic to ignore missing stories using the — preserve-missing argument.
This ensures that during our pull request process, we only test the relevant stories and not the whole repository.
This yielded amazing results. Our CI/CD duration was reduced by 80%!
However, we wanted to take it a step further, and fully harness Chromatic to our advantage.
Out with unit tests! In with Chromatic tests!
OK, OK, it’s not that we’ve completely let go of our unit tests, but we’ve reduced them considerably in favor of tests that helped us cover more ground in a shorter period of time.
Building good unit tests is hard, and maintaining them over time is even harder. On top of that, you’re always facing the risk of overtesting.
Now I’m going to oversimplify this section a bit, but it’ll allow me to explain the benefits of our approach. Let’s say we have a form with inputs that have states which are dependent on other inputs or component props.
Usually, your option for testing such a form would be to:
- Write unit tests with libraries like Enzyme, trigger some events, query for components, and ensure that the result you receive is what you expect.
These kinds of tests don’t really tell you much about what your end-user will see in their browser. - Alternatively, you can use mechanisms like Jest snapshot tests to see the resulting HTML and the wiring of props and states between components. These too don’t really tell the story of what the end-user will end up seeing, and poses the additional risk of someone just accepting a change without understanding what was changed.
- Use browser-based tests with tools like Cypress. While these tests are the most reliable in understanding what the end-user sees, they tend to be slower and require maintenance.
These tests and others all have their benefits, and we still rely on them. However, with React, Storybook, and Chromatic, we reduced these tests considerably in favor of simple stories.
What we did was add the ability to inject states to our components and modules. This allowed us to write stories with specific states to test.
Want to see how the form looks with invalid values? Inject a state with an invalid value!
Want to ensure a specific dropdown is disabled when a checkbox is selected? Inject that state as well!
With these kinds of stories, our unit tests were relieved of a considerable responsibility, and were left with simply testing the creation of new states through events and integrations.
In conclusion
Our journey at Kenshoo to improve our UI's stability proved to come at a considerable cost to the experience of developing. Without providing our developers with good development experience, you’ll find that they end up spending more time fighting your infrastructure than actually building things for your customers.
We learned that finding tools that put the development experience front and center increases our productivity while maintaining great overall quality.
I encourage you to find the tools that make the most sense to your architecture, and use them in creative ways that address not only your customers’ pain points, but your developers’ as well.
For us, the payoff was huge: Considerably increased developer participation, better features, and better quality all around.
I would love to hear how you utilize Storybook in your projects. Feel free to leave a comment.
