Storybook Ecosystem CI

Protecting Storybook users from package upgrade headaches

Upgrading npm packages can be a bit like crossing a minefield. Even a tiny patch upgrade can blow up your entire stack. Unfortunately, this kind of issue is common in modern open-source software. Packages often break due to bugs in their code, bugs in their dependencies, or incompatible transitive dependency upgrades.

As maintainers of Storybook — one of the most popular UI development tools — we’re all too familiar with the terrors of package upgrades. That’s why we’ve created Storybook Ecosystem CI, an early warning system to help protect our users from this chaotic environment.

• 🚥 Public status page that reports across a matrix of officially supported configurations
• ✅ Suite of tests that run on every commit
• 🔔 Notification system to alert us of critical failures

Read on to learn more about what we’re testing and how it’s already paying dividends to improve Storybook’s stability.

Wait, but why?

Storybook sits in the middle of one of the fastest-moving areas of software development, and our users expect us to work with the latest versions of everything.

The front-end ecosystem moves fast. Frameworks and libraries release regular updates, and new build tools constantly emerge. Keeping up is no small task due to the broad range of configurations that Storybook supports, combined with all its features.

Storybook has an enormous compatibility matrix

• 10+ renderers: React, Vue, Angular, Web components, Svelte, Ember, HTML, and more
• 2 languages: JS, TS
• 2 builders: Webpack, Vite
• 3 package managers: npm, yarn, pnpm
• Meta-frameworks like NextJS, SvelteKit, etc.

Plus, various versions of all of the above (React 16+, Vue2/3, npm6/7/8, TS 3+ but also taking advantage of the latest 4.9 satisfies feature, and on and on! 😅)

Your project uses one combination of those options, e.g., NextJS + React + Webpack + yarn. Whereas, Storybook supports a combinatorial matrix of project configurations.

It’s hard to make big changes to Storybook and ensure all project configurations continue to work. Storybook Ecosystem CI simplifies this process by testing this configuration matrix continuously and catching issues early.

Status page

The centerpiece of the Ecosystem CI is a public status page. A broken package is analogous to a “service outage” for users of that package, and we’re trying to maximize Storybook’s “uptime.” So we took design inspiration from popular status dashboards:

Each “heartbeat” bar in our visualization summarizes a daily CI run against an extensive test suite. Clicking on the bar shows which tests failed and gives you a link to see the full CI run for even more details.

As a Storybook user, or potential user evaluating Storybook for your project, you can see whether your configuration is supported and the recent stability of that configuration.

Sandbox testing

The way we test each of these configurations is by creating a sandbox. This is a new standalone project based on a generator script:

'nextjs/default-js': {
  script: 'yarn create next-app nextjs-default-js --javascript --eslint',
}

Our sandbox script then installs Storybook like a user would, adds a suite of stories, and runs a series of tests (Chromatic snapshots, Storybook test-runner and Playwright e2e).

When we converted our stories to Component Story Format 3 (CSF3), we found that we can write generic stories that don’t contain any framework/renderer-specific code. For example:

export default {
  // React/Vue/Angular/etc. component
  component: globalThis.components.Button,
}

export const Actions = {
  play: async ({ args, canvasElement }) => {
    await fireEvent.click(within(canvasElement).getByRole('button'));
    await expect(args.onSuccess).toHaveBeenCalled();
  },
};

This allows us to scalably test new frameworks (React, Vue, Angular, Web-components, Svelte, etc.) without having to duplicate code/work for each additional configuration.

Notification system

We test so many configurations so that we can be notified as soon as they break, whether due to changes in Storybook or updates in the broader ecosystem. Inspired by Vite’s own ecosystem CI, we’ve implemented a notification system that posts to the Storybook Discord whenever CI breaks:

Early wins

Ecosystem CI has already caught multiple breakages in the weeks that it’s been deployed. With Ecosystem CI, we had failing reproductions in CI and a process for resolving the issues. In most cases, we fixed the breakage within 24 hours.

• Webpack: When Webpack made a bad release of enhanced-resolve, it broke all Storybook Webpack users who installed/upgraded. Norbert de Langen and Yann Braga worked with Tobias Koppers, who released a Webpack fix within 24h.
• Babel: When babel made a bad release of @babel/plugin-transform-block-scoping, it broke all Storybook users who installed/upgraded. Norbert de Langen worked with Nicolò Ribaudo, who released a babel fix within 24h.
• Vite: When Vite released its v4 major upgrade, we had already set up Ecosystem CI to test the prerelease. Ian VanSchooten worked with Patak on the Vite team to release a Storybook fix.
• Angular: When Angular released its v15 major upgrade, it broke our Angular test within a few hours of the launch. Valentin Palkovic worked with Alan Agius on the Angular team to figure out the relevant breaking changes and release a Storybook fix within 3 days of the breakage.

Previously, these breakages would’ve been reported by the community via GitHub issues, often without reproductions, and it would take a few days to a week to resolve them in an ad-hoc manner.

Get involved

The frontend ecosystem is diverse and ever-changing and it’s hard to keep up. Storybook Ecosystem CI is our attempt to systematically tackle this problem, to create the best experience for our users. It was created by Michael Arestad, Yann Braga, Tom Coleman, Norbert de Langen, Kasper Peulen, Michael Shilman (me!), and the rest of the Storybook team.

Ecosystem CI is currently running against Storybook 7.0 beta. Try out the beta in your project with npx storybook@next init, or upgrade an existing Storybook by following the 7.0 migration guide. Join us on GitHub or chat with us on Discord. Finally, follow @storybookjs on Twitter for the latest news.