Automating a Design System
There are many things to consider when establishing a design system that would have the legs to scale and grow, and meet the design and development needs for all adopting the system. In terms of delivery, the ability to provide rapid iteration and enhancements can only be taken so far with a team of humans.
Introducing Carbon for IBM.com
Our team provides the suite of IBM.com components (using Carbon Design System under the hood) necessary for building an IBM.com experience in React and Web Components. In addition to the components themselves, there are many shared utilities, services, and styles (amongst other shared resources) between the two offerings. All of this is housed under a monorepo, in order to best manage these packages in unison.
Testing and delivery
Our team is actively improving Carbon for IBM.com, and have a scheduled release every three weeks. Between releases, we release a canary version of the design system with every code merge. We also publish nightlies, and release candidates during our code freeze periods.
To ensure quality control of everything we deliver, we generate deploy previews in every pull request. The baseline that is automatically generated are our React and Web Components storybook instances (and React wrapper version of the Web Components in storybook), as well as downstream test applications for each as a NextJS application and a Web Components handlebars application. Additional labels can also be added to pull requests to trigger deploy previews of the right-to-left (RTL) versions of the storybooks, and special experimental versions of the storybooks that will surface components behind a feature flag.
Our team started out using Netlify for a smaller footprint of this at the beginning, but we were quickly using up a lot of the monthly build time. There also was no apparent way to test our design system changes in our test applications.
We very early on migrated the entire build flow into an internal IBM Jenkins server that runs agents on a Kubernetes cluster, and host a custom docker image agent in IBM’s internal Artifactory. All deploy previews are hosted on IBM Cloud. This move gave us greater flexibility to scale our level of testing across multiple repositories and evaluating downstream impact, as well as more focused integration testing. Integration testing involves creating isolated deployments of our codesandbox examples, then running cypress e2e tests.
Visual regression testing
Deploy previews in our pull requests is just the beginning of how testing is done. There are way too many components and test applications, as well as permutations of storybook to test manually. In addition to the standard suite of unit tests, integration tests, and e2e tests, we heavily leverage visual regression with Browserstack Percy. This gives us a heads up if any visual changes happen across all components within the system. Sometimes changing even a single line of code could have a huge domino effect across the system.
Once pull requests are merged in, more of our team’s robot army comes to life. A canary version of the design system is automatically published to public NPM. For every storybook permutation we have, there is a canary version for them that is also hosted on IBM Cloud. These get updated once a new canary version is available. Specifically for Web Components, our team also provides CDN versions of each component, which is auto-published for every canary, next (release candidates), and latest release.
Release testing and publishing
When it is time to cut a release, we freeze the code into a release branch. From there, release candidates are published to NPM, and our staging environments for storybook and downstream test applications are updated with the latest release candidate version for regression testing. All necessary announcements are made in our internal IBM Slack, and the releases are created in Github. If any hotfixes are needed, they are made on the release branch then merged back down to the main branch to keep it up-to-date.
When we first started out, all of the release steps were documented into a release “playlist”, which were manually executed by the release manager on rotation. This required the full setup to be on the release manager’s local machine, and was obviously took longer and was prone to user error (despite how fantastic our team members are!). It didn’t take us long to ultimately create automated deployment and merge scripts. We still have our release playlist, but has been dramatically reduced to a few button pushes.
Once a full release goes out (NPM and CDN), our production storybooks get automatically updated and published, as well as updating the dependencies in our downstream test applications. We also have template versions of our test applications for IBM.com application teams that wish to use them (NextJS and Web Components templates), which also get automatically updated with latest.
Keeping up with upstream
Another important aspect to consider for our design system specifically is to always stay in sync with the Carbon Design System. Like other tasks mentioned, this was manually managed whenever we were notified of updates in the Carbon Design System slack channel. Today, we have a cron job in Jenkins to always check if there is a new version, which will automatically create a pull request to update our repositories if a new one is available. We take full advantage of the isolated deploy preview testing (especially Percy’s visual regression testing) to see if there is any negative downstream impact while taking in upstream changes.
Our documentation website also has its own “upgrade bot” for Carbon Gatsby Theme. This ensures that it will always stay consistent with the intended digital experience for IBM Design documentation sites.
Github Actions vs. Jenkins
Jenkins was mentioned a number of times, as it is used in a large portion of our overall delivery workflow. Our team also makes use of Github Actions where applicable. The general strategy that we followed is that if there are any tasks that need to happen with forked repositories without exposing sensitive tokens, they are managed behind our firewall in Jenkins. Any tasks that also require happening behind a firewall (like deployments to our CDN or production environments) are also in Jenkins. As you can imagine, Jenkins is constantly keeping itself busy. This is why we would identify any tasks that would make sense running in Github Actions wherever we can. This includes running continuous integration tests on pull requests, NPM canary publishing and deployments to lower environments from origin, or sending dispatch messages to our other open source repositories to trigger downstream deployments.
Humans and robots in harmony
At the end of the day, the Carbon for IBM.com team is composed of a stellar group of designers, developers, product and project managers, and quality assurance analysts. Our brilliant team members are crafting the experience, creating the tools, providing front-line support for our application teams, and delivering Carbon the best way that we can for IBM.com. We let the robots handle the platform’s continuous stream of changes across close to 50 environments, with over 2500 releases to-date. We have a lot more work ahead of us, but we are certainly getting there faster with automation.
