Component QA in Design Systems

Establish a process to deliver on the quality you promise

Nathan Curtis
Jan 6, 2017 · 6 min read

Months after our first release, productivity was strong. We’d reached 20 components. Buttons and forms were behind us. We cranked through cards, tables, and nav bars. As another component went to review, comments unexpectedly overwhelmed the pull request:

“Why don’t we have this variation?”
“It breaks with a reasonably long label.”
“Did you check for accessibility? It’s missing an attribute for…”

As discussion quickened, everyone swarmed to ask more questions. Really good questions. The contributing developer was overwhelmed: a finish line thought close extended again into the distance.

Ugh. We had documented neither what we check for nor how we QA. At all. We just built things, “checked ‘em”, and moved on. System teams must hold themselves to a higher standard than that.

Systems Must Deliver on Promised Quality

In adopting a system, a product team must trust that the system saves time and reduces risk. The system’s quality must be no worse — heck, let’s be honest, better — than what the product team would have done on their own. Therefore, incorporate quality into your messaging, but be ready to deliver the goods.

System advocates highlight quality when marketing their library. For example, Material Design Lite says that they…

… aim to optimize for cross-device use, gracefully degrade in older browsers, and offer an experience that is immediately accessible.

This isn’t whimsical aspiration. It’s a promise, promoting that library parts ascribe to attributes of quality. In fact, one of my favorite system principles is Salesforce Lightning’s Beauty, with a keyword I really appreciate:

Demonstrate respect for people’s time and attention through thoughtful and elegant craftsmanship.

In fact, I’ve observed the tone of a system pitch completely shift based on just one word that affords quality, like:

We guarantee the library includes only quality, accessible components.

Stakeholders–your director or VP–listen for those trigger words. To them, it signifies a library (at least partially) can unburden teams struggling with or ignoring things a leader cares about.

Therefore, you better be ready for the next questions: “What do you mean by quality?” and “How do you assure that quality?” Your answer must be better than “We check stuff after we build it.”

What’s Quality? Our QA Checklist

Here’s a quality checklist my most recent team established:

Does the component apply visual style — color, typography, icons, space, borders, and more — using appropriate variable tokens?

Does it cover all the necessary variations (primary, secondary, flat, menu button) and states (default, hover, active, disabled), given intended scope?

A first pass may not comprehensively addressing states across all button types…

Does it incorporate responsive display patterns and behaviors as needed?

Tab component, with options for responsive behavior

Is each dynamic word or image element resilient to too much, too little, and no content at all, respectively? For tabs, how long can labels be, and what happens when you run out of space?

Tab component, automatically shifting to alternate display based on content length

Does it fit well when placed next to or layered with other components to form a larger composition? For tabs, how can they be applied stacked with or on top of other components?

Tabs in different settings, positioned stacked after or on top of a photograph

Do all behaviors – typically, JavaScript-driven – function as expected? For responsive tabs, are gestures (for sliding left and right) and menus (for overflow tabs) behaving correctly in varied device settings?

Has the design and implementation accounted for accessible:

  • color contrast
  • ARIA
  • print versions
  • disabled CSS
  • svg and img attributes
  • descriptive headers
  • keyboard navigability, …

The list goes on, and could be augmented by testing with a screen reader like ChromeVox.

Has the component visual quality and accuracy been assessed across Safari, Chrome, Firefox, IE and other browsers across relevant devices?

Has the component design been assessed via usability testing with our customers, whether conducted for a product team or by our system team?

We don’t gate releases of an component — particularly the smallest ones — by usability testing. But we do seek out opportunities to assess component efficacy in related product tests as well as commission our own studies of relevant concepts too.

How We Assure Quality: A QA Workflow

It might feel like a pain. Too much rigor. A threat to team momentum. That was my gut reaction to the realities of a QA-then-refactor step. Do this to the whole library, now that we’re behind? This could limit production for sprints! It didn’t, actually.

I credit my team with convincing us that the library is nothing if the quality is poor or below a stated standard. Sure, we’d done light damage. Components in the wild now needed light refactoring. And this did trigger a distracting major semver release with a bevy of small, breaking changes. But the library’s demonstrable quality — and team’s confidence — shot way up when we got our act together.

Takeaway: Devise your QA checklist and process as you build your first components. You’ll justify the promise you make to your customers, and improve confidence and reputation as a result.

It may be tempting to think you can test your variations using that doc page you’ve also started to build. Bad idea.

Documentation page structure tightly constrains component displays to fit in the article’s width and example box, limiting display options. Plus, the page loads all the CSS of everything it displays, conflating your no-longer-encapsulated component styles with everything else.

Buttons documentation page (left) and all 92,000px of vertically scrolled Buttons “Kitchen Sink” page examples (right)

As an alternative, our team creates a “kitchen sink.” The page bursts with examples: states (default, active, disabled, etc), content (too long, too short, just right, none), colors and themes, all across the many variations (text only, with icon, icon only, etc). We limit loaded CSS to only that component and base styles, improving encapsulation.

We publish sink pages named buttons-sink.html in the same folder as thebuttons.html documentation page. There it rests, a useful if orphaned resource for design reviews, quick reference and automated visual differencing.

Takeaway: Separate documentation from testing artifacts.

QA begins when a developer signals they’re ready for a final review of their work in the build step, distinct from the documentation that may run in parallel but usually concludes after build.

For our team, QA is part of the Build task.

Takeaway: Incorporate QA as a clear step or sub-step that’ll prevent an item’s release if tests aren’t passed.

A dedicated QA resource is a luxury item for a systems team. Instead, systems teams I’ve been on blend designers and developers that wear other hats as needed, including QA. We hold a contributing developer accountable to submit components that they’ve already run through the checklist. Our process then also requires a distinct person — a different developer, the component’s designer, or both — to run through the checklist too.

Takeaway: Identify a predictable role expected to review each contribution.

About to embark on a design system, or need to dive deeper to discuss products and players? EightShapes conducts systems planning workshops and coaches clients on design systems. Let’s talk!


A collection of stories, studies, and deep thinking from…

Thanks to Dan Brown

Nathan Curtis

Written by

Founded UX firm @eightshapes, contributing to the design systems field through consulting and workshops. VT & @uchicago grad.


A collection of stories, studies, and deep thinking from EightShapes

Nathan Curtis

Written by

Founded UX firm @eightshapes, contributing to the design systems field through consulting and workshops. VT & @uchicago grad.


A collection of stories, studies, and deep thinking from EightShapes

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store