Flat Design: Why you should question Nielsen Norman’s research on the trendy design style.

The Nielsen Norman Group is probably the most influential usability consultancy in the design industry. So, when they publish an article about their research findings titled “Flat UI Elements Attract Less Attention and Cause Uncertainty”, many designers perk up and pay attention.

In the few days after the article’s release I’ve already seen it shared several times as evidence of the weaknesses of flat as an interface aesthetic. But there’s a problem with that conclusion.

It’s not actually supported by the study that was done.

So what’s the issue? Let’s take a look at the very beginning of the article:

The popularity of flat design in digital interfaces has coincided with a scarcity of signifiers. Many modern UIs have ripped out the perceptible cues that users rely on to understand what is clickable.

While the title had clearly set the expectation that “flat design” was the subject of the research, the article immediately jumps in to an equivocation between flat design and design with “weak or absent signifiers”. It turns out the latter was actually the focus of the study even though we were promised the former.

Leaving aside the issues with calling anything “weak” before actually running the study, one might still expect (based on the title of the article) that the weak signifier scenarios tested were reasonably representative of flat design. However, a deeper dive into the full set of designs tested shows this was not true in most cases. Five of the nine scenarios asked users to find text links that differed only in how they were styled and colored.

The design on the left outperformed the design on the right, surprising absolutely no one.

Visually differentiating links helps users to know they are clickable? Who knew?

The thing is, changing the display of links in this way doesn’t actually tell us anything about flat design. Flat is characterized by a lack of depth cues, but neither of the designs above have any depth cues at all!

Some test cases did more closely resemble a comparison between flat and depth based UI but they came with their own problems. Let’s take a look at the first test case.

Participants were more easily able to find the “Reserve Now” button in the design on the left.

Notice anything strange? Not only are the buttons in the design on the right flat, they’re also “Ghost” buttons which make them almost entirely a different color!

The difference in visual contrast between the buttons acts as a confounding variable. The version on the left may have performed better simply because it had stronger contrast, not because it had more depth cues.

A similar problem can be found in this scenario:

The tab in the design on the left (View/Modify/Cancel a Reservation) was located more quickly than the tab in the design on the right.

Again, color and depth cues were tested simultaneously. The color difference resulted in a significant contrast disparity that may have been responsible for the difference in results.

So how about this one?

Participants more quickly located a slider in the design on the left than the in the design on the right.

While we’ve moved beyond color contrast, there are still confounding variables here. The toggles in the slider on the left are much larger than those on the right.

One test case remains. Could this be the one that fairly compared flat and non-flat UI?

Actually, it is!

The thing is, there was no noted difference between these two designs. The eye-tracking results showed roughly equivalent results. The scenario that most fairly represented flat design didn’t result in a significant difference between either option.

So what’s our takeaway here?

When looking at secondary usability research make sure to read past the headline. Don’t take the conclusion presented at face value without understanding the research that was done. Often a blanket statement about particular design elements will be based on examples that are poorly implemented or examined within a very narrow context. It may be that flat UI does have some disadvantages when compared to UI with depth but it’s risky to make that determination based on a single study like the one we’ve reviewed here.

To Nielsen Norman’s credit, they do share a good deal of the methodology behind their testing. Without that transparency we wouldn’t have been able to consider their study in as much depth as we have. You can view their full article here where enough information is provided for anyone to draw their own conclusion. For organizations that don’t share testing methods up front you should be even more skeptical about adopting their recommendations without doing additional research on your own.