UX heuristics: a closer look at Bastien and Scapin
A heuristic evaluation (an expert review of an interface based on usability heuristics) offers a quick and methodical way to catch and categorize usability problems. The question then becomes, what’s the best way to do it? It’s easy to notice what you’re looking for and difficult to notice what you’re missing, so choosing the right set of lenses from the start is vital.
As someone interested in UX, you’ve probably seen and/or used Nielsen’s 10 usability heuristics (currently the golden standard for usability guidelines). You might even know of a few other sets of heuristics, like Shneiderman’s Eight Golden Rules of Interface Design or the Weinschenk and Barker classification.
These are all very well and good, but have you heard about Bastien and Scapin’s Ergonomic criteria for the evaluation of human-computer interfaces? Popular in French-speaking UX circles, this list of heuristics may have a less catchy title but is gaining more attention by the day.
What is it?
Christian Bastien and Dominique Scapin were cognitive psychologists who worked on usability heuristics in the 1990s. Their list contains 8 main criteria, subdivided into 18 total sections (denoted by asterisks*).
After experimentally validating their criteria, they found that both experts and novices could accurately classify usability issues with the list, and that their list was “complete (i.e., covering most current guidelines); distinct (i.e., independent of one another); and applicable (i.e., usable in all situations considered).”
At first glance, this seems superficially similar to Nielsen’s heuristics; you can see where “User control and freedom” might overlap with “Explicit control”, along with other categories. However, Bastien and Scapin offer a more detailed approach and rationale for each section, as well as clarifying distinctions between each criteria so there’s less chance of confusion or incorrect classifications.
This means once you’re going through an interface and dissecting your findings, you can be more thorough and precise by seeing the same problem from different angles and solutioning accordingly.
For example, you might notice that users find a form long and tedious, breaking Nielsen’s Aesthetic and minimalist design heuristic. With Bastien and Scapin’s list, you can drill down into whether there are too many steps (Minimal actions), missing or extraneous information (Information density), or extra-long inputs (Conciseness).
Bastien and Scapin vs. Nielsen
So how do these two compare? First we’ll take a look at what’s similar and what differs between these sets of heuristics, then dive into how each one performs under pressure.
Content
The table below shows how Nielsen’s 10 heuristics neatly fit into Bastien and Scapin’s major criteria (with the exception of “Significance of codes”). We’ve roughly mapped Bastien and Scapin’s sub-criteria across Nielsen’s heuristics, showing how both sets are related and the degree of depth curated by Bastien and Scapin.
You can also see differences in language; Bastien and Scapin prefer shorter and more concise labels with additional detail offered by the sub-criteria, whereas Nielsen opts for more descriptive labels that encompass multiple facets of the heuristics. Which set of labels do you find more user-friendly?
Performance
The overall goal of heuristic evaluation is to uncover usability problems and allow designers to easily address them, and so can be somewhat quantified by the number and quality of the issues reported.
In Luzzardi et al.’s (2004) case study, the authors found that evaluators who used Bastien and Scapin’s criteria (vs. those using Nielsen’s heuristics) found slightly more problems (46 vs. 39) and a higher frequency of more severe issues (22 vs. 14).
Unfortunately, few cases exist where these two sets of heuristics are compared directly to one another. In the future, it’d be interesting to compare these two powerhouses in something like Hvannberg et al.’s (2007) study, where they stress-test Nielsen’s heuristics and Gerhardt-Powals’ cognitive engineering principles by number of issues discovered, time spent, and ease of use.
After all, a list of heuristics might encompass every aspect of usability, but if it takes too long or is too tedious to go through, then it’s better left behind.
Generalizability
These heuristics have been extensively used on digital interfaces, but as we expand into other domains (like voice user interfaces (VUIs) or VR) with context-specific concerns, how adaptable are these heuristics? Bastien and Scapin warned us back in 1997 about their criteria’s limitations and the need for future changes to evolve these heuristics alongside technological progress.
Take Nowacki et al. (2020); the authors wanted to improve existing heuristics for the VUI space. Drawing from a comprehensive pool of both general and VUI-specific usability guidelines, they attempted to modify both Nielsen’s and Bastien & Scapin’s heuristics to fit the voice medium.
They found it was more difficult to extend Nielsen’s list without losing valuable information, so elected to go with Bastien and Scapin instead. In the end, they kept a majority of the ergonomic criteria, showing how versatile and applicable Bastien and Scapin is to our changing world.
So what now?
If you want to try out Bastien and Scapin for yourself, you can snag a copy of this handy-dandy heuristic evaluation template with their guidelines and conduct a heuristic evaluation on the interface of your choice. We’ve included Bastien and Scapin’s main 8 criteria, all 18 sub-criteria, and Nielsen’s 10 heuristics in the template for ease of comparison.
(If you need a quick refresher on heuristic evaluations, there are plenty of articles to help you out, though most will only teach you about Nielsen’s 10 heuristics).
In any case, both Nielsen’s heuristics and Bastien & Scapin’s ergonomic criteria are valid and useful in their own right. They’ve withstood the test of time, earning their place at the top of the field; their universality has been tested and proven over three decades of heuristic experimentation. The one you use is ultimately up to you: choose wisely!
References
Hvannberg, E. T., Law, E. L. C., & Lérusdóttir, M. K. (2007). Heuristic evaluation: Comparing ways of finding and reporting usability problems. Interacting with computers, 19(2), 225–240.
Luzzardi, P., Freitas, C. M. D. S., Cava, R., Duarte, G., & Vasconcelos, M. (2004). An extended set of ergonomic criteria for information visualization techniques. In Proceedings of the Seventh IASTED International Conference on Computer Graphics And Imaging (Cgim-2004), Kauai (pp. 236–241). sn.
Nielsen, J. (1994, April). Enhancing the explanatory power of usability heuristics. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (pp. 152–158).
Nowacki, C., Gordeeva, A., & Lizé, A. H. (2020, July). Improving the usability of voice user interfaces: A new set of ergonomic criteria. In International Conference on Human-Computer Interaction (pp. 117–133). Springer, Cham.
Scapin, D. L., & Bastien, J. C. (1997). Ergonomic criteria for evaluating the ergonomic quality of interactive systems. Behaviour & information technology, 16(4–5), 220–231.
