How to Use Heuristic Evaluations to Improve Product Designs

Matt Rae
Matt Rae
Feb 5 · 7 min read

Despite the name, heuristic evaluations are not as complex as they sound. In fact, they’re a great tool for teams with various budgets for testing, as they can be executed in a couple of different ways.

Image for post
Image for post

What is a heuristic evaluation?

A heuristic technique is an approach of discovery or problem solving that has broad guidelines, or rules of thumb, without any rules etched in stone. It’s a guided look into a problem or study.

As it applies to UX and product design, a heuristic evaluation is a method of inspecting and evaluating the usability of a website, or product. You may also hear it referred to as a “usability audit” or an “expert review”. Using a set of heuristics, one or more experts will evaluate how well a product complies to these heuristics to define its usability. Often they’ll leverage a score-card, or numeric-based scoring (weighted to the impact on usability) for each heuristic.

The value of heuristic evaluations is prevalent in early stage design and development, and in smaller organizations which may not be equipped with the budget or resources to support a robust user-testing program but still need to validate design decisions, and ensure a good user experience.

Image for post
Image for post

How a heuristic evaluation is different from usability testing

Usability testing leverages users of a product, or people representing the intended target audience, to understand and identify areas of friction or error that may prevent completion of specified tasks, or lead to dissatisfaction in a target group. They may be seasoned users evaluating a new feature, or new users evaluating a particular user story. One primary difference is the resourcing required to run effective usability tests. It can become cumbersome to source users for a test, and once feedback is incorporated into an iteration, new users must be sourced to validate that solution too.

However, with a heuristic evaluation, you’re evaluating with 1–3 experts, who don’t need to be users of the product. If you can’t find an expert (a usability professional knowledgeable in these evaluations) you can perform something called a heuristic markup, as described by Leah Buley in her book The User Experience Team of One.

The heuristic markup approach is a less formal method that can be executed by your internal team, and involves navigating the product along the journey map you expect your users to experience. Make notes along the way and use a set of ‘heuristics’ to help guide you.

Helpful heuristics to follow

A heuristic is a broad set of standards — there are no official heuristics to use when conducting your evaluation. However, there are some very commonly used heuristics, such as those written by Jakob Nielsen of the Nielsen Norman Group. Released in 1994 in his book Usability Engineering the following 10 Heuristics are a staple in the field, and are considered a standard when evaluating usability.

  • Visibility of system status. Users should always be informed of what the system, or product is doing, in a reasonable timeline.

Other principles to consider when developing your heuristics are Gerhardt-Powals’ cognitive engineering principles from the International Journal of Human-Computer Interaction.

These heuristics provide a strong core, however, they were written in the early 90s, at a time when technology products, especially software, were in a very different state. Though for the most part, these hold true today, it may be beneficial to review these and reflect on whether they’re the right heuristics to use for your own particular product. Using these heuristics along with principles and guidelines specific to your product will make for more effective testing and better results.

Image for post
Image for post

Guidelines for effective heuristic evaluations

Choose evaluators

You want to make sure when selecting evaluators, that they are not end users of your product. However, some understanding of the industry or product space is helpful in evaluating the usability for that audience.

If you’re in a large company with the resources to hire external usability experts, many companies, like Nielson Norman group provide consulting services which include usability reviews and heuristic evaluations. Hiring an outside expert can provide a fresh set of eyes, and industry experience to provide valuable feedback on how to improve the usability of your site.

Provide direction

It is likely that evaluating your entire product or website will be a large task, and if you’re hiring experts, could become expensive. Ensure efficiency in your evaluation by providing direction, such as user stories, or particular tasks you wish the experts to evaluate. However, ensure that you provide the same criteria to all experts to minimize the risk of bias in results. Allow for some exploration beyond the user-stories, as there may be avenues the experts wish to explore based on their industry knowledge, and past experience.

Evaluate

For this step it is beneficial to have someone, familiar with the product, in the room to document feedback, and provide answers to any questions evaluators may have. Having a note-taker ensures you don’t miss any valuable feedback that may arise.

When evaluating, it is recommended that you do two reviews. The first, allowing the experts to get familiar with the product, and interactions. It is in this phase they may pinpoint additional areas they would like to evaluate.

In the second phase, the website or product will be evaluated against the defined heuristics to the user stories, and the additional elements identified for evaluation. This is where most of the feedback, and problem areas will arise.

Review

Once you’ve run your tests, it is helpful to bring the experts together to cross-examine the problems identified by each person, it is likely that there will be overlap between experts. Having more than three evaluators, you will begin to see a sharp reduction in returns, and so it is recommended to use no more than three experts.

It is likely that, based on previous experience, the experts may produce some solutions and recommendations based off of the recorded issues. Don’t prevent this from happening, it is valuable feedback that is likely to benefit your product. However, do evaluate all feedback before implementing solutions.

Image for post
Image for post

When to use heuristic evaluations, and when not to

Heuristic evaluations are not the be-all, end-all. They’re one of many usability testing methods UX designers have at their disposal. However, they are still guidelines, and some of the feedback you’ll receive may not actually make sense for your product.

Benefits:

  • Heuristic evaluations can help to identify issues with particular components or sections of a product that have a negative impact on usability.

Drawbacks:

  • Heuristic evaluations rely heavily on the heuristics that are defined or chosen. If they’re misunderstood, you may receive biased results.

When implementing heuristic evaluations, understand the pros and cons, and be aware of false positives in reporting. It is always valuable to pair heuristic evaluations with some level of user-testing to ensure the problems identified by experts are in fact problems impacting your usability. Having multiple experts can help reduce false alarms, and ensure bias is reduced in your results.

For more unique insights and authentic points of view on the practice, business and impact of design, visit Adobe XD Ideas.

To learn about Adobe XD, our all-in-one design and prototyping tool:

Originally published at https://xd.adobe.com.

Thinking Design

Stories and insights from the design community.

Matt Rae

Written by

Matt Rae

Design Advocate for @AdobeXD. Equipping designers & teams to create amazing experiences with Adobe XD. Founder & Organizer of @designchats.

Thinking Design

Stories and insights from the design community.

Matt Rae

Written by

Matt Rae

Design Advocate for @AdobeXD. Equipping designers & teams to create amazing experiences with Adobe XD. Founder & Organizer of @designchats.

Thinking Design

Stories and insights from the design community.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store