Usability — Evaluations

FIT3175 — Monash University — Week 8

There are many ways that usability of a piece of software is evaluated in the modern workplace. Usability much be carefully considered and tested for, as otherwise the product would be very poorly designed and difficult for doing basic tasks.

The Many Tests For Usable Systems

The scientific community has made quite a few tests for systematically categorising usable and unusable user interfaces. Most of these are very similar — even similar to the ISO 9241 standards.

Dr Jacob Neilson’s Traits of a useful system

He defines a useful system as having high usability + high utility. Therefore the system will be user friendly and have the functions necessary to work properly in the environment that it has been designed for.

Dr. Jacob Neilson meaures usability in relation to a number of traits

Usability traits

• Learnability (whether system can be easily taught to a novice)
• Efficiency (of getting tasks done)
• Memorability (when users return after not using the system for a long time)
• Errors (difficult for user error to occur)
• Satisfaction (Pleasant to use)
• Utility (provides functions needed)

Ted Nelson’s Usability Rules

Ted Nelson picked up Dr Neilson’s work above and suggested a few more guidelines. He added that:

• A novice should be able to learn in 10 minutes from guidance of tutor
• UI should be able to be understood in an emergency in 10 seconds

Whitney Queensbery also made a Memonic called the 5 Es

• Effective, efficient, engaging, error tolerant, easy to learn
• You can assign a KPI to each of these and track if you are succeeding to meet it (eg. Efficient => User should find document in 10 seconds)

Usability Engineering

In the late 1900s and early 2000s, usability was seen as paramount when designing new software systems, so much so that a new discipline was created just for usability professionals. Usability Engineering Evaluating products based on usability methods and principles. The process of conducting usability engineering involves heuristic evaluation, looking to know whether or not the product meets certain measures.

Usability Evaluation Techniques

Through the use of Usability Engineering, different methods were devised to evaluate the usability of a system, measuring mainly two distinct parameters:

• Qualitative techniques: the metrics (learnability, efficiency, error rate etc)
• Quantitative techniques (subjective ease of use)

It is worth mentioning that usability evaluation techniques don’t involve testing the functions of the software, only its usability.

Usability Testing

Usability testing is part of the evaluation process. It is generally conducted in labs or controlled environments where a general evaluation is done of the software to see if it aligns with all the design principles and guidelines.

A Simple Framework for Usability Evaluations

1. List overall goals of evaluation. Make sure to list what performance indicators might be valuable and how the goals fit into the wider environment of the business.
2. Explore specific questions that are linked to the software itself. Does it follow user expectations?
3. Choose approach method. How will your product be best evaluated?
4. identify issues with approach
5. ethics considerations
6. interpret/evaluate/present data

No User Required Techniques

These techniques are easy for businesses to do before launching the product to ensure that the User Interface is friendly enough and it works to the business’ advantage.

Techniques which happen without real users are normally faster to do and cheaper, but don’t give the kind of results that real world testing does. Since that takes place in the user’s daily use case, the problems encountered there may be far different

Predictive Models

One way is to develop these models to evaluate behaviour and designs. When doing this we are evaluating the product for a match with the model based on GOMS:

• Articulate user Goals
• Does it fit daily Operations
• What Methods does it serve
• Other Selection Rules

Expert Reviews

Experts are great because they are specialized in the field of UI design and usability, and are able to give advice from their accumulated knowledge. However it is important to remember that they don’t 100% understand users.

• They check guidelines, consistency
• Experts do a walkthrough of the app and identify problems
• A Formal usability inspection is generally conducted (6 step procedure in a room with devs)

Heuristic evaluation

Experts may ask to do a heuristic evaluation of the program. This is based off of Nielsen’s Ten Usability Characteristics below:

The procedure to follow in a heuristic evaluation is as follows:

1. Train evaluators on domain of business
2. Each evaluator separately tests
3. Rating given for priority of tasks
4. Team discussion

User Centred Evaluations

Usability Testing

• This involves observing users and giving them tests/asking questions. It could also use eye tracking heatmaps to identify weak spots in the application.

Other User Centered Evaluations

• Field Testing (windows technical previews)
• A/B Testing: create 2 versions of a page, test which one does better
• User Surveys
• User Acceptance Testing (against certain criteria)