Evaluation techniques for interactive systems

In the process of designing usable interactive systems, there should be a way to assess our designs and test our systems to ensure that they actually behave as we expect and meet user requirements. This is the place where ‘Evalution’ comes to play.

Evaluation of interactive systems…

Simply as mentioned above, it is the way of assessing the functionalities, usability, effectiveness of a interactive system. It does not limit for a single process of designing a system. Evaluation should take place throughout the design process in order to ensure a quality user experience.

Goals of Evaluation

The goals of evaluation can be divided into three main categories.

Assess system functionality and usability

We have to pay keen attention on the system’s functionalities as considering them is very important in assuring user’s requirements. The system should enable users to perform their expected tasks easily. Clarity and the usability of the system should be high to increase the effectiveness of the system. Evaluation supports in measuring user’s performance with the system, to assess the effectiveness of the system implementing tasks.

Assess effect of interface on user

User is the person who directly involve with system and its functional capabilities. Therefore, it is crucial to consider the impact on the user. When determining user effects, we have to concern on system’s usability, learnability and user’s satisfaction with the system. Moreover it includes emotional response of the user as well. So, evaluating those areas are expected in making a system more effective.

Identify problems related to both the functionality and usability of the design

Identifying specific problems related to the system is another goal of evaluation. In this scenario, evaluation may help in identifying unexpected issues, new requirements confusions of the system. Identifying those problems at the right level may improve the overall user experience of the system.

Evaluation through expert analysis

Evaluation through expert analysis is way to assess interactive systems in making them more usable and effective. Professional experts play a vital role in this approach. They evaluate the system and give necessary recommendation to improve the experience with the system to the user using several methods such as cognitive walkthrough, heuristic evaluation, the use of models and use of previous work.

Cognitive Walkthrough

Cognitive Walkthrough is a formative analytical evaluation and simulation process that takes a list of questions surveying experts while completing tasks. In this approach the designer or designing team specifies and performs a series of tasks on which one will evaluate the design of the system.

Heuristic Evaluation

Heuristic evaluation must be carefully taken into account when designing a quality product that users can easily interact with and find value in their interaction. It is a thorough assessment of a product’s user interface with the aim of identifying potential usability issues that may emerge during user interaction and proposing solutions.

Review-based evaluation

Review-based evaluation is an expert-based evaluation technique that uses actual data from the literature (such as that from psychology, HCI, etc.) and experimental findings to confirm or deny various aspects of the user interface design.

Evaluation through user participation

The later stages of development, when at least a functioning prototype of the system is in place, are when user involvement in evaluation typically takes place. These incorporate empirical or experimental techniques, observational techniques, query strategies, and bodily monitoring techniques like eye-tracking, heart-rate monitoring, and skin conductance measurements.

Styles of evaluation

When considering styles of evaluation, we can mainly identify two styles.

• Laboratory Study: In this type of evaluation study, users are asked to participate in controlled tests outside of their normal working environment, frequently in a specialized usability laboratory.
• Field Study: In order to watch the system in use, this kind of evaluation sends the designer or evaluator into the user’s workplace.

Empirical methods: experimental evaluation

This approach involves collecting practical data to validate a specific assertion or theory. The assessor must choose a hypothesis to examine, and any changes in behavior metrics are ascribed to different scenarios. Several factors, such as participants, variables, hypothesis, and experiment design, are crucial to the dependability of the experiment.

Observational techniques

Observing users as they interact with a system is a crucial part of evaluating its effectiveness. This involves asking users to perform tasks while the evaluator records their actions, preferably in their natural environment. Techniques such as Think Aloud, Cooperative Evaluation, and Protocol Analysis can be used to gather data and gain insights into user interactions with the system. These insights can inform improvements to the system’s design and implementation.

Think Aloud, Cooperative evaluation, Protocol analysis, Automated analysis, post-task walkthroughs.

Query techniques

This method depends on explicitly querying the user about the interface. In order to elicit specifics about the user’s perception of a system, query methods can be helpful. We can mainly identify two approaches that involves in this technique.

• Interviews: Provides a direct and structured way of gathering information while asking questions, opinions about user experience with an interactive system
• Questionnaires: Users are asked a series of predetermined questions about their preferences and their thoughts on the design, and the analyst uses the answers to draw conclusions.

Evaluation through monitoring physiological responses

With the help of this technique, evaluation specialists will be able to more clearly observe what users actually do when they engage with a system and gauge their feelings toward it. Utilizing techniques like eye tracking and other physiological measurement techniques, they can quantify it.

ref:

• ALAN DIX, JANET FINLAY, GREGORY D. ABOWD, RUSSELL BEALE HUMAN–COMPUTER INTERACTION THIRD EDITION | Chapter 9
• What Is Heuristic Evaluation? Simple Guide to Conduct One (+ Free Template) (adamfard.com)
• HCI: Cognitive Walkthrough (ed.ac.uk)