Designing Technology with the Human Mind in Mind: Part 2

The Filter

TL/DR: This is the second in a series of articles to discuss what behavioural science knows about human cognition for those in product development. The first article explained why product makers should understand human cognition and introduced the dual system model of cognition. This article will dive deeper into one part of that dual system model, “The filter.” Future articles will dive deeper into working memory, cognitive, workload, and stress. All articles will explain why this knowledge is important to making technology that has a great user experience.

The Filter

Everything that a human senses proceeds through the filter, which can then decide to (1) take action on the sensations, (2) send sensations to working memory for further processing, or (3) do nothing with the sensation, essentially ignore the sensation. There are three factors that determine which of these three actions occur are:

a. patterns in long term memory

b. current goals

c. salience.

Patterns

Let’s start with patterns in long term memory. We begin learning these patterns as infants (maybe even in utero). Thus, there are some patterns that are universal across humanity. The most basic of these universal patterns are the gestalt perceptual principles.

Gestalt perceptual principles are a series of laws discovered by German psychologists in the early 20th century. These laws explain how humans perceive certain combinations of sensations. For example in the drawing of the dual system model above everyone recognizes that the bubble with “decision making” is in the foreground and the bubble with “working memory” is in the background. The fact that we see one as the foreground and one as the background, even though it is a 2 dimensional image is one of the Gestalt principles.

Another nearly universal set of patterns are basic mathematical symbology including numerals. Though there are some cultural differences a lot of mathematic symbology is universal to humans. Other ubiquitous but not universal patterns are the scripts, phonemes, and morphemes in a person’s native language(s).

Though there are numerous patterns that are universal or wide spread, every individual develops their own patterns as they go through life. These idiosyncratic patterns are are often what differentiates experts and novices. Many different terms have been used to describe these individual patterns. Just a few of the terms that have been used are mental models, heuristics, top down processing, skill-based reasoning, rule based reasoning, knowledge based reasoning, automaticity, and so on. Malcolm Gladwell’s popular book Blink: The Power of Thinking Without Thinking is about these patterns.

To be clear, patterns are also not limited to just one-time sensations. Rather, the patterns stored in long term memory can be quite complex. They can include a series of sensations as well as actions. One such example of complex patterns with actions are affordances. Affordances are the actions that are possible within the environment. Such as a chair can be used as a step stool or a seat. Affordances were originally reported by JJ Gibson, an American psychologist, in the 1960s and then popularized by Don Norman in his book Design of Everyday Things in 1988.

There are some complex patterns where a sensation triggers a response. Those in sports often use the term “muscle memory” to describe this process. Psychologists use the term automaticity. The textbook example of automaticity is known as the Stroop effect. In the 1930s, American psychologist John Stroop replicated a study that had been done by German psychologists. In the experiment he showed participants something like the images below and asked them to say the colour of the ink symbols were presented in as quickly as they could. For those who read English as their native language, naming the colour of ink is far easier with the symbols on the left than the symbols on the right. This is because reading is learned to a level of automaticity; where colour naming is not. Almost anything can be learned to automaticity, but reading in ones native language is by far the most ubiquitous. When something has been learned to a level of automaticity, the individual experiencing this automatic response cannot explain it. This is because the sensations bypass working memory. So, the action takes place unconsciously.

This unconscious action only occurs when a there is a well learned stimulus-response pattern. Not all of the complex patterns in our long term memory consist of a stimulus-response. We learn many other complex patterns that do not include a single action response. In these cases the pattern is sent as a whole to working memory instead of the individual sensations. This results in lower workload and greater decision making abilities. This is a key component of expertise and will be discussed more in a later article. Sometimes the pattern is sent with information that was not sensed or may not exist in the world. This is what happens when we make a false inference or experience an illusion.

The process for developing these patterns is called sensemaking. Sensemaking is the process of creating new patterns or enhancing existing patterns. Anytime sensations are sent to our working memory sensemaking will occur. Sometimes we are conscious of this process. But other times we are not. This is something we are always doing — likely from the moment we have neurons. For example, infants are often seen staring at corners or shadows — their brains are creating the Gestalt patterns through sensemaking.

Current Goals

The second factor contributing to what happens to sensations in the filter is the person’s current goals. Sometimes we are engaged in tasks that require or encourage very narrow focus, for example counting items, driving on an unfamiliar road in bad weather, or learning a new skill. In such tasks, we actively ignore other stimuli (i.e., selective attention). In other cases, such as driving a familiar road in good weather or cooking a meal we have cooked many times before, we may be multi-tasking or not paying attention to any one thing (i.e., divided attention).

In the case of selective attention, those sensations associated with the focus of our attention have a higher likelihood of being passed on by the filter and those sensations not directly relevant to that focus are less likely to receive further processing. When the filter chooses to “ignore” sensations, psychologists call this inattentional blindness. Magicians rely on inattentional blindness to perform their illusions. Inattentional blindness needs to be considered when designing all nature of systems from websites to military systems. With regard to websites Banner Blindness is where people ignore banners believing them to be irrelevant to the task. Essentially, focusing on one task can lead us to ignore other critical information.

Salience

Regardless of any specific goal, humans’ sensory systems are always taking in information. This brings us to the third factor that the filter considers in its decision to pass information on to working memory or ignore it: salience. Salience refers to either a sensation that is significantly different than the background (e.g., a loud noise or bright light) or a pattern in the person’s filter with a strong reflex (e.g., your own name). Psychologists have named these the pop-out effect and the cocktail party phenomenon respectively. Conversely, exceptionally monotonous or commonplace sensations are unlikely to be passed on for additional processing regardless of the context. This is the reason for not remembering driving after a routine commute, not remembering if a daily medicine was taken or not, or not remembering if you locked the door. The events are typical and thus not salient enough to be processed for sensemaking or the task at hand.

Impact to Design

You may be thinking that is all very interesting, but why is this important to the design of technology? You probably have heard of “dark patterns”. These are user interfaces that take advantage of the patterns in our long term memory to bypass working memory — so that we take an action we might not normally take. Similarly, when we describe a product as intuitive, it is because using it aligns with the patterns we have in our long term memory.

Not aligning with the patterns users have will make it more challenging to use a product. Sometimes, we want to introduce friction and engage working memory. We must be careful that we are engaging the right pattern otherwise it can lead to significant errors.

One of these errors is negative transfer or mode errors. Negative transfer is when a well-learned action is correct in some contexts but an error in other contexts. In 2020 the FDA reported such a design problem with the color coding of Hydrogen Peroxide Vapor Sterilization kits. The 3M Comply Hydrogen Peroxide Chemical Indicator 1248 Card uses pink for processed and blue for unprocessed. The Aesculap MD334 Process Indicator Card uses pink for unprocessed and blue for processed. Hospitals that use both kits could have mistakenly used unsterilized equipment. The FDA was working to develop a standardized color-coding scheme to prevent these errors. In computing the most common error of this kind is typing in passwords when the caps lock key is on.

To prevent negative transfer and create more usable systems it is important to follow design standards. Some design standards are written down by organizations and regulatory agencies (e.g., ISO, ANSI, AAMI, FDA, etc.). Many standards are not written down. These are called “De Facto standards.” For example, when buying something from the web, people look for the shopping cart in the top right corner. No one declared that this is where the shopping card should be, but it is where people have come to expect it, because it is where it has most commonly been. Internet users have learned this pattern over time.

In addition to ensuring the interface does not violate standards, the development team needs to consider the users’ current goals when developing alerts. If the alert is more important than what the user is doing, than high salience alerts are appropriate. For example alerting an individual to an important meeting or taking a medicine probably is higher importance than anything else the user is doing. Conversely, reminding the user focus time is starting or that an email from an automated system is coming in may not require high salience alerts. In the medical field, there is a standard “60601,” which specifies the design of alarms. More specifically, 60601 describes prioritization of alarms and defines the salience that is appropriate for each priority. The standard also indicates that the user shall have the ability to silence alarms for situations in which alarms may be annoying or distracting.

In summary, it is important to understand that errors made by humans are often the result of normal human information processing combined with sub-optimal user interface designs. Humans take actions for reasons. The design of the user interface that takes into consideration what is known about the filter can reduce errors associated with negative transfer and mental model mismatch. In addition, applying knowledge of the filter can help to ensure that attention is directed to the appropriate aspects of the user interface at the appropriate times avoiding inattentional blindness and change blindness.

Future Articles

Future articles will
1. Explain why low demand tasks (such as monitoring autonomous vehicles) can be incredibly challenging to do.
2. How stress and task demand interact to impact human-system performance.
3. Among other behavioural and cognitive science topics.