Prediction & Surprise in Design
One of the main, overarching, prevailing views of the human mind in cognitive science at the moment is that it is a prediction machine, at pretty much every level of architecture. Philosophers and scientists such as Andy Clark, Karl Friston & Anil Seth describe a picture of cognition in which we (our brains, our minds), are constantly engaged in an ongoing task of prediction about both the outside, and our inner worlds. This runs counter to the more classical type of information processing that was previously thought to be going on in our minds, and counter to a lot of our intuitions as well. Understanding the general features of ‘predictive processing’ or ‘predictive processing’ can yield some interesting ideas about design.
A few decades ago, when the more classical information processing metaphor for the mind dominated the cognitive sciences, the theory was that we receive a rich and bountiful stream of sensory data from our environment, and our brains then cleverly construct a representation of reality from that, in real-time, all the time. Psychologist and cognitive scientists also knew however, that expectations heavily affect what we perceive and understand about the world- but the actual mechanisms for this were not understood. The view of the mind as a prediction engine has its roots all the way back in the 1800s with Hermann von Helmholtz, but it is only over the past 20 years or so that the theory has been fleshed out in a lot of detail in neuroscience and cognitive science (as well as in artificial intelligence).
The basic idea is that as an adult human, you know a lot about your environment, about your body about other people and so on, as a result of your previous experiences. Your brain and nervous system have been calibrated, and have learned commonalities and regularities, and complex hierarchies of likely events and qualities of objects. This prior knowledge, stored primarily in the brain, is then used as the basis from which to form predictions about the environment, one’s body and other people. These predictions could sometimes be in the form you might imagine — something like ‘if I turn this coffee mug around there will be a handle on the other side’, or ‘my stomach is going to be upset after eating these sour berries’, or ‘Jim is going to feel guilty after doing what he did’, but this is just the very tiny sliver of the top of the iceberg. These predictions also come couched in terms of probabilities, so we always (unconsciously for the most part) have some certain level of confidence in a given prediction.
So what does the rest of the iceberg look like? Well, there are predictions at every level of cognitive architecture, so you can think of it in a variety of different ways. It could be that part of your brain is saying ‘given that that tiny pixel of my visual field is red, it is likely that the surrounding pixels of my visual field will also be red’. Or it could be that given that these particular five clumps of neurons are firing in just this manner, it is 63.5% likely that these other two will also fire very shortly. Whilst data from our senses comes into our cognitive systems from the ‘bottom up’, these predictions are simultaneously flowing from the ‘top down’. Every millisecond of your conscious life, your brain is constantly predicting how every aspect of your reality is going to turn out, from tiny bits of sense data that form the edges and corners and so on that in turn form the foundation of the scene you see with your eyes, right up to more large scale social events with lots of other interacting people.
And here is the truly mind-blowing thing about this account — it is only the errors (‘prediction errors’) that actually filter through from the bottom up, rather than all the raw sense data that we previously assumed made up our conscious realities. It is only in the cases where all those levels of prediction that are flowing down from your mountain of previous experience have mistakes in them, that actual data from the outside world ‘gets in’ to your picture of reality. Now of course the reality of this kind of processing is unimaginably complex, and of course over moment is riddled with such prediction errors, because we have never experience a moment quite like this one before, but still — this picture of the reality you see, hear and feel right now is probably not quite what you had imagined. But it makes total sense, even intuitively, when you consider a few things we pretty much take for granted. We know that on our daily commute we become almost blind to our surroundings. We can arrive at work with almost no memory of the drive or train ride there. Often we feel like we know what lots of people we know will say or think about something, before they open their mouths. We act in familiar surroundings without effort, without having to think about anything, or figure anything out. This is because our well honed predictions are doing such a good job, that our brains hardly have to process any ‘new’ information. Conversely when we travel to a new place, or learn a new skill, or get surprised in some way, things can seem intense, almost like a hyper-real version of reality, and even time can seem to slow down. Our previous experience in these cases isn’t able to offer nearly as much in the way of confident protections, and prediction errors are streaming up through our cognitive systems, causing increased attention and rapid learning.
And this is where I think things get interesting from the perspective of design. Both ends of the spectrum are useful to understand, and to ‘exploit’ when designing something. In some cases we want our design to be invisible — not visually, but in terms of the attention and effort someone needs to interact with it. In this case then we want to give people as many familiar markers as possible, and design processes that fit seamlessly with how someone would expect them to work, given all the information available to them. This means designing in accordance with established conventions, giving simple and clear instructions, obvious interaction controls and so on. On the other hand, sometimes you want something you design to be completely engaging and magical — full of surprise. This could be a piece of interactive art, or a VR experience. In these cases then you almost want to do the opposite — breaking expectations at every turn, even seeming to defy our most foundational beliefs, like the laws of physics.
In many cases, you will want to strike a balance between these two extremes. You want some aspects of an experience with what you design to be magical and surprising, but also never to hinder actual processes that someone has to complete. A VR world should be mind-blowing, but controlling the menu system, and one’s progress through the world will ideally be obvious and intuitive. Always understanding that whoever uses your design will be in a constant state of prediction can help you find this balance, and design something truly wonderful.
If you want to find out about more articles from me, and updates about the book about cognitive science and design I am writing, then sign up to my newsletter! http://eepurl.com/biojpj
Originally published at iaminterface.com on October 11, 2017.
