The best (though deep and complex) way to design novel, brilliant interfaces
I recently wrote about prototype theory, and how it supports the notion that we should abide by design conventions, or at least introduce new design patterns very gradually. Let’s face it though, that’s not where the fun, the fame and the glory lie. Truly memorable, creative and powerful design is often defies convention and introduces something that has never been seen before, but is at the same time completely intuitive. To understand how to do that, you have to understand a few things about the human mind, and this shit gets pretty deep — down to the root of what we are, and how we conceive of the world.
First up let me clarify what I mean by ‘interface’, when talking about designing interfaces. I am certainly not only talking about software or app user interfaces, but rather ‘interface’ in a slightly broader meaning of the term. ‘Interface’ should be understood here as anything about a designed object or product that both shows someone what the object/product is, what state it is in, what is happening with it, but also allows us to affect and control the object/product in some way. In this broader definition, the handle of your kettle is interface, as much as a date-picker on a web form. Okay…now we’re ready to begin.
Many philosophers and cognitive scientists have puzzled over the ages about how we come to gain new knowledge, and specifically how we can come to understand new structures and relationships of information and systems out there in the world. There are many answers to this deep and complex question, but one direction of enquiry in particular is of interest to us here. It has to do with analogy and metaphor, and something called ‘image schemas’. George Lakoff is one such cognitive scientist who, together with Mark Johnson has put a lot of effort into research into image schemas, embodied cognition, and their role in the roots of language, knowledge, understanding, and even the very nature of ‘Truth’. We don’t need to delve quite so deep, but the ideas Lakoff and others have developed in this area are incredibly valuable and powerful.
You can start off by thinking of image schemas as mental representations of structures of information — representations of systems, with parts that fit together in particular ways, and have particular kinds of relationships between them. The special thing about image schemas is that they are based on, and formed from our bodies, they way we use our bodies, and the way our bodies interact with the world around us. Our bodies and how they exist in space have many very specific properties. We lie down horizontally to sleep, we stand up to walk around, with our heads at the top and our feet at the bottom. We generally move with our bodies facing in one particular direction — ‘forwards’, with the front of our body pointing in this direction. When we go somewhere we move in space, on some kind of surface (be it a road or the forest floor). We generally have two arms, two legs, ten fingers and ten toes. We can use our hands as a container to move things from one place to another, or to give things to someone else. When we accumulate things they form a pile, which gets bigger and piles up the more of those things there are. All these things have been true for an incredibly long time over our evolutionary history, many of them from before we were homo sapiens, but all of them from before we created agriculture, or even the most basic form of civilization. Perhaps all of these things were true before we had language — certainly many of them were.
These facts of our embodiment are mirrored to some degree in neural and cognitive structures — the fact that our head is at the top of our body, and our feet are at the bottom (literally, when we are standing) is represented in our minds. This and many other of these kinds of representations stem from the very basic and fundamental functions that our brain provides — mechanisms for movement and control of our bodies, so that we can go about our business in the world, trying to achieve our evolutionary goals of surviving and mating, which for humans further subdivide into finding shelter, eating, socialising and so on, which in turn for modern humans subdivide into the effectively infinite variety of human behaviors, from cooking a perfect poached egg, to manipulating national media, to doing a one armed pull up, to playing classical guitar and so on. Everything you can possibly do in the world, including anything achieved by the use of language, necessarily involves movement of the body, and effective movement of the body necessarily requires some type of representation of that body and its parts, and the ways it can interact with the world.
The truly powerful insight is that we use these structures, these representations, these image schemas as bases from which to gain new knowledge about the world. In fact some would argue that this is in fact the only way we can gain new knowledge about the world. The basic mechanism by which we do this, is analogy. Here is an example — if I say to you that ‘the Dow Jones went up 70 points yesterday’, you will immediately and intuitively understand that the ‘size’ or ‘value’ of it increased, even if you have no knowledge of financial markets at all. The reason for that is because of the word ‘up’. It seems almost meaningless to say it, but when you think about it, there is nothing fundamental about the Dow Jones that went ‘up’ in any way at all. There is nothing about the change from 22,400 to 22,470 that is necessarily directional, other than the way humans usually think about it. And why do we usually think about it like that? ‘When we accumulate things they form a pile, which gets bigger and piles up the more of those things there are’. Fundamentally, our understanding of quantity is tied to our physical experience of it, and the way in which the physical properties of something (a pile of berries, for example), change as the quantity increases.
The same is true for our conception of time: If I say to you ‘moving forwards, we need to stop making such addictive digital products’, you immediately and intuitively understand that I am talking about the future, because of my use of ‘forwards’. Once again though, there is nothing directional about time, aside from how we humans think of it. And why do we think of it like that? ‘We generally move with our bodies facing in one particular direction — ‘forwards’, with the front of our body pointing in this direction’. In George Lakoff and Mark Johnson’s book ‘Metaphors We Live By’ they detail and analyse many examples like this and of far greater levels of complexity and abstraction in great detail — I thoroughly recommend it. Also, while these examples, and many in the book are linguistic in nature (describing how the language we use derives from image schemas), it is arguably the case that image schemas operate outside of linguistic thought, and could very well predate linguistic thought. This interesting article on Aeon argues that our imagination is truly ancient, pre-dating language, and it fits well with the notion of image schemas playing an important and foundational role.
This is an endlessly fascinating topic for me. I have only scratched the surface in my research, and only scratched the surface of my research in this article. I think though that this is enough of a foundation to push on to talk of design. I started off this article promising to reveal the best way to design novel interfaces. My argument is that the way to do this is to faithfully map the functionality that you are hoping to reveal and/or control with the interface you are designing, onto an existing image schema. The interface must be an analogy, or a metaphor, that connects the workings of the object or product to an existing image schema in people’s minds. In fact, all good design is already like this, and many good designers design like this intuitively, even if they are not aware of the details of the theory. Sliding volume controls on graphic equalizers that slide up to increase volume map on to the same image schema that allows you to understand ‘The Dow Jones went up yesterday’. They use the analogy of vertical space and distance to map the functionality of increasing volume in a hi-fi, onto our existing image schema relating to physical quantity, and our analogical understanding of increases of anything being just types of increases in physical quantity.
This type of approach also reveals both what was right, and what was wrong about the visual skeuomorphism that was so popular a few years ago. Of course it makes sense to try and use visual analogy to help people to understand how to use a new piece of software, for example by designing a piece of note-taking software like a literal paper notepad. But this misses the deeper and more powerful insight that taking notes on paper is also just a recent technology (like personal computers), with its own simple interface (a pencil and writeable surface), that is itself a kind of physical analogy for something far deeper and more innate in us. We have an image schema for information and knowledge as a physical substance — we can be ‘full’ of ideas and ’spread’ them around. So to store knowledge or convey it to others we must use a container to put it in. That container can be a sheet of paper, or an interface on a piece of software which is simply a white space with a drop shadow, denoting it as a sheet of ‘material’. We don’t need to use literal visual metaphors from our current world to achieve truly intuitive design.
This is not easy, rather as the title of this article suggests it is ‘deep and complex’, but I believe large gains in good and creative design can be made by even just familiarising ourselves with the most well researched image schemas, and by thinking about them in the context of design. With a good understanding of a couple of handfuls of our most potent image schemas and embodied conceptual metaphors, you will start to see design problems from a different perspective. You can start to see the functionality and affordances of the object or product you are designing as a little information system of interrelating parts, with a specific kind of structure. You can start to find similarities and parallels between the structure in the object or product you are designing, and one or more of our innate image schemas. Once you can do this, you can see the process and skill of design as creating interfaces that act as analogies between these two worlds — as creating maps for people to be able to intuitively grasp the inner workings of a piece of technology that is otherwise a black box to them. It is in this way that truly novel and groundbreaking, wonderful design patterns can be found, that appear to break current convention, but do so by reaching deep into our evolved beings and attaching onto the very mechanisms by which we understand and construct our realities.
Ending note: I am about to begin writing a book about this and many other magical and wonderful ways in which cognitive science and philosophy can improve our practice of design. If you think such a book sounds interesting, then please let me know in the comments, and/or sign up for my newsletter in the menu sidebar — this will help me gauge interest and get going!
If you want to find out about more articles from me, and updates about the book about cognitive science and design I am writing, then sign up to my newsletter! http://eepurl.com/biojpj
Originally published at iaminterface.com on October 3, 2017.
