Embodied Sketching
The other day I read an interesting paper from a couple of years ago in CHI, the most important conference on Human-Computer Interaction. The paper is by Elena Marquez Segura and her colleagues at Uppsala University. You can see her talk in the video below and the full paper link is at the bottom of the page (though it is not open access, so non-academics might not be able to get it, sorry).
Movement Interaction Design
It tackles a problem that I’m really interested in (and will one day publish my massive acadmic paper on): how to design interaction based on human movement.
There are lots of techniques that for interaction design for interfaces based around a screen, mouse and keyboard. Because the mouse and keyboard interaction are so standard, we don’t really need to think about them in interaction design, and we can concentrate on designing what is on the screen. Since this is 2D, we can mock it up with 2D drawing, so a lot of interaction design is about sketching and paper prototypes. The same goes for touch screens.
Interaction design in VR is very different. You are not sitting in front of a screen (or at least, you don’t see the two screens in front of your eyes as screens). You are immersed in a 3D world, so prototyping in 2D will not work as well.
Also, for the experience to feel real (for it to trigger plausibility illusion), you need to interact with the world as you do with the real world: using your full body movements, not just the restricted movements we commonly make with a mouse and keyboard. That means that simply prototyping the thing you are looking at (whether 2D or 3D), is not enough, you have to prototype the way people are moving.
Embodied Sketching
Segura’s paper is not actually about VR, it is about other forms of movement based interaction, but it is super relevant to the kind of interaction design we do in VR.
The idea of the paper is that a lot of design is about sketching, but, as we’ve just seen, 2D sketching won’t work, you have to sketch with your body, hence the title: embodied sketching.
What does that mean in practice?
The authors take a playful approach to interaction design. Designers physically play with possible interactions. For example, one of their design sessions was in a gym with lots of gym equipment. Their designers tried out lots of ideas: like hanging from bars and balancing on gym balls.
Perhaps the most important features is that designers have to actually, physically try the movements, not just think about them or draw them. That is the only way you know how a movement actually feels.
Another important feature is that the embodied sketching is a social experience, the designers work and play together. They all try the physical games and are together while they do it sharing their experiences. That leads to new designs as people bounce (some times literally) ideas against each other.
The paper presents 3 uses of embodied sketching.
Sensitization
It can be used to sensitize designers to body movement. This happens before the design process. Designers will (as a group) engage in some specific physical activity or experience (in this case an anti-gravity fitness class — something cool, but not as cool as it sounds), to get them to think and feel with their body more. This can lead to better designing later.
Bodystorming
Bodystorming is part of the actual design process. Designers work together to come up with new ideas. It’s name is based on the “brainstorming” technique for collaboratively coming up with ideas, but instead of just thinking and talking, the designers move and use their body (this was the example in the gym).
Participatory Embodied Sketching
Embodied sketching doesn’t have to be just for designers, it is a great way to get users involved in the design process (called participatory design). In the paper, the authors describe a session where school children invented new movement based games using a digital toy.
These are just three examples, but I’m sure there could be many more. In particular, they are all quite early in the design process, but we can use embodied ways to develop ideas and prototypes as the design develops.
Embodied Sketching in VR
None of the examples in the paper are about VR, but the general idea is very relevant. When designing for VR, we should take our body movement into account, and design in an embodied way. Paper and pencil will only get you so far, you should physically move and perform the actions in the game.
VR developers are already using related techniques. The article below describes a technique used by Schell Games called Brownboxing.
Brown boxing involves mocking up VR games in the physical world using simple props like chairs, tables and the eponymous brown boxes:
They built a telephone by putting lights and knobs on a cardboard box, then placing a telephone handset on top of it. They mocked up a water pump that needed to be repaired by writing “Water Pump” on another cardboard box and filling it with bits of hardware that needed to be combined by the tester. They effectively created the environments where some of the game’s missions would take place, in their own offices.
Brown boxing allows designer and play testers to physically experience the game mechanics in the real world.
One of the most important features is that the demoes are fast to develop, just like pencil and paper sketches. They recommend taking no more than 2 days to set up the demo. The quicker it is to make, the easier it is to change and iterate (and the easier it is for you to throw it away if the mechanic isn’t working).
Your turn
So for your next VR experience try brownboxing, or invent your own embodied way of sketching.
If you would like full Embodied Sketching paper, it is here (ACM access required):
