Action, interaction, reaction
First published in Blueprint magazine (UK), August 2001
Coming up fast from nowhere, interaction design is becoming the discipline that influences almost any decision a designer — and a client — can make. The implications for the world to come are legion. Nico Macdonald introduces the issues, areas of activity and personalities in the field, explains why it will soon be everywhere and supplies information sources
Interaction design, the best term we have to describe the skills we have acquired to mediate our interactions with the digital and networked world, will be one of the key practices of our century, and executed well will help realise the potential of the network society that has been tantalisingly hinted at in the past five years. If you thought Changing Rooms and Space magazine made design mainstream, you haven’t seen anything yet.
Of course we have always interacted with products, or tools as we called them before we invented the consumer society, but the past 20 years has seen a qualitative change in the kind of products we interact with, as the things we manipulate have become digital and increasingly flow over networks — that laptop on your desk, those tricky ATMs, all the wretched voice-driven customer (dis)service lines, London Underground’s fearsome touch-screen ticket machines.
And still we are only in the foothills of possible interactions. Increased computing power in smaller, less power-hungry chips creates the possibility of enhancing and creating many products beyond the PC, while the proliferation of the internet and wireless networks allows broad connectedness. “Interaction has moved beyond the PC,” comments Colin Burns [see ‘Read on…’], director of Ideo’s London office, noting that this produces different scales of interaction. “Architecture is about body-sized, furniture about hand-sized, and PCs about finger-sized interactions.”
And now chips are more flexible, observes Durrell Bishop [see ‘Read on…’], who with long-time collaborator Andy Hirniak recently joined Ideo in London. “The interesting thing is not power of chips but single-purpose chips, for internet access for instance, and reprogrammable chips that can be used to cheaply demonstrate a concept.”
Related to but beyond these developments, society — at least in the major economies — has moved towards services. “Services are increasingly digital in delivery, and this allows new interaction possibilities,” argues Michael Andrews, a London-based interaction designer. “Increasing value-added intelligence in products means interactions are no longer predefined, but must be defined by the user to gain the full value of experience.” Successful and satisfying interactions with information technology are becoming ever more significant.
What is interaction design?
Many of the concepts now discussed as interaction design have been around for years, but they tended to be embraced by disciplines such as ergonomics, psychology and human factors, which had other fish to fry. The mechanical and electro-mechanical roots of systems would determine some basic relationship between a user’s action and its outcome. These tools tend to be used by experts, and their scale meant that human-sized interactions were of primary importance. Then we invented computer games, the PC and VCR, and (after a pause for breath) mobile phones — metaphorical black boxes with inputs and outputs unrelated to perceivable mechanics.
Back in the early Eighties, when a green, glowing screen of characters passed muster for a user interface, Ideo (the successor to Bill Moggridge’s IDTwo) was working for GRiD systems on the design of the first laptop computer. Bill Verplank [see ‘Read on…’], a human factors engineer at Ideo, realised that interface design was where the real interaction was, although this was still the domain of GRiD’s engineers, not Ideo. Later he started calling it “interaction design” instead of “user-interface design”. “As soon as there are mapping and modes, built-in behaviour, programmability, that’s when really interesting things start to happen,” he observes. “How you manage the complex interactions is interaction design.”
A useful model for understanding interfaces was captured by full-time guru Don Norman [see ‘Read on…’] in his 1990 book The Design of Everyday Things. Norman notes that a good interface to a tool has four elements: It should be visible (the user should be able to see its current state); easy for the user to form a conceptual model of the tool; there should be a good mapping between the interface and its functions, and there should be feedback to the user on the result of their actions.
Issues and concepts
The field of interaction design presents designers with challenges that, due to its virtual character, have no counterparts in traditional design. Lancaster University Professor Alan Dix notes the irony that while “traditionally one might architect or design an artefact to achieve certain design goals with given materials in the physical world, software interface designers end up designing the materials and architecting the world”.
“The designer must create a myth from the bottom up so that the user can build the cause-effect chains that are more easily, and sometimes automatically, afforded by physical artefacts”
University of Sunderland Professor Gilbert Cockton [see ‘Read on…’] (who dons a more business-like hat as project director of the North East’s Digital Media Network) sees the digital as unique in another respect. “Everything’s just too free, which means designers need to be more careful and cautious — the medium won’t protect you at all as it can in traditional design disciplines. The designer must create a myth from the bottom up, so that the user can build the cause-effect chains that are more easily, and sometimes automatically, afforded by physical artefacts — such as a corkscrew which affords turning and gives immediate feedback about the state of the cork and the screw”. “Unlike many existing artefacts,” he notes, in the digital world “…the user’s response is experienced by the artefact and transforms it”.
‘There has been a tendency to believe artefacts in the physical world can be swapped for digital equivalents … but it is false. The digital world will enhance, not replace, the real world”
While the digital medium is clearly novel, there has been a tendency to believe that artefacts in the physical world can be swapped for digital equivalents. These ideas have clear appeal to clients, given the relative cheapness of developing digital products, but it is a false position. The digital world will enhance, not replace, the real world. Colin Burns reports that Ideo has been working on “scenarios for binding real and virtual experiences”. One way of achieving this is with RFID tags — barcodes that work in three dimensions, a possible for Rem Koolhaas’ Prada stores — but Burns asserts “they have to work naturally with the space”.
It turns out that the physical world has quite a lot going for it when it comes to interaction design. Durrell Bishop argues that the form of a product gives it meaning but that this meaning doesn’t have to be conveyed by an iconic form, as he demonstrated with his networked products piece for the Royal College of Art’s 100th anniversary show. He contends that physical products also have many layers of communication (”the embodiment of the potential of the thing in the world”) that are lost if they are confined to the digital medium.
Rather than stick a screen on every object, or converge many objects in to one (”adding functions till you can no longer find the mental boundaries and perceive the product”), Bishop argues for soft and hard representations to work together. One of his network products is a CD player represented by a “hard” object in to which CDs are inserted vertically but which has only three buttons (back, forward and eject). Bringing a “soft” flat panel display to this adds an interface with richer functionality.
Interaction design must also consider people’s location when they need to perform a particular task. Burns describes the recently launched Wizard Web Signs, a product of an alliance between Steelcase and the UK-based Appliance Studio, and a collaboration with Ideo. The product addresses the scheduling and re-scheduling of meeting rooms in an office with the familiar scenarios of someone needing to find the person who booked the next meeting as theirs is running over, or the group that decides it wants a meeting, hasn’t booked a room and needs to grab one. The Web Sign is hung at the door of a meeting room displaying, via a connection to the office electronic diary, what is and will be happening there and who booked each meeting.
It is at the door deliberately, as that is where room negotiations take place (the person running one meeting physically defends the space their team has occupied). “You have to mediate people’s normal interactions,” argues Burns. “As soon as you move from that you had better be thinking pretty hard.”
Mirroring the discussion about hardware and software being complementary, much debate in interaction design has been over how computers, and computer-powered objects, might interact with us by assuming human characteristics. Bill Verplank refers to this as “the anthropomorphic distraction”. “There is a lot of loose talk about cyber-this and cyber-that. But people haven’t really faced the practicalities,” he maintains. “There was a grand promise that things would be human-like, but when we had cars talking to us it wasn’t very satisfactory.” He argues for objects that are complementary to and extend us.
Utilising “intelligence” in computing to help users has been a cyclical fashion since the concept of artificial intelligence arose in the early 1960s. The best known application is the ubiquitous Microsoft Help paper clip, though its fame appears to be a function of its failure to address the problem of help systems adequately.
At this year’s Computer-Human Interaction conference in Seattle, local resident William H Gates III asked a colleague to demonstrate the “priorities application” he has been working on to help people manage the appointments and tasks that flood in from all sides. While his solution is technically marvellous, Burns is dubious about it from an interaction design perspective, arguing that computers should mediate, not automate, our activities. “You cannot successfully automate these deeply nuanced social mediations,” he contends, suggesting that to solve such problems requires “going out to look at real people and what they are doing”.
Interaction design is currently moving beyond its roots to address a host of new challenges. “We have learned a lot about interaction design from software, now we will look at products and architecture,” notes Bill Verplank, though he acknowledges that we will have to move on from the graphic user interface, just as the “character-only interface was adequate for spreadsheets”.
“Almost any interface has to be an improvement on 12 buttons, a rocker, a red and green button and a black and white postage stamp-sized screen, but the options are restricted by weight and size (which are fashion issues as much as anything)”
Mobility is presenting some of the major challenges for interaction design, particularly considering that a mobile phone is the birthright of most kids in the developed world, and even their great grandparents have discovered value in them. Almost any interface has to be an improvement on 12 buttons, a rocker, a red and a green button and a black and white postage stamp-sized screen, but the options are severely restricted by weight and size factors (which are fashion issues as much anything). Voice is the oft-cited interface of the future, though the difference between recognising words and understanding commands and concepts (let alone tone of voice) is often missed. There are also broad social issues about when and where a voice can be raised.
The corollary of mobility is location, and designing interactions that are sensitive to your location (and beyond that, to your current activity) is a major challenge. Finding a way to prevent mobile phones ringing when you are at the cinema, while leaving their owner in control, is a major challenge in itself.
Mobility is often associated with the user engaging in other activities, and when that activity is behind the wheel of a car the development of good interaction design gets just a bit more serious. Digital products will increasingly be used in environments that are physically or sensually restricted (from a seat in a plane’s economy class to the factory floor) and the challenges presented will make software design look like child’s play. New York-based Digital Image Design’s project to design wireless handhelds for Goldman Sach’s traders had to address a restricted and pressurised environment, while preserving the character of the traders’ established way of interacting with their order tickets and execution forms.
Beyond but including the mobile phone, Andy Proehl, Interaction Design manager at Sony’s San Francisco Design Center, considers “appliance computing” to be a major challenge, citing products such PalmOS handhelds, iMode phones, and his company’s consumer-oriented Airboard Web pad that allows for e-mail and Web access in the home and for interaction with your TV. Sony’s vision represents current practice but appears to be in opposition to Durrell Bishop’s concept of network products. However there are clear dangers in trying to combine too many devices into one, and anyway, how many things do you want to do at once?
An increasingly significant issue is presence. Damon Clark, who worked on a project in this area at Nortel Network’s Design Interpretive in Harlow, Essex, picks up. “This is the perception of you on the network by others — animal, vegetable or mineral. This manifests itself in regular communications such as ‘who can contact me when’ and who I permit under what circumstances to bust through my ‘communication buffer’. It is kind of best practice for a virtual secretary. Some of the rules for this are implemented by the user — others are triggered by the network.” These problems have only just begun to be addressed in the office environment (with Microsoft’s priorities application) and mobility and other environments only add complexity.
Anyone who has kicked off a search on Yahoo! only to receive five figure results, with no clue how to proceed other than by starting at the top, may have thought about visual solutions to information searching and manipulation. This was a problem addressed by Ben Shneiderman’s [see ‘Read on…’] team at the University of Maryland Human-Computer Interaction Labs, albeit in another context. It worked on the idea of direct manipulation of a number of data axis to allow the user to see the scope and scale of the results of their choices in real time. It is now being used for product development in the real world (in its commercial guise as Spotfire) by most of the big pharmaceutical companies.
There are many related information visualisation problems to be addressed. One is how to represent the status of information related to you; as we shop around more, how can we represent comparative prices and costs? Given that we make many small transactions each day, how can they be visualised? As more information about us is stored on the network, how do we show who has access to it, and how can it be corrected or changed? How do understand and manipulate information about our interactions with others?
A related challenge is to help manage our interaction with systems at a distance, or ones that we want to act in our absence. Bill Verplank sums it up eloquently: “When I do something, what does it affect? What kind of monitoring do I have for what’s going to happen? What predictions and simulations can we use? What models are there for trying it out? How do I know whether something I intended to happen did happen?”
“How do you describe interaction of the product?” asks Durrell Bishop. This is the next logical step from solving an interaction design problem. Once we have described it we may be able abstract a solution so that it can be applied in similar situations. One need for this, albeit a trivial one, is on the web where the interaction with forms and search engines appears to be re-designed every time the problem is addressed.
The challenges of interaction design for designers are great, but for business they may be even greater. “How are we going to adequately design our interaction with technology when we’re swimming in it — and when companies don’t even realise how important interaction design is?” asks Sally Beardsley, a Copenhagen-based interaction designer.
Chris Pacione [see ‘Read on…’], a former design professor at Carnegie Mellon and co-founder of the Pittsburgh-based BodyMedia, has observed the rise of the super-informed consumer and the levelling of the competitive landscape for companies. He argues that: “Enabling quality interactions between the public — consumers, customers, partners — and their products and services is paramount if they want to stay in business.”
Pacione rails against the “silo mentality” of “companies who merely engineer or design or market and do not understand fully that while what they are making may be a gadget, or a cool piece of software, what they are ultimately being judged by is the quality of the interaction these products and services provide” — which is determined by how “useful, usable and desirable” they are.
He also has some thoughts for our friends the brand gurus, observing that brand loyalty becomes less of a competitive advantage when geography and access to information is no longer a barrier. “Brand is more intimately tied to experience and interaction than marketing, and is now a company problem.”
“When someone has a problem they blame themselves rather than the product but they will often buy another solution from the same”
While he doesn’t dismiss the importance of the marketing message, “if you can’t deliver on that promise, your brand is screwed”. Delivery is further confounded by the possibility that one company’s service may be tied in to another’s hardware, a point picked up on by Gitta Salomon [see ‘Read on…’] of Swim Interaction Design. When a user has a bad experience in this situation, which company do they blame? And whose brand is enhanced by good experiences?
Educating the next generation
It is one thing to know we need more people with interaction design skills, but another to find, educate or train them. “There are precious few ‘trained’ interaction designers and the web has created incredible variety,” observes Sony’s Andy Proehl. “This also means that there are a lot of people re-inventing the wheel.”
One of the first, and most celebrated, workshops for wheelwrights is the RCA’s computer-related design course, established by Gillian Crampton-Smith [see ‘Read on…’] in 1990 with input from, among others, Ideo’s Colin Burns. CRD set out to be deliberately multidisciplinary, and to create students from a wide variety of backgrounds who could think out of the box and consider the social effects of technology. Alumni include Durrell Bishop and Andy Herniak, who went on to found Itch, and collaborators Tony Dunne and Fiona Raby, recent cover stars of FT magazine The Business.
Crampton-Smith recently left the RCA to establish the Interaction Design Institute Ivrea. While following in the spirit of the CRD course, Ivrea will explore business in addition to design and technology. Crampton-Smith believes that today there is an “art” in imaging new business models, and is also aware that, partly because of their broad education, design graduates often move in to strategic roles in companies and need to be equipped to learn for themselves.
The courses offered by CRD and Ivrea are both post-graduate, but Karen Mahony [see ‘Read on…’], founder of London-based all-media consultancy Xymbio, comments that there is a parallel between the education of interaction designers and architects. “You wouldn’t train someone to become an architect with just a one- or two-year post-graduate course,” she argues. “Similarly, there just isn’t enough time to teach someone how to become a fully qualified interaction designer in the same two-year period.” To purse this belief she is currently investigating the possibility of establishing a school in Prague.
There is a lot of work to do, some great challenges in evolving interaction and making it a core design skill. It will be a challenge for business to work out how to manage it better than it has in the past, and this challenge has a return on investment. Most new products and services launched fail, or are only qualified successes. This is not because the product is technically bad, but because it doesn’t provide a high quality of experience. Part of the challenge will be for businesses to redesign themselves to better support their products and services. As Chris Pacione observes: “There are many points at which a consumer or customer interacts with a company and its products and services. Everything, from the website to the out-of-box experience, to use of the service or gadget, customer service response, to how we answer the phone, are interactions, and therefore can be looked at as a potential design problem which can be improved upon.”
We should also be more aware when we are using products and services with poor interaction. Harold Thimbleby [see ‘Read on…’] notes that “when someone has a problem they blame themselves rather than the product” but they will often buy another “solution” from the same company.
Durrell Bishop believes that we can go one stage further and that non-designers (“the people who run the pub around the corner”) should be able to master the new means of communication both at an interaction design and a technical level, in the way that the creation of printed material has become a basic skill.
Interaction design is the great design discipline of the 21st century, and we should expect our interactions mediated through the digital world to be as seamless as they appear in the Hollywood imagination — but with more satisfying ends.
[image to come] Ideo’s work for Handspring’s camera module installs the software automatically
[image to come] Interaction Design Institute Ivrea
[image to come] Durrell Bishop’s “Comment” display at the Science Museum
[image to come] Smart handhelds for Goldman Sachs by Digital Image Design
A subjective and partial tour of the people, organisations, publications and events related to interaction design.
Bill Verplank is a member of the steering committee of the Interaction Design Institute Ivrea. He was one of the team that designed the Xerox Star at its Palo Alto Research Center, and subsequently worked with Ideo and at Interval Research. He is co-chair of the Designing Interactive Systems 2002 (DIS2002) conference, which takes place in London next June, and teaches at Stanford University. (See notes for one of his lectures on interaction design.)
Gillian Crampton-Smith is the first director of the Interaction Design Institute Ivrea, whose ‘Explorers Club’ of adviser includes many of the most original thinkers and doers in interaction design. She founded the Royal College of Art‘s Computer-Related Design course which she ran for ten years. She was interviewed about her work in New Media Creative (December 2000) and Create Online (March 2001). Her essay ‘Humanising Technology: could do better’ is included in Design Renaissance, edited by Jeremy Myerson (Open Eye, 1994). She is involved in the DIS2002 conference.
Irene McAra-McWilliam, formerly at Philips Design in the Netherlands, has taken over as director of CRD. She delivered a talk entitled ‘One-D to Ten-D: The Evolution of the Interface’ at the Doors of Perception Lightness conference last year and spoke on this theme and Philip‘s Design‘s Living Memory project at the SuperHumanism conference this year.
Colin Burns is director of Ideo in London. Ideo was founded by Bill Moggridge and David Kelley, and Bill Verplank was an early employee. Moggridge is a keynote speakers at DIS2002.
Donald A Norman is president of UNext Learning Systems and Professor of Cognitive Science at the University of California at San Diego. He was an Apple Fellow, heading up the company‘s Research Laboratories, became an executive at Hewlett-Packard, and went on to co-found the Nielsen Norman Group with usability guru Jakob Nielsen. He is best known for his book The Design of Everyday Things (MIT Press, 1998 ISBN 0262640376 Amazon.co.uk) in which he started a discourse on ‘affordance’ , which has continued ever since. He also published The Invisible Computer (MIT Press, 1999 ISBN 0262640414 Amazon.co.uk) which deals with some aspects of interaction with information appliances. His personal Web site refers to ‘just noticeable differences’ .
S Joy Mountford also worked at Apple where she created and managed its celebrated Human Interface Group. She was subsequently at Interval REsearch (which was established by Microsoft co-founder Paul Allen) and now runs her own consultancy, idbias, in Silicon Valley.
Brenda Laurel is a designer, writer and researcher who consulted extensively with Apple and worked at Interval, which spun of Purple Moon, a transmedia company for tweens. She teaches on the Media Design Program at Art Center in California. She was commissioned by Mountford to edit The Art of Human-computer Interface Design(Addison-Wesley, 1990 ISBN 0201517973) which includes essays by Don Norman, Ben Shneiderman, Bruce Tognazzini and Gitta Salomon). She is also the author of Computers as Theatre (Addison-Wesley, 1993 ISBN 0201550601).
Gitta Salomon worked in the Advanced Technology Group at Apple Computer and then at Ideo, after which she founded the San Francisco-based Swim Interaction Design Studio. She studied at the MIT Media Lab.
MIT’s Media Lab is home to Marvin Minsky, author of one of the key books on artificial intelligence The Society of Mind (Simon & Schuster, 1988 ISBN 1559406828). For a broader view on the implications of computing for interaction design see Lab director Neil Gershenfeld‘s When Things Start to Think (Hodder and Stoughton, 1999 ISBN 0340750391) and The Unfinished Revolution: Human-Centered Computers and What They Can Do for Us (HarperCollins, 2001 0066620678 Amazon.co.uk) by Michael Dertouzos, director of MIT‘s Laboratory for Computer Science. David Small studied under John Maeda at the Media Lab‘s Aesthetics and Computation Group (originally studying with Muriel Cooper at the Visual Language Workshop) and founded Small Design Firm in Cambridge, MA. Alumnus Jennifer Tidwell has done some interesting work on interaction patterns, a theme that builds on work originally done in architecture. The Media Lab has an outpost, MediaLabEurope, in Dublin and is now active in India.
Jef Raskin was a key player on the Macintosh project at Apple in the early 1980s. He is author of The Humane Interface: New Directions for Designing Interactive Systems (ACM Press, 2000 ISBN 0201379376), a serious look at software interface design issues.
Harry Saddler worked in interaction design at Apple and subsequently joined MetaDesign in San Francisco. He is interviewed (along with Karen Mahony of Xymbio and Terry Swack) in issue 1 of Loop, the AIGA Journal of Interaction Design Education.
Lauralee Alben, of Alben Design, was commissioned by Saddler while he was at Apple to work on a series of desktop themes. She was one of the founders of the ACM’s interactions magazine. She wrote the seminal articles ‘At the Heart of Interaction Design’ and ‘Defining the criteria for effective interaction design’ (first published interactions May+June 1996, volume 1113).
Terry Swack is co-founder, with Clement Mok, of the American Institute of Graphic Arts (AIGA) Experience Design group, which addresses interaction design as a component of experience design, and is developing experience design theory in the context of professional practice and education, creating a richer understanding of the subject in the business world. Along with Loop (which it co-publishes with the Center For Design Studies at Virginia Commonwealth University) the AIGA publishes Gain, a journal of design for the network economy, which also addresses interaction design.
Karen Mahony is completed some of her study at the Royal College of Art (though not in Computer-Related Design) and is developing a project to establish a school and studio focused on interaction design in Prague. She is the founder and director of the London-based all-media consultancy Xymbio.
Professor Gilbert Cockton of the University of Sunderland is an adviser on Mahony‘s Prague initiative. He is also project director of the Digital Media Network and chair elect of British Human-Computer Interaction Group executive.
The British HCI Group is a special interest group of the British Computer Society which approaches interaction design from academic and research perspectives on human factors and usability. It publishes Usability News, an online publication for anyone interested in or working in the field of usability. It runs an annual conference, the 2002 event in London in collaboration with the Usability Professionals Association.
The Association for Computing Machinery Special Interest Group on Computer-Human Interaction (ACM SIGCHI) is the US counterpart to the British HCI Group. Since the early 1980s it has hosted an annual conference which attracts 2–3000 attendees, and reconvenes in Minneapolis in 2002, and is responsible for the Designing Interactive Systems conferences.
Professor Ben Shneiderman of the University of Maryland was given a Lifetime Achievement Award at the last CHI conference. His Human-Computer Interaction Lab developed the Spotfire project, building on his research around direct manipulation. He is author of one of the key textbooks on interaction design Designing the User Interface: Strategies for Effective Human-Computer Interaction (Addison-Wesley, 1998 ISBN 0201694972)
Chris Pacione of Pittsburgh-based BodyMedia was previously a professor at Carnegie-Mellon University‘s school of design. His article ‘Making Meaning’ in issue 1 of Loopdescribes his approach to teaching interaction. Carnegie-Mellon has an HCI Institute, at which Jodi Forlizzi is an assistant professor, currently working on theories of interaction and experience.
Cooper Interaction Design in Palo Alto is a leading interaction design consultancy founded by Alan Cooper, author of The Inmates Are Running the Asylum (SAMS, 1999 ISBN 0672316498), which argues that technology should work the way we think.
Professor Harold Thimbleby is Director of the recently founded Interaction Centre at University College London.
The Illinois Institute of Technology Institute of Design, directed by Patrick Whitney, offers post-graduate study that covers interaction design.
First published in Blueprint, August 2001
(Article as published. Read on… has been extended.)
‘Action, interaction, reaction’, Nico Macdonald, Blueprint, August 2001, pp44–50