Talking about ‘interaction’
What do we mean when we talk about ‘interaction’ in human-computer interaction or interaction design?
- What drives calls to go “beyond interaction”, or consider what might be “after interaction”?
- Does ‘interaction’ conceptually no longer articulate the right kinds of ideas when we talk about our (HCI) research?
- Does it have enough expressive power for us as HCI researchers?
- Do we as a community need a definition of ‘interaction’ to proceed coherently in doing HCI research together in future?
- … Or is there just too much baggage to the concept of ‘interaction’ — has it had its day?
The idea of ‘interaction’ is an interesting thing to return to, I believe. But thinking about it made me quite confused. I ended up with a sketch of a discussion about it.
I thought: maybe we should return to ‘first principles’, by which I mean questions like these two:
1. What jobs has ‘interaction’ done for us in our HCI research communities?
2. What might we mean to say when we talk about ‘interaction’?
For nearly 40 years, ‘interaction’ has been used in HCI and beyond as a way of talking about the myriad forms of use that emerge between people and computer systems.
Reading Alex Taylor’s ACM Interactions article “After interaction” along with some of the discussions online raised some questions about the very idea of ‘interaction’ for me. However before I address them first I’ll recap some of what I think Alex’s “After interaction” article says. Alex argues that ‘interaction’ has often been articulated in HCI in terms of “human-machine interactions”. Alex also argues that ‘interaction’ and its sense has been tied to idea of ‘the interface’ — and this has maybe hindered HCI conceptually. In other words, he says that “interaction hinges on an outmoded notion of technology in use”.
I think Alex’s use of ‘interaction’ itself might be trading on certain ways of working with the word — I found this interesting, particularly, his coupling of ‘interaction’ with ‘discrete’ as in “discrete interaction”. I take this to mean that ‘interaction’ has been lacking the kind of expressive power that might help us talk meaningfully, deeply about the embedded and fluid ways with which we are implicated in (or “entangled” with to use his terminology) technologies in our everyday lives. Alex also ties ‘interaction’ conceptually with the materiality of the user interface, arguing that ‘interaction’ as a concept has led us to “concentrate our attentions on the interface” to the exclusion of other things.
Finally, I noted that the call sent to “HCI after interaction” workshop participants argues that the concept of ‘interaction’ implies a long “assumed binary of ‘user-computer’”. So that’s another way of expressing similar problems with ‘interaction’.
Now, on to the first question where we ask “what jobs ‘interaction’ has done for us” — i.e., as a matter of our discourse, our academic talk, and so on.
Firstly we could say that ‘interaction’ has been and perhaps still is a usefully under-defined concept for HCI.
When we talk about ‘interaction’ in HCI communities we can and do use it to say a great many things — things that may conflict and be incompatible ways of saying ‘interaction’. We can say that someone tapping a touch screen is ‘interacting’, just as we can say that posting on social media is ‘interaction’, just as we can say that someone being tracked by their location is ‘interacting’ with a system, just as we can say ‘interactions’ are taking place with / around / through technologies embedded in the social life of the home, just as we can say someone hiring a Boris Bike is ‘interacting’ with a network of systems and data and other people and — even things like ‘political’ and ‘ethical worlds’.
I think the concept of ‘interaction’ also brokers relationships between a range of diverse research communities which dip into the HCI cauldron at some point or another (to mix some metaphors). For instance, one way of talking about ‘interaction’ in HCI is a kind of software engineering oriented way, which emphasises the parallel workings and misalignments of the user and the machine (makes me think of Suchman’s studies here). The engineering sense of ‘interaction’ is framed in terms of the computer’s technical needs of formatted input, output, events, interrupts, etc. This is a nice point I think Alex makes about Englebart and the Mother of All Demos — talking about ‘interaction’ like this is about bringing the machinic requirements to the foreground.
But, I think there are other disciplinary ways we talk about ‘interaction’ too. For instance, psychologists and sociologists of different flavours and persuasions have added their own alternative and sometimes incommensurate ways of talking about ‘interaction’ in HCI — and of course these then become ways of talking about ‘interaction’ into which technologies become enmeshed when they hit the HCI community. For example we might consider how ‘interaction’ can be a way of speaking of (on the one hand),
1. a model of stimulation and response between people,
which we might contrast with,
2. ‘interaction’ as an interpretive process performed by members of social groupings.
There are of course many more examples like these. The point is that there are many ways of talking about ‘interaction’ which can be ‘at play’ at any time in no distinctly differentiated way when we talk about it in HCI. ‘Interaction’, being a promiscuous concept in this way, is probably both good and bad: it fuels a vibrant HCI community but at the same time can submerge perspectival differences.
Next: the second question, about what we might even mean when we talk about ‘interaction’.
I think we have sometimes forgotten to keep in mind that ‘interaction’ is a metaphor that is doing some potentially interesting but confusing things for us by blurring the social and the technical.
I came to talk about this with Barry while doing empirical work together on how social media use is embedded into everyday life.
My point is that when we say technologies, systems, devices are ‘interactive’, we also must necessarily embed them within mundane social order — i.e., our social interactional world. In other words we leverage ordinary understandings of ‘interaction’ to talk about what people do with computational technologies at the selfsame time as we might ordinarily speak of what people do with one another.
This socio-technical blurring of ‘interaction’ — this drawing of the idea of interaction from our ordinary language — then suggests a relevant family of ‘interaction words’ like ‘response’, ‘react’, ‘alert’, ‘remind’, ‘interrupt’, etc. These are all things we might say machines also do.
But, we have to keep in mind these are ways of talking about machines. And these ways of talking about machines are grounded necessarily in ordinary language. This means that these ways of talking about machines, this family of ‘interaction words’, borrow from the everyday sense.
So, we might ask, what does it mean for a machine, a computer, a program, an app, a system, a bot, an agent, and so on, to ‘respond’? We say these things might ‘respond’ to us but is this a ‘response’ in the social, human interaction sense or some other sense? What does it mean for us to use the metaphor of interaction to talk about and ascribe things like ‘responses’ to technologies — things which ordinarily we might say are things that people do?
At this point we could think about this as a problem of language in use. For instance, I think Ryle talks about something similar when he speaks of philosophical confusions around ‘thinking words’ — so, how we talk about things like our ‘intentions’ or our ‘beliefs’ in ordinary language compared with philosophical programmes to formally locate or define ‘belief’ or ‘intention’. I’m left wondering whether ‘interaction’ and the way we talk about it is part of a wider set of troubles around how we talk about machines in general. Take for instance the idea of calling machines ‘intelligent’ and compare it with how we talk about ‘intelligence’ in an everyday sense. Do we want ‘interaction’ to be usefully ambiguous or will we get caught up, and confused about the difference between the family of ‘interaction words’ in ordinary language and their metaphoric application to things like machines?
Maybe we can try to sort through these language confusions: when we say a system ‘responds’ to us does this leverage the methods of ‘response’ that people employ in everyday social interactions? Are they the same? Are they different and how might they be? Maybe we should try to describe them? It suggests that we could take a closer look again at ‘interaction’ and the very idea. It feels like there is a lot of this ground that has been left unexamined with HCI’s expansion.
In closing, I think there might be value in rediscovering ‘interaction’ as a concept. Perhaps we might be a bit more cognisant of the multiplicity of ways in which ‘interaction’ — perhaps in an often confused way — lets us talk about what is a massively varied phenomenon. In this way I think many of the valid concerns expressed by Alex’s piece can be addressed in an inside-out, ‘interaction’-first way and not necessarily by doing away with it.
This piece was originally presented at a Microsoft Research and Mobile Life workshop hosted at MSR Cambridge by the Human Experience & Design group on the 9th of March 2016. The title of the workshop was “HCI after interaction”; it was organised in response to Alex Taylor’s ACM Interactions article “After interaction” and the subsequent discussions that ensued on Alex’s blog here: http://ast.io/back-to-interaction. I have adapted and expanded my talk slightly to work more clearly on this blog.
Originally published at notesonresearch.tumblr.com.
