Through the development of a speculative design piece, Office Humour, we explored the culture that might emerge when human data is de-humanised and agency is restricted in the age technological advancements. Especially, in the light of the tech-based responses to Covid-19, our work serves as a reminder that stronger collaborations between humans and machines need to be carefully designed [1][2][3][4].
Insights gathered from this experiment fed into research around human-and-AI collaboration and design as part of AI systems.
Part 1: Office Humour
The democracy of AI tools
In technology design and development ethnographic methods are being used alongside more quantitative methods, to offer new ways to investigate and analyse questions around human behaviour — while supposedly placing human needs at the centre of the design process. At the same time, tech giants, developers and data scientist are contributing to a fast growing repository of ready-made and pre-trained advanced analytics models. Quoting the Harvard Business Review [1]:
“[…] nontechnical users have tools at their disposal that can deliver data-based insights without involving analytics specialists, including data scientists.“
As a result, a much larger group of non-experts has gotten access to deploy data science as an off-the-shelf tool which enables them to interpret new kinds of data.
Quantifying humans
With this new power, practitioners are enabled to process a variety of data, including images, written text and sound recordings. But that also comes at a risk. As most of these data types are a by-product of human communication, measuring what up until now have been purely human outputs at a massive scale can obscure the intuitive human elements of it by treating them purely statically. Alike, using great volumes of data arbitrarily in research analyses can lead to spurious conclusions when falsely treated as a proxy for human behaviour and emotion. The same consequence can occur when analytics methods are being applied without understanding the greater depth of a statistical model.
While researchers might have the best intentions, it is possible that user knowledge could be abused if handled carelessly. Similarly, on an organisational level stockpiling user knowledge can create a power imbalance which can work against the user’s needs.
Agency and AI systems
“What does it mean for human agency when the most human elements of ourselves can be benchmarked? What are the differences between working with AI systems and being a subject to one?”
As designers in a work space that sits between innovation and the application of AI technologies, this is a set of questions that has greatly impacted our design process. It is the same set of questions that has led to the development of AI design principles beyond the HCI community, and a research stream that has evolved from a Fjord Trend into its own paradigm: Designing Intelligence.
Discussions around agency, ethical use of data and human-machine interactions on an individual level and within larger systems concern not just AI engineers and designers in Technology companies. These are discussions to be had with communities and anyone who makes use of technology.
Speculative design and the making of Office Humour
In order to engage in critical conversations with a broader audience as well as research reactions to being subjected directly to an algorithm, my colleagues James O’Neill, Francesco Pini and I built a speculative design piece called Office Humour.
Office Humour imagines a future in which emotional responses, particularly laughter, are collected about users and used against them by recording and monitoring their behaviour.
The main algorithm is based on a neural network that takes live measurements of laughter from its environment and places them within the narrative of a satirical productivity product. Participants have the opportunity to interact with the product in two ways. As they do, they experience how laughter — a very natural and personal sound — can be turned into a data point and used to monitor human performance in an inhuman way.
Through the means of a live data visualisation, the piece also tells a story that encourages participants to reflect on the sorts of data they gather in their own work and the purposes to which that data may be put in the future.
In total, Office Humour consists of three parts: the laugh detector, a can of laughter and an AI generated joke book.
The Laugh Detector
The laugh detector classifies ambient noises into laughter and no laughter and prints the result in real-time using an open-source Tensorflow classification model, categorising laughter verses non-laughter. A raspberry pi computer samples the sound from the environment every three seconds, classifies it, and renders a visualisation. The visualisation is printed by a thermal receipt printer joining to the previous visualisation. The participants can see their immediate results as well as how their interaction has been translated into a data point visually. The result is a live trace of the laughter in the environment which slowly begins the fill up the space it’s in. Making the intangible data material and visible.
To enhance the interaction participants can make use of the canned laughter to deceive the detector or the joke book to produce different results.
Canned Laughter
The Can of Laughter is an interactive laughter device in the from of a drinks can. Upon pushing the button on top of the can the sounds of stock laughter emerges from a speaker on the bottom. Inside the can is another raspberry pi computer with a sound hat, and battery pack. Participants were encouraged to use the stock laughter to fool the laugh detecting algorithm, and in doing so subvert its agency.
K.I.’s Dictionary of Humorous Anecdotes
The Joke Book, designed and edited almost completely by a text generation neural network. The neural network was trained with a large corpus of pre-cleaned jokes and anecdotes. The results are presented in the book unedited, only cleaned of profanities.
Reactions and perception
An interesting side effect was that interacting with the fake laughter often sparked genuine laughter from the participants. In the end, no actor is immune to the effect of their own actions, and the line between fake and real is increasingly blurring.
But although the piece invites participants to engage with it in a playful and curious way, the resulting conversations were rather serious and reflective. Researchers who we met at EPIC in 2019, an ethnography and design research conference, opened up about their research practices and types of data they use. These conversations pointed out a poignant discomfort about human-machine interactions that lack (human) agency, transparency about the purpose the data is collected for, and the kinds of personal data that human-machine interactions are feeding off. However, above all the piece demonstrated the need to research and redesign these interactions in anew.
Part 2: Designing Intelligence
Design as part of an AI
It helps to look at the problem space through a systemic lens, instead of speaking about human redundancy in the light of artificial intelligence technologies or labelling AI technologies as flawed. Even with the efficiencies brought by technology, the scale of the problems effecting our world means that humans will likely increase their role in the workplace, working along with AI.
We can shift the focus of these conversations towards human-machine collaboration and use it to address intractably difficult problems. The future of work therefore is a collaborative system of humans and machines, constantly negotiating agency and control — this is what is also known as ‘Centaur Systems’ [5]. Designing these Centaur Systems involves reframing AI as a teammate, rather than a tool.
To incorporate this new paradigm we propose the following:
Organizations must move beyond the automation mindset with better tools, more careful consideration of AI’s economic and social effects and a commitment to designing for human intelligence and optimizing the relationship between people and machines. As this shift accelerates, organizations will need new, systemic approaches for unlocking the full potential of humans and AI working together.
Moving forward: the impact of design
We’ve identified three areas where this will have dramatic impact:
Firstly, AI can be used to create experiences that are not just personalized but help us extend our perceptual capabilities, enhancing our vision, extending our understanding, and making us better learners. In short, enhancing the human experience.
Secondly, empowering people in complex systems by mapping systems relationships and designing interfaces that make the logic of AI visible and open to human direction [6].
Thirdly, envisioning new products and services. As AI is applied to more complex activities like simulation and decision support, innovation will accelerate the design and innovation of new products, services and even entire business models [7].
AI’s ability to problem-solve in the face of incomplete information is a big step toward making it applicable to real world business challenges [8]. But technology development alone will not solve for all problems. Moving forward the input of design in areas across the board will become increasingly important and it will be even more crucial to pair human and machines as collaborators.
Part 3: UX Design Award 2020
We are humbled and excited to announce that Office Humour has won a nomination for the UX Design Awards 2020 #UXDA20.
References and footnotes:
[1] Financial Times: Harari, Y. (2020). The world after coronavirus.
[2] Society-centered Design: IF (2020). Society-centered Design.
[3] Harvard Business Review: Deloitte (2019). Democraticing Data Science in your organisation.
[4] Reuters: Busvine, D. (2019). Rift opens over European coronavirus contact tracing app.
[5] MIT Press: Case, N. (2018). How To Become A Centaur. Journal of Design and Science. https://doi.org/10.21428/61b2215c
[6] Example: Tokyo’s Railways Technical Research Institute (RTRI) and the Swiss Federal Railways (SBB) are researching the use of neural networks and AI applications to help dispatchers and controllers combat train delays and optimize train scheduling in their respective complex and dense railway networks.
[7] Example: Designer Philippe Starck has been collaborating with Kartell using their generative AI design software to create the first chair designed through human/AI collaboration.
[8] Example: AlphaStar –AI developed by DeepMind that can beat professionals at complex video game StarCraft II.
Frauke Hein is a Data Designer with Fjord at The Dock. Her work combines analytics insights with design practices to build digital experiences.
James O’Neill is a Systems and Service Design Lead with Fjord at The Dock, Accenture’s global centre for Innovation. His research focuses mainly on the human experience with AI enables systems.
Many thanks to our fellow team members at Fjord and The Dock for sharing their knowledge and experience around this topic, and for valuable feedback on this article, especially Francesco Pini and Daniel Dallago.
