Design fiction: Addressing the societal and ethical questions raised by future technologies
Today there’s more information about each of us and our environments than ever before. AI-powered systems can analyze this information captured through ubiquitous sensors and act accordingly in real-time. AI is in fact transforming all human activities by enabling products and services that are increasingly flexible, specifically targeted to us and our contexts, and the situation is well set to continue in such a way for the foreseeable future. Yuval Harari [1] addresses this point in his book Homo Deus:
“If Kindle is upgraded with face recognition and biometric sensors, it will know how each sentence you read influenced your heart rate and blood pressure. It will know what made you laugh, what made you sad and what made you angry. Soon, books will read you while you are reading them.”
This phrase paints a very clear picture of what we can expect to happen. ‘Affective computing’ is an area of HCI that has focused on the creation of machines that can understand and react to human emotions [2]. Today, we can already see such systems able to automatically identify our behaviours and how our bodies react to different situations. Smartwatches can track our heart rate, and computer vision systems can identify our emotions. The website Stealing Ur Feelings entertainingly presents how these technologies may already be used in some digital products, such as Snapchat, to recognize our feelings through video analysis to better understand our favourite content and personalize our experiences.
At the beginning of 2020, I attended the IXDA Education Summit conference in Milan, where Belgian sociologist Derrick de Kerckhove presented the concept of ‘digital twins’. Such a concept actually appeared several years ago, but was initially applied to machines [3]. The idea is that a physical product has a digital twin informed by sensors that provide information from the real-world object. This digital twin can run simulations in the virtual world and provide information that can help improve the performance of the physical product, while making predictions about potential future failures.
Kerckhove presented the same concept of digital twins, but this time applied to humans. So what happens when we get a digital twin of ourselves? We could say that such digital twins already exist, or at least ‘small digital twins’ of many of us, distributed across different platforms such as Google, Facebook or Amazon, which capture and analyse information about us. These platforms use the information to, amongst other things, personalise our experiences. But what happens if a more integrated system is developed? A system that accumulates information from sensors capable of capturing much more information about us and able to analyse it with increasingly advanced algorithms? These systems would be able to know how your body reacts to each situation or person. What if they know, for example, that you’re in love even before you realise it yourself?
We delegate a growing number of decisions to digital systems. For instance, when I open Spotify, I often don’t even select songs. I simply listen to what the algorithm recommends because it usually matches my taste in music. What if we take this to the extreme and consider systems that, by analyzing our data, know better than us what’s best for us? What happens if we begin to delegate the kind of decisions that are generally considered to make us human?
These systems are entering our lives in subtle ways as they know more and more about us. They’re capable of influencing us in ways that are difficult to grasp. As designers of products and services, our job is to define technology’s role. This means answering questions such as what should be automated and what not, how should the system and the human collaborate and which kinds of interactions would effectively enable such collaboration. These aspects, which can be traced back to the industrial revolution, are increasingly relevant today. Technology has become more complex and more widespread, infiltrating areas of our lives that were previously distinct and independent of automation. In this context, asking the right questions becomes increasingly relevant, as much as to whom we should be asking them to.
Our approach
As designers (and as people) we bring our own ethical views into what we do and, although we may have the best of intentions, we cannot base our decisions solely on our beliefs when designing products and services that will affect the lives of others. Our approach towards this challenge is to take the role of facilitators, giving voice to those involved and potentially affected by the products we envision to reach a compromise between what may sometimes be very contrasting views. In practical terms, we achieve this by bringing the aforementioned questions to the forefront during our design process through different participatory methods, depending on the stage of the process.
Participatory design sessions involve stakeholders in the creative process, actually embedding their beliefs directly into the ideas and concepts generated during the activities [4]. Moreover, these sessions help participants understand one another’s position (i.e. designers, engineers, company officials and users) and work towards a unified vision rather than pushing their personal agendas in their isolated roles. Also, through semi-structured interviews and other ethnographic studies we can engage them in conversations that can help us understand their activities and values.
Lately, we also began to integrate new activities that leverage fictional elements to expose the participants’ ethical perspectives towards the topics of interest and assess potentially unexpected consequences of the products and services we’re designing. Introduced by science fiction author Sterling [5], researchers have relied on design fiction and other methodologies to better understand prototypes’ societal impact and values around future technologies [6,7,8]. One of the speculative methodologies we used during one of our more recent projects is called futuristic-autobiographies (FABs) [9]. This method, inspired by design fiction, helps elicit values and perspectives from participants such as prospective users, designers and researchers, on future technologies. It presents short fictional stories that illustrate a possible future involving the product being designed. These stories are written specifically to fit the interviewee’s profile, presenting a provocative futuristic scenario that engages the person in a discussion about the ethical implications of what we aim to create with the project. Besides informing design decisions, by introducing these questions, the exercise is meant to influence those involved in the design process, e.g. designers and developers of robotic technologies that will shape the future of these products.
In our case, we used this methodology in the context of a project focused on designing and implementing a robotic platform to support employees with cognitive disabilities when carrying out different maintenance activities in office buildings. In the exercise, we involved two managers of the company where the robots would be introduced, four roboticists in charge of developing the technology, and two designers of the robotic platform. The FAB interviews we conducted were individual and each lasted between 20 and 30 minutes. We began by reading the story to the participant then asking a series of questions about it following a semi-structured interview format. Here is an example of one of the stories written targeting the managers of the company:
“After weeks of working with robots without problems, store managers have recently started reporting incidents. Employees with impairments have shown signs of stress after interacting with robots. A few of them have suffered outbursts, even reaching the point of hitting the robots. What caused these negative reactions? Why did things change suddenly after weeks of successful collaboration? How do employees interact with robots?”
The audio recordings of the interviews were then transcribed and a thematic analysis was done by the lead researcher to generate themes from the collected data. These themes were reiterated through discussions with other researchers. Our analysis focused on how the participants responded to ethical and social questions regarding the role of robots, its users and their behavior. We uncovered potential unintended consequences that might otherwise have been ignored, and we were able to understand participants’ views on automation and job replacement, the role of the robots, expected robot behaviour, and control and surveillance, all of which would inform the design of our platform. Through the discussions we learned, for instance, that the managers were open to replacing managers with robots to reduce the employee to manager ratio. This would ultimately reduce their operation costs and increase the number of employees with disabilities, the latter being their primary goal. While we would initially argue against the use of robots to replace human workers, discussions like these showed us that it’s not always easy to take an a priori ethical stand and confirmed the value of conducting this type of participatory exercise.
Conclusion
As technology advances and reaches new areas of human life, it’s increasingly important to discuss the role it should have and get a better understanding about the possible impact of the systems we create, as presented in our example. To successfully achieve this, we need to make an effort and bring discussions around ethics and values to the spotlight during our design process, leveraging new methodologies to understand everyone’s perspectives and the potential future implications of what we design.
Project contributors:
Shreepriya, UX Research Scientist; Danilo Gallo, UX Research Scientist (Author); Tommaso Colombino, Senior Research Scientist.
References
1. Harari, Y. N. (2016). Homo deus: A brief history of tomorrow.
2. Höök, K. (2012). Affective Computing: Affective Computing, Affective Interaction and Technology as Experience.
3. Grieves, M. (2014). Digital twin: manufacturing excellence through virtual factory replication. [White paper]. Retrieved January 26, 2020, https://www.researchgate.net/publication/275211047_Digital_Twin_Manufacturing_Excellence_through_Virtual_Factory_Replication
4. Iversen, O. & Leong, T. (2012). Values-led participatory design — Mediating the emergence of values. NordiCHI 2012: Making Sense Through Design — Proceedings of the 7th Nordic Conference on Human-Computer Interaction. 468–477. 10.1145/2399016.2399087.
5. Bosch, T. (2012). Sci-Fi Writer Bruce Sterling Explains the Intriguing New Concept of Design Fiction. Accessed January 26, 2020, https://slate.com/technology/2012/03/bruce-sterling-on-design-fictions.html
6. Muller, M. and Liao Q. V. (2017). Exploring AI Ethics and Values through Participatory Design Fictions. (2017), 9.
7. Blythe, M. (2014). Research through design fiction: narrative in real and imaginary abstracts. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems — CHI ’14. ACM Press, Toronto, Ontario, Canada, 703–712. DOI: http://dx.doi.org/10.1145/2556288.2557098
8. Britta F. Schulte. 2016. Using design fiction to reflect on autonomy in smart technology for people living with dementia. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing Adjunct — UbiComp ’16. ACM Press, Heidelberg, Germany, 1110–1113. DOI: http://dx.doi.org/10.1145/2968219.2972717
9. Cheon, E. & Su, N. (2018). Futuristic Autobiographies: Weaving Participant Narratives to Elicit Values around Robots. In Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction — HRI ’18. ACM Press, Chicago, IL, USA, 388–397. DOI: 10.1145/3171221.3171244.
Cover image created using resources from freepik.com
