More-than-Human Design for the Future of AI in the Home
By Iohanna Nicenboim, Prof. Dr. Elisa Giaccardi, Dr. James Pierce
The ThingsCon report The State of Responsible IoT is an annual collection of essays by experts from the ThingsCon community. With the Riot Report 2018 we want to investigate the current state of responsible IoT. In this report we explore observations, questions, concerns and hopes from practitioners and researchers alike. The authors share the challenges and opportunities they perceive right now for the development of an IoT that serves us all, based on their experiences in the field. The report presents a variety of differing opinions and experiences across the technological, regional, social, philosophical domains the IoT touches upon. You can read all essays as a Medium publication and learn more at thingscon.com.
Inhabited by smart devices and intelligent assistants, our future home will be certainly more-than-human. Expanding to almost every fabric of our everyday future, the Internet of Things (IoT) and Artificial Intelligence (AI) promise new exciting possibilities for designers. But they also surface new anxieties (see for example the projects Network Anxieties and Objects of Research).
Existing anxieties such as privacy and democracy become even more prominent when IoT is combined with AI. AI is seemingly everywhere, but the term actually means different things in different contexts. Beyond the humanized and romanticized ‘he’ or ‘she’ that we see in science fiction movies and television, AI comes in many forms, and it is already underlying many of the products and services we use today, from social media to public services. Even though we interact with AI every day, its complexity and opacity makes extremely difficult for people to ‘see’ it and grasp its benefits and pitfalls. This situation leaves us uncertain on whether we need to protect ourselves against the ‘entity’ AI or rather against the people building, training, and operating it.
Understanding and critically evaluating AI is difficult also for designers and researchers. Complex AI agents often exhibit emergent behaviors that are impossible to predict with precision, even by their own programmers. MIT researchers explain that to evaluate AI algorithms it is not enough to simply look at their source code or internal architecture. For this reason, they recently proposed a new field called Machine Behaviour, the scientific study of machines not as engineering artifacts, but as a new class of actors with their unique behavioral patterns and ecology. This new field overlaps with computer science and robotics, but it is different, because it treats machine behavior observationally and experimentally.
When we start looking at algorithms from an anthropological perspective, we begin to see that they are “unstable objects that are enacted through the varied practices in which we engage with them” (Seaver 2017; Giaccardi et al. 2016). This is interesting in IoT, because intelligent algorithms are performed by everyday objects, which might be fundamentally different than other enactments of AI. The way AI is enacted or performed by everyday objects can foreground certain issues while occluding others. For this reason, Nick Seaver proposes that critical researchers should research algorithms ethnographically, seeing them as heterogeneous and diffuse sociotechnical systems, rather than rigidly constrained and procedural formulas. To do so, he suggests thinking of algorithms as part of broad patterns of meaning and practice that can be engaged with empirically.
So when it comes to design, we too can begin to observe smart objects ethnographically to evaluate the future of IoT + AI. This means to research AI not just as code or behaviour, but as performed by everyday objects in the context of mundane practices, within the messy ecologies of our homes. But design can do more than understanding algorithms ethnographically. Design can help us imagine our more-than-human home, and the role algorithms will play in that future.
For this type of inquiry, traditional design methods such as human-centered design might be insufficient. In the new domestic landscape of IoT+AI, not only people will interact with objects, but also objects with each other. To better understand these complex ecologies, we need to include also the perspective of things, and actively enlist them as partners in the design process (Giaccardi 2018). Thing-centered design is a novel design approach that gives designers access to fields and trajectories normally unattainable to human observation. This ethnographic engagement is called thing ethnography, and it is usually applied to existing things.
To research ethnographically future things instead, and help us imagine the future of AI in the home, we have combined thing ethnography and future-oriented design techniques. Future-oriented techniques, such as design fiction, make future paradigms of technology more tangible, and develop critical discourse on the impact technologies might have on individual lives and society at large. Designers often use fictional techniques to project and evaluate the encounters people may have with a technology, and the actions and decisions that people may take in response.
Thing-Centered Design meets Design Fiction
When taking a thing-centered approach in design fiction work, we not only access future perspectives of humans, we also gain access to the nonhuman perspective of things. These new perspectives can “enhance, complicate, and sometimes even challenge the perspective of humans” (Giaccardi 2018). We have explored these possibilities in two projects: “Affective Things: Entanglements of the Connected Home”; and “Unpredictable Things: Objects that Withdraw”.
Affective Things: Entanglements of the Connected Home
This project is a series of design fictions in the form of videos that explore the complex interactions of things with things in the more-than-human home of the future. By positioning everyday objects within complex ecologies, these works show how things may become entangled with us and with each other, and how they might co-perform tasks. In this work, smartness is explored as fluid and things have the possibility to become ‘other things’ as this smartness is shared and gains meaning through interaction.
Unpredictable Things: Objects that Withdraw
This project investigates the boundary of what algorithms can see (and recognize) and what they cannot see, as a productive design space for resilience against surveillance at home. The work explores how things could hide from different cameras by altering their materials and shapes. It does so by proposing different strategies to make objects unique, and thus impossible to be captured by object recognition: from a virus for digital fabrication codes to a home lab to create diversity. The design process in this project was done by co-designing with things themselves, for example, by letting living organisms reshape everyday objects in unique and unpredictable ways, or by crafting together with a machine by looking from its view to make design decisions.
Implications for IoT design
Taking a thing-centered approach in design fiction work can help designers explore a future everyday from a novel perspective and gain unique insights that might be difficult to obtain with traditional design methods. Conducting a thing ethnography by means of a speculative prototype can shed light on the ecologies that may configure around a future thing, including what meaningful data it might collect and how people would react to it. It can also help designers to finally imagine interactions that are less animistic, avoiding the tendency to excessively anthropomorphize or zoomorphize machines.
A thing perspective offers a fundamentally different role for things in design beyond their functional use. As argued in Giaccardi 2018, it helps us “cast things in design as partners, overcome our human biases, problematize our design space and possibly be more humble in our worldview”. For example in a more recent project, Connected Resources, a thing-centered approach helped us to reframe and imagine smartness as something shared between people and things, instead of an exclusive property of artefacts or humans.
Connected Resources
A family of recombinant sensors for older people, designed to emulate in physical form and digital functionality the material affordance of the mundane objects used by older people in their everyday strategies of resourcefulness.
In summary, combining future-oriented techniques and a thing-centered approach can contribute to understand how algorithms will be enacted by smart objects in everyday futures, positioning things as agents within complex socio-cultural ecologies. This can help us reframe the design space and inspire more ethical and resourceful approaches, where smartness is not exclusive but shared.
Credits
Affective Things: In collaboration with The Incredible Machine. Photos Andreas Dhollandere. Thanks to Design United.
Unpredictable Things: In collaboration with Daniel Suarez. Photos by Bart van Overbeeke. Drawings by Alexandra Sebag. Thanks to Everyday Futures Network.
Connected Resources: Graduation Project by Masako Kitazaki. Photos by Andreas Dhollandere. Part of ‘Resourceful Ageing’ funded by STW under the Research through Design program (2015/16734/STW).
References
Giaccardi, Elisa. 2018. “Things Making Things How Things Will Help Us Design the Internet of Reinvented Things.” IEEE Pervasive Computing, 2018.
Giaccardi, Elisa, Chris Speed, Nazli Cila, and Melissa L. Caldwell. 2016. “Things As Co-Ethnographers: Implications of a Thing Perspective for Design and Anthropology.” In Design Anthropological Futures, edited by Rachel Charlotte Smith, Kasper Tang Vangkilde, Mette Gislev Kjaersgaard, Ton Otto, Joachim Halse, and Thomas Binder. Bloomsbury Academic.
Seaver, Nick. 2017. “Algorithms as Culture: Some Tactics for the Ethnography of Algorithmic Systems.” Big Data & Society 4 (2): 2053951717738104.
Iohanna Nicenboim is a design researcher at the Connected Everyday Lab, TU Delft. Her research focuses on designing for the IoT as part of complex socio-technical systems in everyday futures. Her background brings together industrial design with digital media, and currently focuses on IoT and AI. Following a thing centered speculative approach, her designs often use Design Fiction to provoke reflections on more desirable futures. She received the Internet of Things Award for the Best Design Fiction project in 2015–16, and is a Thingscon IoT fellow since 2017. She participated in residency programs, gave talks and exhibited her work in different international exhibitions and conferences, like CHI, DIS, FutureEverything, Transmediale, Milan Design Week and Dutch Design Week.
Elisa Giaccardi, Ph.D. is Professor and Chair at Delft University of Technology, where she is director of the Connected Everyday Lab. After conducting pioneering work in metadesign, collaborative and open design processes, Elisa has during the last years focused on the challenges that a permeating digitalization means for the field of design. Her recent research engages with ‘things’ in new ways, with the starting point that these now hold both perception and possible agency (e.g., AI), and thus ‘participate’ in design and use in ways that previous industrially produced objects could not. Her online course “Thing-centered design” can be found on the TU Delft Online Learning platform.
James Pierce is an Assistant Professor at California College of the Arts and research scientist at the University of California Berkeley. As a design researcher, he practices and reflects upon design as a mode of inquiry, critical engagement, and speculative exploration. His recent research focuses on issues of digital privacy, security, and surveillance.
ThingsCon is a global community & event platform for IoT practitioners. Our mission is to foster the creation of a human-centric & responsible Internet of Things (IoT). With our events, research, publications and other initiatives — like the Trustable Tech mark for IoT — we aim to provide practitioners with an open environment for reflection & collaborative action. Learn more at thingscon.com
This text is licensed under Creative Commons (attribution/non-commercial/share-alike: CC BY-NC-SA). Images are provided by the author and used with permission. Please reference the author’s or the authors’ name(s).
