Surveillance capitalism, by design
What is going on with things?
This is a question that has been animating my research for the past few years. It has led to identifying some basic shifts underway in how things are made, when, where, and by whom, and what they are made of. Due to networked connectivity, things can be active and responsive, dynamically configuring themselves to particular users and contexts and pulling together networked resources to assemble themselves on the fly. Because of these characteristics, Johan Redström and I have termed these things fluid assemblages (in our book Changing Things) in order to conceptualize them in a way that can help us come to grips with how things are changing and what this means for design and use. This is a shift that we argue has profound consequences for design, as significant as the shift from craft to industrial production.
And what do these things do?
This is another fundamental question that also leads to recognition of massive shifts underway in the roles that everyday things play. This question is often addressed from a perspective of focusing on human experience, even as experience can be seen as co-constituted through (technologically-mediated) relations (as in postphenomenological theory and analysis).
But things that are fluid assemblages require other kinds of questions. As I wrote at the end of my chapter in the book Postphenomenology and Media, expanding on the traditional ‘what things do question’:
But no, really, what do these things actually do? What do they do when we are not watching? What do they do that we did not ask them to do? What do they do with each other? What do they do that we do not know about? What kinds of worlds do they show us? What kinds of stories do they tell us? What kind of stories do they tell about us?
And what should we do about it?
I have found that investigating what things that are fluid assemblages actually do leads very quickly to issues of surveillance / cybernetic / platform capitalism as the overarching forces and logics that drive the way that most industrially-designed and produced connected things operate and the functions that they perform. They are not only or even primarily things for use and users. They are things that render users and their activities visible, computable, accessible, and potentially even modifiable for industrial actors in a position to benefit from this god’s-eye view and access. These are processes fuelled by data that is produced (never raw) for particular purposes, tuning actors and entities in massive real-time cybernetic feedback loops in which everyday life and reality are mined for resources that can be processed to generate value elsewhere.
These dynamics do not negate the very real value that connected things can provide. They can indeed be quite useful, even delightful, scaffolding types of creativity, connection, and understanding that would not be otherwise possible. Yet at the same time unease with these things seems to be growing, as many now struggle openly with managing attention when devices always seem to be clamoring for it, and with clawing back some level of privacy from things and systems that seem bent on preventing us from doing just that (while technically remaining within existing legal frameworks that have not even begun to keep pace with these changes).
This leads to considering the role of design in all this. Because the “dark patterns” of interface design that steer users toward the desired ends of producers, the strict separation of end use from primary purpose, and the user-facing shells that conceal what things actually do are made by design. And yet, with a few exceptions (e.g., the discussion about “dark patterns” of interface design, efforts to certify IoT devices as “trustable”), the field of design does not seem to be addressing this.
Darker patterns
There are at least beginning to be discussions of machine learning and AI as design materials; and interaction designers also work more with data scientists as design work is increasingly done through data-driven progressive optimization rather than crafting of complete and final products in the way that industrial design worked under industrial capitalism and systems of mass production. But rather than being central to industrial production, as it was under industrial capitalism, design seems to be on the sidelines in relation to where much of the action currently is.
In fact, if we look at the basic position and role that industrial design was in — that of mediating relations between production and consumption— it becomes clear that it is no longer the central actor in this position. If human experience and more general reality is the resource that is mined as a data-fied resource and used to produce audiences, behavioral futures markets, and means of influencing behavior as products, then it is data science and analytics that is mediating these new basic relations of production and consumption. Interaction design now designs only the mining tools.
Of course, it can be protested that interaction designers do much more than that, which is true to a certain extent. Their typical concern for supporting the richness of everyday life continue to make them quite useful when designed things being enmeshed in people’s everyday is a corporate objective. And things that are fluid assemblages and their design are also not inherently tied to surveillance capitalism. Designers can and do work to design things that allow users to preserve their privacy and integrity.
But as Shoshanna Zuboff’s recent work shows, surveillance capitalism is now the dominant logic and form of capitalist accumulation. And once we understand the fundamental new logic of surveillance capitalism, as Zuboff so quotably puts it:
it becomes clear that demanding privacy from surveillance capitalists or lobbying for an end to commercial surveillance on the Internet is like asking Henry Ford to make each Model T by hand. It’s like asking a giraffe to shorten its neck or a cow to give up chewing. Such demands are existential threats that violate the basic mechanisms of the entity’s survival. How can we expect companies whose economic existence depends upon behavioral surplus to cease capturing behavioral data voluntarily? It’s like asking for suicide.
In this new landscape where data is the new oil and everyday connected things mediate the production of this basic resource, we might hope that interaction designers will at least be conscientious in designing to resist the worst forms of exploitation and abuse around data production. This is indeed an important topic to now include in interaction design education and professional discourse. Yet in industrial contexts and outside of more one-off artistic productions (if even that), we must also be realistic about the fact that these types of efforts directly contradict the imperatives that companies (that hire designers) must follow to survive in a capitalist marketplace.
Production and consumption, take two
Design has been here before — still is in fact. Under industrial capitalism, industrial design facilitated the production of consumer goods to fuel the endless appetite for consumption in wealthy parts of the world that was the foundation of economic growth (and cause of unevenly distributed benefits and damages). Recognizing this now in the context of (designed) unsustainabilities, there are now serious efforts to turn the capacity of design toward envisioning and transitioning toward sustainable futures. And yet, these efforts exist in parallel (and with far fewer resources than) business as usual in industry — or at least, more as usual than is even remotely desirable given the magnitude of the problems and required changes. While design has rich capacities to grapple with complexity and design for ecology, it has been mostly tethered to responding instead to the demands of the capitalist economic system rather than the larger ecological system in which it is embedded and on which it actually depends (as Joanna Boehnert has incisively articulated).
If industrial capitalism called mass production and industrial design into being and sustained them, it might be argued that surveillance capitalism has called fluid assemblages and big data analytics into being and sustains them. This is not to say that things could not be otherwise. Design in a broad sense is about configuration of the artificial and care for the possibilities and futures that it opens up or forecloses, and for whom.
In search of design, in search of possibilities
Things that are artificial could by definition be otherwise. As design theorist Clive Dilnot beautifully writes in concluding his chapter in the book Design as Future-Making:
The paradox of our time is that we have made that which we cannot yet think. The artificial, understood aright, is our possibility as well as the source of the dangers that beset us, though these lie, as we have seen, as much if not more in the attitudes we bring to the artificial rather than to any essence of the artificial. Thinking the paradox of the artificial — in action, through the manner in which we remake the world — is turning the prosaic nihilism of our age towards a resonant affirmation of what is possible for our history beyond accumulation and catastrophe. Reasons to be cheerful? Not quite. Reasons for possibility? Certainly.
What, then, is the role of design in relation to the thoroughly artificial edifices and mechanisms of surveillance capitalism? Is it to rearrange pixels while possibilities for other ways of ordering economic and social systems become submerged, to keep us as users distracted while our lives are extracted and their data shadows sold to the highest bidder? Is it reasonable to pursue the possibilities that connected things that are fluid assemblages offer while ignoring the very real entanglements that pull their operation inexorably toward participating in modes of dataveillance?
Design seems in a way both too big and too small. In its world-making capacity it can imagine futures that are too grand, too distant, or too exotic to have much of an impact in the present; and in its more quotidian professions, it is often (kept) too small to substantively affect larger systems in which it is embedded at scale. Yet in terms of grappling with the artificial world and finding its more enlightened and diverse life-affirming possibilities, design in the truest sense is needed now more than ever.
As my Google search for “surveillance capitalism design” accidentally but pointedly puts it:
Missing: design. | Must include: design.
📝 Read this story later in Journal.
🗞 Wake up every Sunday morning to the week’s most noteworthy Tech stories, opinions, and news waiting in your inbox: Get the noteworthy newsletter >
