Trees of Knowledge: Designing with AI in the Urban Landscape

from the 2018 AAAI Symposium on the UX of AI

In March 2018, my collaborator (Xiaoxuan (Sally) Liu) and I presented our ongoing speculative design project Topos at the UX of AI Symposium held at Stanford University. Below is an edited version of our talk. You can learn more about the project here and here.

Before we get into our project, we wanted to provide you with some background context of how this work emerged. Topos was developed in a class taught by Phil van Allen and Ben Hooker that gave us a crash course in what AI and ML can already do, so that we can think through its near-future implications. Our projects centered around how AI and ML fit into the neighborhood, where people interact with the urban at a human scale.

The two of us were interested in AI for very different reasons…

[Godiva] I was interested in the idea of AI as a public service; I took the context of the neighborhood very literally and imagined a Mr. Roger’s-like figure being the entity that could deliver this service to people

[Sally] And I was interested in how AI could behave more like plants rather than animals. And how the training and learning of AI systems is similar to the tending and pruning of plants.

Although our interests were very different, the common thread we found was the idea that there is a structural opacity to AI systems and ML processes. You can’t see them or touch them, but we wanted to see if we could speculate on how to make AI systems (and how they work) more transparent

Topos addresses the possibility of ubiquitous AI systems, and anticipates the shadows they might throw on the urban landscape. When a city’s infrastructure is embedded with autonomous AI systems that can track, redirect, and predict urban rhythms faster than humans can, the machine readable city might become illegible and inaccessible to humans.

If the inner-workings of AI systems that drive the city are neither visible nor tangible, then how we design AI interfaces can illuminate the algorithmic dimension of the city for the people living in it.

Topos imagines that the hidden intelligent systems controlling the city are made readable to citizens in the form of physical, manipulable, tree-like structures. How might pruning and tending these civic interfaces — these trees of knowledge — literally and figuratively reshape the urban landscape?

We propose a new typology of public space that combines the mechanical qualities of urban dashboards and city control rooms, with the permeable and spatial qualities of public parks.

We imagine that these “AI-parks” contain trees of knowledge that physicalize what is otherwise invisible to citizens: the algorithms, decision trees, and neural nets that have taken on or augmented the responsibilities of city departments and bureaus.

AI-parks are where civic decisions happen in plain sight and in real time. The trees of knowledge are tangible user interfaces that allow humans to read and revise civic AI systems by interacting with them through the physical environment.

We were really influenced by Shannon Mattern’s survey of city control systems, and determined that these trees of knowledge would not only function as AI-interfaces, but also as civic symbols and platforms of alternative governance. Trees of knowledge are not slick graphical summaries of quantifiable city data — they have tangled branches and deep roots — they are complex and messy interfaces (on purpose).

To model our proposed human-to-AI interaction, we created an experiential prototype that simulates reading and revising AI systems through the trees of knowledge. Our first prototypes for the trees of knowledge were civic monument-scaled forms that manually contract and expand.

The outer faces of this form serve as input layers and output layers for the neural nets that learn from city data in different ways; by unfolding and expanding the form, citizens and civic workers can respectively read and revise the hidden layers — where AI systems transform city data into intelligence.

By externalizing civic AI systems into public space, Topos aims to create active and participatory modes of collecting data used to train these systems, as opposed to the passive accumulation and the infinite siloing of data.

Our second prototype uses graphics and animation to illustrate our concept. For this iteration, we didn’t focus on what the human-to-AI interaction would look like, but rather how the act of “pruning” these trees of knowledge would affect the larger urban landscape.

We started to think about these bigger effects by illustrating what a city full of AI-parks and their corresponding trees of knowledge might look like. Imagine every city department and bureau being an open-air park, instead of a building with offices.

To bridge the gap between this larger context and the human-to-AI interaction proposed in our first prototype, we made a short animated video where a person prunes a tree of knowledge, and the effects of their inputs are simulated in the abstracted city.

At its core, Topos envisions a model of AI-embedded urbanism that guarantees what Henri Lefebvre calls the “right to the city” — an idea and social movement that advocates for the participation of individual and collective agents alike to shape the city.

Knowing that an urban landscape without AI systems can sacrifice this “the right to the city” to the demands of privatization and capital, UX designers must take on the problem of designing civic AI interfaces that will not allow this human right to disappear.

It is easy for AI systems to reproduce more of the world we already have, but it will be up to designers to bring the messiness of human-to-city interaction to the surface, and make it usable.

How is the human agency to reshape a city mediated by AI systems? What agency do AI systems have to reshape our cities?

If our trees of knowledge worked in a real world context, we realize that things could go very wrong very quickly. For instance, someone could tend to or prune the trees of knowledge in a damaging way — disrupting a balance between the AI-parks and its outputs in the urban landscape.

Some questions that we have for those who work with AI systems on a more practical level include:

  • How do AI systems “learn” from inputs that come later and are not part of the initial training data?
  • Could we design a “defense mechanism” into the trees of knowledge so that human inputs can align with a greater sense of public good?
  • How can trees of knowledge (a speculative interface) highlight a negotiation process with civic AI systems?