Cropped illustration by Rob Sheridan entitled “Leaving Town”. Used under a creative commons license.

Finding Common Grounds between Artists and Machines

Margaret Lam
Oct 30, 2020 · 9 min read

What is the role of artists in our increasing digital society?

I will come back to this question later, but first let’s talk about information representation, historical records and artificial intelligence.

Dr. Sarah Bay-Cheng, co-author of Performance and Media: Taxonomies for a Changing Field, is a theatre historian who is most often found pushing the boundaries of how we define theatre and live performances by incorporating digital and interactive elements into the mix. She was appointed in 2019 as Dean of the School of the Arts, Media, Performance and Design at York University, the alma mater of Mariel Marshall, founder of StagePage, a recommender system for the performing arts.

At Marshall’s invitation, Bay-Cheng gave a talk on August 21, 2020 on the topic of information representation in the performing arts. It began with an anecdote of a particularly lively discussion Bay-Cheng had with members of an international working group (on digital technologies in theatre and performance history and historiography, if you must know).

The discussion was a philosophical one about the difference between an “actor” and a “performer”. The question on Bay-Cheng’s mind was related to the creation of ontologies as it is used in computer science: how would machines understand that difference, if at all? And how would that influence a machine’s understanding of human descriptions of performing arts events?

Thought about in this way, getting the distinction between an actor and a performer just right becomes incredibly important. As Bay-Cheng elegantly framed it, this can be the difference between “something getting recognized and something becoming completely invisible”. In her talk, she discussed five distinct areas in the performing arts sector to underscore the importance of the act of translation from human text to machine language (and back again).

Slide from Aug 21, 2020 presentation (courtesy of Mariel Marshall)

The first area is around the curation of information related to performing arts. Particularly information related to specific events and productions, and all of the nuances and details that go with it. Beyond the overwhelming volume of such information that is available for human consumption, machines are also “reading” and “interpreting” the same texts for their own purposes. Those very same machines are generating our social media feeds, all kinds of other information in human-speak and actively shaping what we perceive as the most relevant and interesting.

The second and third areas are criticism and history of the performing arts. For Bay-Cheng who is a historian, textual descriptions of performances are sometimes the only records that are available for performance that occurred prior to the existence of recording technologies. Today, historians and archivists are adapting their practices in response to the incredible amount of digital content and records that exist publicly around a variety of performances. This requires fundamentally rethinking how we archive, how we interpret, and how we disseminate that understanding in the digital domain.

This is all fascinating, of course, but what does it mean in practice? Why should those in the performing arts sector care? Let’s address this question before we talk about the last two areas in Bay-Cheng’s talk.

The Relevance of Information Representation in the Performing Arts

As you may already suspect, this isn’t only of interest to humanities scholars who must evolve in the context of a digital society; it has real world implications in the arts and culture sector. Daniela Rosu, a computer scientist and post-doctoral fellow at the Centre for Social Services Engineering, specializes in co-developing the core of advanced computational systems in domains such as medicine and social science. She offered some real-world examples of challenges that the arts sector will need to respond to in the form of questions:

How might a machine use the publicly available audience responses and reactions to various shows to create a personalized “critic”? What kind of values would this machine have? What kind of art would it prefer over others? What if previews and synopses of shows are automatically generated based on such a “critic”, and directly influence audiences’ decision as to what shows they will attend?

Perhaps the most critical question of all: if machines have such a powerful influence over the careers of artists and their creations, are there artists at the table when these technologies are being developed?

The Canadian Performing Arts Sector’s Response

In the short-term, there is a critical need to keep up with the most influential machines on the internet: search engines and voice assistants. There are standardized web “housekeeping” practices that can be done to structure online public event information to improve their discoverability, such as setting up single web pages for individual events, to ways to improve the discoverability of event information when using event plug-ins for WordPress.

Many arts institutions are getting better at all this, having been introduced to it by organizations like Culture Creates, CAPACOA’s Linked Digital Future Initiative, Digital Arts Nation and independent consultants that work in this space. However, we are still a long way from ingraining these practices into individual artists and arts organizations who maintain a digital presence.

What Bay-Cheng’s talk emphasized is the more long-term and complex need to influence the way these powerful machines understand the arts. This is a priority area for the Canadian performing arts sector that Culture Creates, CAPACOA and their partners have worked tirelessly over the last few years to address. In particular, they have focused on the work involved in linking up existing data from different organizations with each other, and figuring out the best way to make them available to machines as linked open data.

Slide from Aug 21, 2020 presentation (courtesy of Mariel Marshall)

In this process, there is a constant need to translate human language, which is full of nuances and shades of grey, to computer language, which is quite rigid and clearly defined. We do this by creating taxonomies and ontologies, which is the lens through which machines know what meaning to give certain words, concepts and relationships between them.

Taxonomies are no means perfect representations of the world, and nor are they intended to be. As Rosu elegantly puts it, there is no “one taxonomy to rule them all”, rather we need a multitude of taxonomies to reflect the needs of different domains and perspectives. When we put together multiple taxonomies across domains, we get an ontology, which identifies and distinguishes concepts and their relationships.

As important as it is to get the representation of performing arts information “right”, small and medium sized organizations simply do not have the capacity or bandwidth to engage in this work in a meaningful way. Yet, having a diverse representation of artists and patrons involved in the process is probably one of the most critical success factors. It is the only way to ensure that the art world as artists and patrons see it is represented in the world of machines. By extension, it is also the only way to make sure that the machine works for us, not the other way around.

Slide based on Aug 21, 2020 presentation (courtesy of Sarah Bay-Cheng)

The Next Generation of Artist-Innovators

The last two areas in the performing arts that Bay-Cheng highlighted, namely theory and education, are areas in which the sector can develop and evolve to improve its readiness to engage in the creation of taxonomies, ontologies and other advance technological solutions for the performing arts.

Theory is about the way we see things, how we frame or approach a problem. Bay-Cheng’s own cross-disciplinary approach as a digital humanities scholar, informed by her artistic practice, exemplifies how different theoretical lenses can can help us articulate emergent problems and develop innovative solutions to address them.

From an education perspective, this kind of thinking needs to be built-in to the way we train the next generation of artists and creators. This is a much greater topic that requires its own separate post, but this is not about helping students “keep up with technology”. Rather, it is about equipping our future artists to play an active role in our society, one that is now irrevocably digital.

Mariel Marshall is an example of such an artist. In addition to being co-founder of StagePage, she is also a core member of the ground-breaking performing arts group bluemouth inc. In her pursuit of creating a recommender system that will leverage the latest research and technologies to help audiences discover performing arts events, she has been pushing her own boundaries of what it means to be an artist in a digital society.

PAIR W3C Group: Bridging Artists and Machines

By bringing together a computer scientist and a digital humanities scholar, both of whom have an interest in the arts as a domain and as a practice, Marshall has assembled an interdisciplinary team that has the expertise and skin in the game to facilitate the creation of taxonomies that represents the values and interests of the arts.

Together, they have created an open community group under the umbrella of the World Wide Web Consortium (W3C), an international community that develops open standards for the internet. Envisioned by Marshall and Rosu, and chaired by Bay-Cheng, the Performing Arts Information Representation community group is dedicated to addressing the challenge of translation between artists and machines.

The goal of the group, as Marshall states is to: “collaboratively create digital infrastructure to support performing arts online tools. Imagine having an expert arts curator in your pocket, or a ‘Netflix’ or ‘Spotify’ for the performing arts. These tools are possible, but only if we work together to build a representational model that speaks to the current methods, modes and ways of making art.”

The group is still in its infancy, with their first meeting slated for November 9, 2020 at 2pm EDT. They are seeking participation from organizations and initiatives that are already engaged in similar work, as well as creating opportunities to engage artists and patrons at the community level. If done right, this can be a tremendous opportunity to figure out how we can bring artists into a technology creation process in a meaningful and reciprocal way.

Being an Artist in a Digital Society

To return to the question at the start, what is the role of artists in our increasingly digital society? My answer (and yours) is perhaps not very different had I taken out the word “digital” from the question.

For me, one important role of artists is to help us examine the contours of our world more closely by engaging the public through encounters with their art. Artistic experiences can reflect our most deeply held values and assumptions, and also open us up to new perspectives and possibilities. The importance of this has been brought into relief in a world that’s becoming increasingly fractured and divided.

As to how artists are supposed to fulfill that role when digital technologies are thrown into the mix, Marshall and others like her offer a direction and a vision. Fundamentally, it is about finding and building common grounds between artists and machine through the world of data. This project is one of many that is contributing to a nascent yet growing community of practice in Canada around arts data literacy. Our collective success will hinge on our ability to put artists first in the process, and develop viable business and governance models to sustain this effort.

Like many artistic creations, this will be an ever-evolving work-in-progress, and you are invited to be a part of it.

Click here for more information about the PAIR W3C community group. Registration is now open for the first public meeting on November 9, 2020. We are hoping participants with a variety of backgrounds and experiences will attend.

To learn more about arts data initiatives in Canada, check out CAPACOA’s Linked Digital Future Initiative for in-depth research reports, data literacy workshops and so much more. Connect with Culture Create as they build a national knowledge graph for the arts. Follow Arts Impact AI as they engage artists in conversation to explore social issues around AI. For those who are looking to orient themselves a bit first, check out Digital Arts Nation.

Special thanks to Dr. Sarah Bay-Cheng, Frédéric Julien, Tammy Lee, Mariel Marshall, and Caitlin Troughton for feedback on early drafts.

BeMused Network

Conversations and ideas with arts professionals…

BeMused Network

Conversations and ideas with arts professionals, technologists, designers, researchers and everyone in between who are working on digital transformation in arts and culture in Canada and around the world.

Margaret Lam

Written by

Founder of @BeMusedNetwork. Designer. Researcher. Synthesizer. Occasionally writing at the intersection of culture, technology and the performing arts. (Canada)

BeMused Network

Conversations and ideas with arts professionals, technologists, designers, researchers and everyone in between who are working on digital transformation in arts and culture in Canada and around the world.