Week 2: From Theater to Music Performance

Moving Towards Embodying Bodily Movements In A Remote Setting

Original Idea–Remote rehearsal collaboration between a director and two actors

Based on our interviews and research on remote theater performance, we worked on refining our ideas in areas including the problem context, target users, and what technologies are involved. We first listed all the potential options and then discussed the pros and cons of each.

Discussion on problem context, target users, and technologies.

Our discussion led to several consensuses:

• Technology: We will focus on a tangible interface and perhaps with the inclusion of AR and a wearable device. We will not be working with VR since it’s mainly emphasizing the visual experience.
• Target Users: We will design for the remote collaboration between staff (i.e. directors, choreographers) and performers.
• Problem Context: The solution will be used in rehearsals where the staff and performers are in different locations. The final performance can be both remote and live.

After narrowing down the scope, we built upon our previous storyboards and created a new story for the mid-critique. The new story involves three users including a director and two actors. The actors are both transitioning from other gigs so they have decided to collaborate remotely to give flexibility for their individual schedules.

The director and actors use a tangible interactive device and projection mapping to help them navigate where they should be located throughout the scene. The final performance will be live and in-person so they have to consider how to move through a scene in relation to one another.

Storyboard iteration 3: Remote collaboration between a director and two actors through a tangible device.

We presented the idea in class for the mid-critique and received generally positive feedback. There were several questions raised that we need to consider for improvements. For example, how can the system be made expandable? Can it provide benefits to users other than the director and a small number of performers? If it can be used by people taking other roles in the acting/performance/theater work, that would make the concept more convincing. We also need to think about how can actors not be blocked by themselves since shadow may block the visual cues. Furthermore, how can we incorporate gestures in the feedback in addition to spatial position.

New Direction–Communicating conductor gestures to the musicians during remote rehearsal

Considering what we have heard from user interviews and the in-class feedback, we decided to move from theater to music performance and design for remote collaboration between the conductor and the musicians for the following reasons:

• Music /audio is a major pain point for remote performance. Currently, music in remote performance is prerecorded. Prerecording can make a performance sound perfect but also has the risk of being too sterile. We want to preserve the liveness of the sound because it gives people tension and excitement.
• We are more interested in feedback through bodily movements. In the previous storyboard, the feedback is around the spatial location which would be addressed most efficiently by VR technology. We decided that we want to explore tangible interfaces and ways to physicalize gestural input.

We then started exploring the set up of music performance and what feedbacks conductors typically give through their gestures.

Visualizing the set up of music performance.

To construct the new story, we broke down the conductor-musician collaboration into several steps and listed their potential interaction in each step:

• Before the rehearsal starts, the conductor signals the start of the performance through a gesture that triggers a buzz on the musician’s device.
• During the rehearsal, the conductor look at the screen with performers while giving gestural instructions. The screen also has icons for each group. When giving instruction for a particular group, the conductor gaze at the group’s icon and the group’s devices are turned on.
• The device translates conductor gestures into light patterns. It also marks the conductor’s notes for future reference.
• When the musicians practice by themselves, they can turn on the device to play a recording of the music as well as the conductor’s notes.

Based on those steps, we came up with a new storyboard for the collaboration between the conductor Ava and the T1 group in the ensemble.

Storyboard iteration 4: remote collaboration between a conductor and the musicians in an ensemble.