Communication through performance and random sounds
Environment and Information Studies Major Freshmen
Makoto Nagase
Over the course of the fall semester, I have personally worked on several projects in the X-Music Lab. In this report, I would like to introduce once again what I have been doing for the past 6 months, as well as go into further detail in regards to the effects and discussions.
One of the projects that I have worked on was the exhibition. On the 17th of November, our lab successfully conducted the “Synthesis Exhibition”. The performance that I worked on was called “Clothes Sound Synthesis”. The performance tried to figure out the possibilities of sounds created by the relationship between clothes and human body movement. We have created the performance with 3 people, each with distinctive tasks. I was in charge of the technical side and the changing effects during the performance. We connected 5 contact (tuner) microphones to the specially made clothes for the performance and put effects during the performance. The aim of the project was to introduce the audience to ordinary noises turned into music and use them effectively to create an interesting sound.
The difficulties were the balancing of entertainment and the experimental side of the performance. For instance, it was completely fine for the experimental side of the performance to get a similar sound with little differences when moving differently with the body. However, since it was a performance with the audience in mind, the performance required somewhat of a noticeable change for entertainment. Thus, radically different shapes were introduced to visually entertain the audience, while mixes of reverb, delay, EQ filters and other effects were used to alter the sounds just the right amount. Our goal here was to allow both the performer and the effect changer to have the authority over changing the sound, meaning that one should not have more dominant control over what kind of music and sounds would come out.
Another difficulty was working together as a team to create one particular performance. Although the team members had the same goal — that is successfully deliver a performance that gave some meaning to the audiences that watched the performance — working together as a group was one of the difficult tasks that the group had encountered. Each of the members were unique human beings with different ideas to make the performance better. Just how our performance focused on the equality of power between the performers, the same attitude was present during the process of creating the performance. However, it was not rare for the schedules of the members to be very different and made it very hard to cooperate. At the end, the members figured out a way to efficiently distribute the work between them to get more things done, as well as manage the time better during meetings to share only the needed information to get the maximum amount done. I personally feel that this is one of the difficulties that comes with online classes where we do not get to share information regularly face to face, which can create a huge knowledge gap between the members.
Video of the performance : HERE
The result was very interesting as we have found some patterns and new findings in regards to the clothes, as well as the technology used to perform. One of the findings was that the relationship between the movement of the body and volume of the sound did not correlate as opposed to the hypothesis. Instead, the volume of the sound correlated more with the velocity in which the clothes moved across the space. This meant that small movements causing big actions on the cloth made far louder sounds than just making big movements. Another finding was the relationship between the pitch and movement of the cloth. The pitch that the microphone received changed according to the velocity of clothes. From this result, a conclusion that when a cloth is used as an “Instrument”, its velocity determines at least two of the major factors to create music — pitch and volume. The performance brought new findings to the technical side of music. That is, the further possibility of using wireless contact microphones as well as the effects used to alter sound during performance. The combination of contact (tuning) microphones and the wireless transmitters worked very well throughout the performance. Fully functioning wireless audio transmission can be done within 500 grams of weight and 2000 yen. This setup could further encourage unique music creation with its portability as well as affordability. The live effects of the performance allowed us to see the versatility of simple effects such as reverb and delay. One finding that I personally found interesting was the use case of reverb. Initially, it was used to create space and dynamism to the performance, however, during the practice and performance, it became obvious that reverb could be used more effectively to alter the sound even more. Two precise examples were groove and pitch. Using the time of the reverb as a fixing point for a constant rhythm, the performer was able to establish a loop that added groove to the performance. The pitch altering is also very interesting as the reverb allowed for an artificial howling effect with ability to control the howling effect using the time of the reverb. Overall, the performance opened several new possibilities for both this particular project and other projects as well.
Another performance that I have worked on this semester was a alteration of werewolf game or in Japanese, “人狼”. The summary of the plot : 4 people had been sent to space for exploration, however, during the exploration a “music virus” was detected. When infected, all of their words become sounds of an instrument. The aim of the project was to figure out the possibilities of verbal communication used to create a certain sound and rhythm as well as trying to get multiple people verbally controlling the sounds so that we effectively get a session.
Because of the COVID-19 situation, we had to perform the game online which made our task very complicated technically. On a DAW, we have prepared four channels with each individual SONOBUS (an app which allows users transmit audio with minimal latency) rooms. The testers (or the players) each entered a different room, and each voice was separated. An AUX channel was created with a plugin to change audio to MIDI signals, and finally, the midi was sent through the DAW to several samplers with different instruments. The final audio was shared through Zoom conference applications so that all of the players and observers were able to hear the sound.
Because of the technical difficulty and the instruments not recognizing the individual MIDI, we could not test out with multiple people at once, however, we did get to have one person not being able to be heard but sounds from sampler was being heard. The game made it very hard for the players to figure out the mystery behind the plot because of a person who could not speak but instead played sounds from the sampler (from now on, the person will be referenced as the “infected person”). In several plays that we have conducted, some managed to capture the victim whereas others did not. On the musical side, some interesting findings were observed. One fact was that words with “K” consonants were picked up on average 2 octaves higher when changed to midi signals. Also, the “T” and “S” consonants were picked up an octave higher on average. For further research and development of the game, using such patterns in the recognition of the words, providing the audience with keys to figure out what the “infected person” is saying, would further encourage the “infected person” to speak and allow the session to occur with discussion.
Video of the performance : Here
As a conclusion, it definitely was one of the busiest semesters of my life yet I found very interesting discoveries. Although many findings were valuable, perhaps the most valuable finding throughout the projects that I have worked on, is the fact that I found an interesting research question that I might want to work on over the years that I’m in the university. That is the following: How much can musical melodies and other factors of music (excluding lingual communication such as lyrics) deliver one’s emotions or feelings. Music is a form of communication, and I have learned this with the two major projects that I have worked on this semester. I want to figure out how effective music and sound is without using distinctive and defined symbolism such as language.
