Symbiosis
13 June 2018
“ The purpose of art is to impart the sensation of things as they are perceived and not as they are known. The technique of art is to make objects ‘unfamiliar’. ”
~ Victor Skhlosvky
Symbiosis is a mutually beneficial relationship between different organisms.
What techniques can we use to create new forms of embodiment communication between organisms? Exploring the disparity of perception within the real and the virtual, this installation set questions about what does experience mean to us.
This portfolio is the documentation of the project called “Symbiosis”, that developed for the “Physical Computing & Design Prototyping” unit. Here you will find the process of work, which includes contextualization, research, inspiration, conceptualization, design prototyping and experimentation. This project developed after the research conducted in the parallel unit Interaction “Futures & Speculative Design”, which was entitled “Reset of Perception & Construction of Experiences in the Digital Age”. The purpose of the whole brief of the project called “Other Machines” was to research, speculate and build designs that challenge and question normal notions of what a machine is or could be.
Process
Starting this unit, I had already rich sources of research about people’s perception and experiences in the digital era. Focusing on Virtual Reality projects and Robert Nozick’s theoretical frame of the Experience Machine I started to search for inspiration in other practitioners’ work. I was especially interested in machines that seem chaotic and in the performative expression of feelings of discomfort that these machines could have.
Then, I started developing my first concept which included a machine that the audience would connect and through this would communicate with another body, that would perform a symbolic disembodiment situation. To explain my ideas to others and to make them clearer for me I always sketch them. After sharing them with presentations and tutorials I had feedback from students and tutors that were helpful to continue.
I was confused many times on finding the meaning of my work and I needed to go back to research that I had conducted or read new books and articles. Furthermore, because of my interest in choreographic installation as a form of design, I attended many performances that helped me to structure my work. Another useful component in the process of rationalizing my work was the seminars that provided to us during the course.
Writing my project’s Mission Statement was the initial motivation for starting to create something tangible. Additionally, I made a Plan that I had to follow for a seven-week period of experimentation, prototyping, testing, final implementation and documentation. So, I started my practice experimenting with VVVV — a software for real-time motion graphics — and Arduino, to create interactive electronic objects. In the same time, I had to communicate with choreographers and develop and practice my dancing piece.
My project included of many layers and different design practice and it was very complicated to describe it. So, I had to make a cardboard and paper mock-up for the whole setting of my idea, that I presented in the Prototype party that we had in the class. My first prototype consisted also from some interactive VVVV visuals made of particles and an Arduino to Arduino wireless communication with LED and buttons.
The second prototype consisted of three parts. Firstly, a VVVV implementation of a real-time skeleton made of particles, which I captured with a Kinect camera. Secondly, an Arduino connection between four e-textile buttons and vibrators. Finally, an 8-minute dancing choreography demonstrated by me.
To connect these pieces, I collaborated with a choreographer — Efi Klidara, a musician — Ivan Acher, and my classmate — Beatriz Lacerda, who helped me to create a video for the third prototype.
For the final prototype and the crit of my work I had to book a dark room, laser cut the vibrators cases, sew an e-textile costume, request some equipment from the university — like projector and speaker, test and buy some equipment — like transparent fabric to project on, set everything in the space and document the performance.
The future plans for this installation are the improvement of the animations, the collaboration with a professional dancer to perform and improve the choreography and the recording of dancer’s movement’s sounds and their transform in electronic sounds.
Background
The choreographic installation “Symbiosis” was created as a piece of artistic practice focusing on exhibition’s audience. It will be exhibited in Ars Electronica Festival — with the theme of “Error — The art of imperfection”, in September 2018.
The background of this project originated from research in Platonic notions of subjectivity, the phenomenology of perception, the Experience machine theoretical framework, embodiment and disembodiment studies.
Symbiosis is inspired from a large range of Virtual Reality projects, performances, choreographies and visual graphic works.
Research
The eye of the skin: Architecture and the Senses ~ Juhani Pallasmaa
This book inspired from questions of perception and refers to the crucial phenomenological dimensions of human experience in architecture. The writer claims that “my body remembers who I am and how I am located in the world” and he also, inspired of Merleau-Ponty’s philosophy, argues that “the human body is the center of the experience world”. This book supports my assumption about the role of the body in perception and especially the importance of touch, which is the sense of intimacy, contrast to the eyes that create distance.
How Bodies Matter: Five Themes for Interaction Design ~ Scott R. Klemmer, Björn Hartmann and Leila Takayama
This paper focuses on theories of embodiment, stating that “our physical bodies play a central role in shaping human experience in the world, understanding of the world, and interactions in the world”. Their main argument that inspired my research is that although digital world can provide advantages, we have to take care before unreflectively replacing the physical world.
Inspiration
The Look Elsewhere ~ Samuel Fasse and Morgan Belenguer
The look elsewhere is a performative artwork with three dancers wearing VR-headset and experiencing virtual worlds in a given, defined space. This anesthetization is a very successful representation that shows that VR space is limited in the physical scale. The video version is interesting as it is mixed with uncanny virtual landscapes, that blurs the boundaries between the real and the simulated, while expanding the artwork into the virtual and make viewers see the differences between the spaces.
In the performance, the dancer has no contact with the audience and the audience cannot perceive the immaterial world. However, each of the dancers carries a nomadic structure on which a scarf of the collection is integrated. The print of the scarf becomes the reflection of their experience. This is a very artistic and abstract way to show an impression of the world that the dancer experience.
Body talk exhibition ~ Gatehouse Gallery
Body Talk is an exhibition by six artists who present works in performance, sculpture, video, and large-scale installation. Through the exhibition, the visitor explores how technology is rapidly changing the human experience by promoting a reengagement with the corporeal.
What I found interesting in this exhibition is the work of Manners, a collaborative project of Lisa Rybovich Crallé and Sophia Wang. This project undermines the realms of fitness regimes and exercise tools to disrupt the static nature of art objects displayed in the museum context as well as routine behaviors.
Manners’ large-scale sculptural environment Basic Edition, created for this exhibition, extends their exploration of “somatic learning,” or learning through the entire body, to the formal realm of the gallery space.
Hakanai ~ Adrien M. & Claire B.
This installation is not so relevant with the virtual worlds that I explore, but with the fleeting nature of dreams and the fugacity of life. However, I am interested in this project for the aesthetics. It shows a Dance choreography performed in the immersive environment of a moving cube. They project real-time animations based on physical movement modelling. Also, they project in invisible fabric that produces a very interesting effect. The choice of a cube is also important for me because of the VR physical space boundaries in a rectangle room. Except from the performance, the audience has also the opportunity to explore the stage installation.
Smoke Screens ~ Renaud Wiser, Patricia Okenwa, Mária Júdová and Andrej Boleslavský
Choreographers and digital artists collaborate to create “Smoke Screens”, a combination of a dance performance and an experimental mixed reality game. This art project explores the disparity of perception within the real and the digital space, where the line between reality and fiction is blurred. The body is interchangeable and the person disappears under the digital layers. This project makes audience question what makes us choose to connect in these worlds that reflect black chasm.
The performative part called “The disappearing act” was very successful. They used some props to make the audience understand that the dancers experience in a virtual world. Specifically, they used some interesting headsets — not the real VR-headset, that were uncomfortable to wear them. In this way, they wanted to show the loosing of control and identity below the machine. They also used the projection of some human body parts into some white panels that the dancers used to cover some parts of their body. In this way, they succeed with an easy way to mix real with virtual and created an aesthetically interesting result.
The dancing part called “Glitch” had more to do with the virtual worlds that combine a shiny fantasy and a black chasm at the same time. But in my opinion, it wasn’t so successfully connected with the whole concept.
Syntax Error ~ Bart Bratke
This installation is the creates a digital sculpture representation from recorded motion data of a dancer and blurring the lines between physical and digital realm through an interactive feedback loop. What is relevant to my concept is the fact that only the body movements create a reference to the otherwise invisible space.
The project uses interesting techniques that I would like also to explore. The performer moves to an interactive noise field, where a simple modification of the random seed could iteratively create new versions of the video, each offering a different composition of the recorded performance. She was recorded by depth Kinect cameras, in which the intersection of the images was later put together to a three-dimensional volume for further process.
Forms ~ Memo Akten and Quayola
“Forms” is an animation based on human motion and its reverberations through space and time. It explores techniques of extrapolation to sculpt abstract forms of athletes, who push their bodies to their extreme capabilities.
What it interested in me in this work is the choice of graphics. Although they are digital, they are like geometrical shapes from metal and even the sound makes you feel that is a moving sculpture. Furthermore, they are constructed in a very successful way so as to understand that is a moving skeleton that jumps or swim. I would be interested to see the production of these visuals in real-time performance from athletes too.
Momentum ~ schnellebuntebilder
This project creates a synaesthetic experience of sounds and visuals generated from body movement. The person and his surroundings transform into a fluid creature in an ever-transforming system of particles. A virtual reality, in which movement becomes sound and music and matter seem to dissolve.
This gradual transition from something physical in something completely abstract and digital is very successful and inspire my future practice.
The Choreographic Coding Lab and Motion Bank
The Choreographic Coding Lab, an outcome of Motion Bank project, offers unique opportunities of exchange and collaboration for digital media artists who have an interest in translating aspects of choreography and dance into digital form and applying choreographic thinking to their own practice.
They have many inspirational projects in this field, like Momentum and Syntax error that I mentioned above. I would be very interested to participate in any future workshop of this lab and explore new techniques in this artistic combination of the ancient art of dance and the contemporary technology.
Peak Performance ~Rachel Rossin
This show explores how we lose touch with our bodies in virtual spaces. The artist had been working for a year on a Virtual Reality series and through this exhibition, she expresses a critical undertone for these systems that she uses to create art. So she created this exhibition in response to the disembodied feeling she got while sculpting in VR. As she claims “Instead of looking at a reference image, I was recalling the memory of what having a body was like. In VR, you feel like the memory of a body, the emotional memory of a body. I thought about what parts of my body I remembered.”
In the specific interview, her words are very inspiring for me, but also her artistic practice. She created paintings and virtual reality environments made of plexiglass pieces, which were 3D prints of those environments. She printed the paintings and then used a blowtorch to form these substrates. She put on a flame-retardant suit and folded the plexiglass around her body, giving it these kinds of impossible hugs.
This abstraction and metaphorical connection with the feelings of a continues VR experience is magnificent. I would love to use her technique for transforming the acrylic pieces of hands and feet that I have created, to give the visitor the feeling of hugging from a plastic material.
Phi ~ Stratofyzika
Phi is an audiovisual dance performance inspired by the slowly changing, repetitive structures of minimalist music, and their incremental alteration of perception.
I am interested in the way that they use the transition between light and shadow, orbiting the dancers’ bodies, alternately illuminating and obscuring. But also, the sound choice that emits poly-rhythmic, electronic soundscapes where the dancers express a repetitive continuity of movements, like experiencing and remembering.
Warehouse Samba ~ Gabriel Shalom
In this dance film, a trio of dancers moves through an empty industrial warehouse. Contact improvisation technique creates encounters between bodies and architecture. The sounds created by the dancers’ interactions with space serve as concrete music.
I like that the dancers dance to music made from the sounds of their own movements. For my own future practice, I would be interesting to use this technique and going one step further, I would like to transform the physical sound that the dancer creates with her movements into a digital form, so as to add value in my concept.
Riders ~ Lenka Vagnerova
Riders is a dancing performance that draws inspiration from birds. Birds are witnesses to our lives and stories, but their worlds are elsewhere and it is a world of different dimensions.
Seeing this performance executed from the dancing company VERVE, I made a symbolic connection with my concept, because of this idea of looking in a world of other dimensions, a non-human place.
I inspired also from the strength and energy of this choreography, and also, I came across the music that I wanted to use for my composition.
Concepts
Nostalgia
In the initial idea, my goal was to create a Machine and a Choreographic Installation to criticize the discomfort and disembodiment of futuristic experience machines. To do so, I wanted to create two different situations. Firstly, a person- the audience, connected to a haunted machine that makes him/her feel loose of control, only passive visual and audio receiving experience and disembodiment. This would be a symbolic representation of the experience machine. Secondly, a “blind” and lost body in a plastic world — an unstructured no-space. My concept based on the memory of the body that virtual systems make you feel. When you are connected and stay inside for hours you forget your real body and you focus more on the virtual one. That’s why I entitled this idea “Nostalgia”. In this way I wanted to locate an expanded awareness of “how we physically engage the world, using our entire bodies — not just our heads”.
Symbiosis
In this concept, my initial idea changed and focused more on the creation of new forms of embodiment communication. In this scenario, there are three layers. In the first layer, a body is trapped in a virtual world that the audience cannot perceive. The person disappears under the digital layer of the virtual self of this body, that is ultimately dependent, symbiotically, on the physical body. These bodies swing between oblivion and memory, experiencing an endless fusion between real and virtual. The audience is connected with this symbiotic organism through external tactile stimuli and is being questioned about what does experience mean to us. In this way, I create a new form of experience that can contribute to the field of embodiment communication in virtual spaces.
Implementation
In my first prototype, I tried to explain the different roles, the scenario, my setting, the interaction and the visuals that I wanted to present. So, I made a mockup from paper with the two roles of the dancer and the audience and I highlighted the points — hands and feet — of tactile connection between them. I also had implemented some interactive graphics using VVVV and particle systems. I created them based on some tutorials that I watched, so as to learn the software.
Finally, I had created a connection between two wireless Arduino boards with buttons and led to show the output that the one role would have from the input of the other. For the whole Arduino coding, I was helped by Tom Lynch, a prototyping lab technician.
Later, I started working with the Kinect 2 in VVVV, that recognizes people’s skeleton. It was very useful that I used Vinzenz Reinhardt’s implementation of a particle system to create my final visuals. I also changed the Arduino buttons system to e-textile buttons and I used Adafruit Flora and Lilypad for the performer’s hands and feet input. The wearable workshop from Emilie Giles was very important in order to know what I need to do with this technique. Additionally, I used vibrators instead of LED, to create the tactile stimuli that come from the performer to the audience.
Trying to experiment with a video edition of this implementation I collaborated with Beatriz Lacerda to film and edit it. We filmed the choreography in an LCC room that was ideal for the concept of the differentiation between a sunny physical space and a body trapped in a digital no-space, seeing through a dark headset. In the editing used particles, developed in VVVV, that follow the skeleton and the movements to create this blurred line between the physical and virtual self. The editing of the initial material made with Adobe Premiere and After Effects. My goal is to improve this video and to create a tactile sensation for the audience that watches it when I am not presenting the performance in the exhibition.
Here you can watch the first video that we created:
My final implementation takes place in a dark room, where I dance using 2 different songs from Ivan Acher’s archive. He is a specialist composer in contemporary performances music. So, I asked his permission to use his music and we talked about future collaboration for an original music composition. I also collaborated with Efi Klidara, a dancer and choreographer, who helped me to choreograph the performance. I took inspiration also from other dancers, like Maria Papadopoulou, Andre Kamienski and Faye Revlon. I was practising the choreography in a Student’s Union activities room for a month. The goal of the choreography is to show a body that is always connected in a virtual world. It sleeps and wakes up there. It experiences a passing through from something that existed in something that is coming, between and betwixt a no-space. At the end of the choreography, there is a feeling of discomfort from so much seeing and over experiencing. The body acts like wanting to escape from this world that is trapped.
I use an invisible fabric on which I project real-time skeleton made of particles that represent the virtual self that is trying to be shown out of the glasses. This body is like playing with the particles and at the same time, it is made of them. This fabric screen which stands in front of me symbolizes also a divider between the 3 symbiotic organisms, the physical body, the virtual one and the connected audience.
The props that I used for the performance are some handmade glasses of good quality plastic paper, that symbolize a virtual reality headset, but also a costume, that I sew with the e-textile buttons, highlighting the connection points with black fabric. I have also laser cut a pair of feet and hands made of acrylic, where I have placed the vibrators that are connected with the buttons. So, the audience is called to stand in the feet and touch the plastic hands to feel the external stimuli of the dancer’s movement. The goal of this construction is to make people empathize with the other person but also with the no-human — alien body. The choice of the cold hard plastic material was made to create a distance of the physical body.
Conclusion
Symbiosis was a really interesting project for me. I experimented with many different technologies and methods. I also collaborated and took feedback from a lot of people. However, I believe that the project started as a speculation with the Experience Machine and the discomfort of these technologies, but then the final result is a new way of experiencing. If I would do this again, I would change or improve some things as I said above, but also, I would try to include fewer components, to be clearer. On the other hand, through this project, I gained experience in many different prototyping platforms and I experimented both with technology and tangible constructions. The final video and the photos documentation of the darkroom are not so successful, as they created and edited in the end in a limited time. My future plans are to improve the project and the documented media in order to present it in Ars Electronica.
Bibliography
Akten, M, and Quayola. Forms. http://www.memo.tv/forms/ (accessed 18 June 2018).
Badiou, A. (2012). Plato’s Republic. Cambridge, Massachusetts: Polity Press
Bardainne, C., and Mondot, A. (2013). Hakanai. https://www.am-cb.net/projets/hakanai (accessed 18 June 2018).
Bennes, C. (2018). Over-simplification and over-production in the context of a research-based transdisciplinary art practice. Seminar presented at the Interaction Design Communication course in LCC. UAL. London
Bratke, B. (2015). Syntax Error. http://bartbratke.com/portfolio_page/syntax-error/ (accessed 18 June 2018).
Carter, T. (2017). Frank Kolkman creates death simulation machine using virtual reality. Dezeen. https://www.dezeen.com/2017/11/23/frank-kolkman-outrospectre-near-death-experience-virtual-reality-technology-robots-health/ (accessed 18 June 2018).
Dunne, A., & Raby, F. (2014). Speculative Everything: Design, Fiction, and Social Dreaming. MIT Press.
Fasse, S., and Belenguer, M. (2017). The Look Elsewhere. http://www.morganbelenguer.net/projects/le-regards-ailleurs/ (accessed 18 June 2018).
Joreg (2017). NODE17 workshop material. VVVV. https://vvvv.org/blog/node17-workshop-material (accessed 18 June 2018).
Klemmer, S., R., Hartmann, B., and Takayama, L. How Bodies Matter: Five Themes for Interaction Design. https://hci.stanford.edu/publications/2006/HowBodiesMatter-DIS2006.pdf (accessed 18 June 2018).
Merleau-Ponty, M. (2002). Phenomenology of Perception. London: Routledge Classics
Motion Bank. (2013). http://motionbank.org/en.html (accessed 18 June 2018).
Nozick, R. (1974). Anarchy, state and utopia. Oxford: Blackwell
Oertelt, N. Rachel Rossin’s Virtual Realities: How we lose touch with our bodies in virtual spaces. Fold Magazine. http://www.foldmagazine.com/rachel-rossin-vr (accessed 18 June 2018).
Owen, A. (2015). Body talk. Gatehouse Gallery. http://www.dirosaart.org/wp-content/uploads/2015/06/BodyTalk_brochure_web.pdf (accessed 18 June 2018).
Pallasmaa, J. (2012). The eye of the skin: Architecture and the Senses. John Wiley & Sons
Schnellebuntebilder. (2015). Momentum. https://schnellebuntebilder.de/projects/momentum/ (accessed 18 June 2018).
Shalom, G. (2015). Warehouse Samba. http://www.gabrielshalom.com/#/warehouse-samba/ (accessed 18 June 2018).
Shklovsky, V. (1917). Art as Technique. https://warwick.ac.uk/fac/arts/english/currentstudents/undergraduate/modules/fulllist/first/en122/lecturelist-2015-16-2/shklovsky.pdf (accessed 18 June 2018).
Stratofyzika (2017). Phi. https://stratofyzika.com/Phi (accessed 18 June 2018).
The Choreographic Coding Lab. (2017). http://choreographiccoding.org/ (accessed 18 June 2018).
Vagnerova, L. (2018). Riders. Vimeo. https://vimeo.com/258975033 (accessed 18 June 2018).
Watkins, H., and Esse, M. Down with Disembodiment; or, Musicology and the Material Turn. https://www.esm.rochester.edu/uploads/WatkinsEsse_Down-With-Disembodiment.pdf (accessed 18 June 2018).
Westhof, R. (2014). Video tutorials. VVVV. https://vvvv.org/documentation/video-tutorials (accessed 18 June 2018).
Wiegel, L. (2010). Perception in the digital age: Analysing aesthetic awareness of changing modes of perception. https://dspace.library.uu.nl/handle/1874/179589 (accessed 18 June 2018).
Wiser, R., Okenwa, P., Júdová, M., and Boleslavský, A. (2018). Smoke Screens. https://camouflage.org.uk/ (accessed 18 June 2018).
