Spatial Projections

Article for Veto Magazine #25, February 2013

The method of 3D projection mapping was born at the moment when the need for a new level of space in electronic audio visual performances met the initiative of visual performance artists. In this article the author looks into the variety of projection mapping, as well as dwells upon accomplishments of Finnish and Lithuanian artists.

The method of 3D projection mapping was developed owing to the artist’s aspirations to modify and destroy the face of classical screen; this method has provided video artists with a possibility to expand the meaning of a space. Due to recent changes, nowadays visualizations can be projected on spatial objects and predefined surfaces, as well as can be used both in architectural and stage design. The method has supplemented or, to be more precise, expanded the boundaries of visual expression.

An opinion exists in the contemporary art that space in itself is also a medium. Usually artists — those who fill up galleries — stick to this viewpoint. When conceptualizing the content into the form it often needs to be expanded to the extent where art reaches to walls and interacts with them. Sound visualizers or video artists who are responsible for the final visual output at electronic music events and video artists at festivals of light who work on facades of objects, have similar space perception. The concern about filling up the space has led representatives of all the above mentioned categories to one common result — development of projection mapping method.

Projection mapping or real time projections adjusted to spatial objects is a recently developed body of electronic audio visual media methods which allows stepping over the standard size projections, multiplying the picture, as well as adjusting it to various surfaces.

Further in the text the method will be considered from various viewpoints — the floor will be given both, to specialists of the given discourse and artists as well.

Projection mapping

In the course of our discussion with the Latvian motion graphics experimenter and projection specialist VJ Wickiss or Victor Keino we will try to specify the classification of projection mapping and to work out what should be taken into account and what must be understood in order to be able to work with this method.

Projection mapping can be subdivided into two basic types — projections meant for specific surfaces and projections adjusted to the surfaces. By first subtype we usually understand installations and objects of art, as this subtype does not include in itself texturizing free spaces. The video material is prepared so that it can be shown in one go. The second subtype is more experimental, suitable for real time performances and Vjing — the process of texturizing surfaces includes remixing samples, cutting out and adjusting fields to various surfaces.

Back to the first type it should be noted that when making projections for certain surfaces, the work is first done in 3D and animation software — classical animation methods are applied, such as lightning and special effects, narrative. The most classic example is projection mapping on architectural facades used at light festivals. The buildings are modeled first (or ready-made, modeled prototypes of them are used) and the story is created taking into consideration the design of the particular building. The process is connected with very accurate work where each mistake (let it be no more than a millimeter) both, in animation process or in the projection on the particular surface can be seen as a bias which looks like a defect. Therefore this process cannot be associated with real time performance, if only pressing the “play” button” is not considered to be an artistic performance.

Thus we have reached a topic connected with the significance of electronic and experimental music events in audiovisual performance context. Real time interaction of visualizations and sound is exactly what characterizes performance. Its elements make up the meaning in contemporary culture space. Projecting visualizations using mapping method performance obtains spatial and much wider meaning. Applying the method of surface texturizing the story can be told on several planes. The same three to ten layers can be used in video sampling, as well as programs meant for projection mapping to place the layers in different places in a room. Unlike mapping projections used in installations the story usually tends to be more abstract and the particular moment’s caprice (emotions, expression) is added to it, making the performance a unique event. Once in a discussion with Latvian video artist Martins Ratniks we came to a conclusion that abstractions in video art and performances act as a separate language. It is more likely to be felt, not seen — metaphysical state in particular time is expressed in geometrical symbols and multiplied pictures. Projection mapping method helps this kind of projections live in any room, on any object.

Regarding technical side VJ Wickiss reveals the history of the type of a software which allows the usage of such a method. Everything started as a problem — sometimes there was no place for the screen at some event or other, or it was not delivered, or else — it was placed in an absolutely unsuitable place where the light beam was broken partially on the ceiling, on one of the walls or even some other surface. There was no appropriate software at that time to place the projections efficiently in the room — it was only possible to zoom or turn the beam up or down.

Finally specialists of the field worked it out — if gamers had various advanced features for using a room in 3D environment then VJs could use the same method for dividing the room by walls and thus providing a more in- depth overview — leaving the classic standard screen. VJ Wickiss adds that in such case the narrative can be a little damaged. As soon as the picture becomes deeper, the story cannot be gathered any more — the room is not perceived as one screen, as a medium any more. Under the new conditions the room immediately turns into a light effect.

In projections meant for particular surfaces light and shadow interaction has always been topical — it is an important component of 3D graphic and is still being used. The first platforms where the early projection mapping method could be used were Processing, vvvv, Max/MSP, which used OpenGL (Open Graphics Library) environment as it allowed to stretch the screen whatever way it was necessary. These possibilities first appeared approximately in 2006. Users of standard VJ programs Resolume, Modul8 only then understood that the method can be very advantageous in VJing processes. Creators of Modul8 started work on MadMapper software as they had understood that a picture should not be scattered but a program is needed which could be a medium between a video source and final output. It is also necessary to provide a possibility to put a picture generated by any other program into this particular program. The unique tool is created for Mac OS; unfortunately nothing so handy has been created for PCs yet.

MadMapper is a simple and intuitively understandable solution which allows making projections adjusted to the surface. This program prevents complications that initially were connected with reinvigorating the idea letting artists and designers focus on creation of static and mobile visual content to texturize physical objects in real time. MadMapper can use Syphon sources from other content generating programs, such as VDMX and Processing. It is achieved by launching a Syphon server in the content generating program. Syphon client, built-in MadMapper, then connects the program via the Syphon server.

Using such a medium in practice is connected with technical quality which is not to be understood as conceptual or visual quality. As it was mentioned before every corner and every edge in the final output must be perfect, because when working with this method, no defect will become an effect. The method is meant to be accurate and precise, and provides that the screen can be slid exactly as it is necessary.

Recently in Tate Modern in London a video object hanging on fine cables, could be seen — light beam projected a picture on some thin, hard surface, possibly an LCD screen. Obviously the screen was mapped on the hard surface, but — either a visitor to the museum had pushed the object out of its exact place, or the screen was not mapped accurately and a thin line of light could be seen on the ceiling. However VJ Wickiss notes that all the VJs and artists do not have to know the method of projection mapping. It would be enough if technical staff knew it perfectly. An artist does not have to be responsible for equipment, but it would be good if they were competent in technologies, at least in order to know the terminology of the field.

One of the most advanced recent artists group is an international visual artists association AntiVJ. It comprises artists who make material interpreting conventional techniques aesthetically and on high quality, breaking their frames, deconstructing the picture and natural environment space. They combine digital video mapping and holographic illusion methods building scenography from pictures, light sculptures and monumental projections. AntiVJ group has inspired artists all over the world.

During the digital art festival Alpha-Ville in 2012 (in London), cooperation of two AntiVJ group participants, Simon Geilfus and Joanie Lemercier could be viewed at the Hackney Empire Theatre — it was a performance made together with real time sound artist Murcof. It was during this performance that 3D mapping, projections and generative video were used. The artists were separated from the audience by black, semi-transparent screen in front of the stage, and only when a light beam fell on the fabric, the white graphic elements of the projection, shining through the screen illuminated artists themselves. Classical elements of visual art were used to create the picture — composition principles and dark and light element ratio in plane which neatly changed from one composition plan to another thus creating a moving and three-dimensional animation. In creation of visuals interactive design principles were also applied. The fact that the work was done on a very professional level was evidenced by visual elements extremely accurate reaction to sound cues. This was a reason why at the VJ forum in Latvia an issue was put forward if some part of the material had not been prepared before — if rendered material had been used and later it had been “glued” together with the background sounds. Huge visual volume did not hinder even for a fraction of a second during the whole performance.

Finnish video artist Petri Ruikka has participated in an AntiVJ workshop where artists presented their self-developed program in openFrameworks environment, and as Petri acknowledges, the program is really a real- time based program developed so perfectly that everything works without any hindrance. The mapped screen was placed so that it covered the front stage making an impression that there is some interspace between the audience and performers, and entering this interspace could demonstrate the presence of another world and time.

Unfortunately it was not allowed to film or take photos during the performance, but entering the word AntiVJ into the Google browser one can find a lot of amazing pictures and videos where AntiVJ accomplishments can be seen.

VJ Wickiss mentions that he had known all the AntiVJ participants even before they became a common association of artists. Almost every one of them comes from a different country and is a specialist of the field. They were the first ones who started making qualitative mappings and they were great. VJ Wickiss says he has not been influenced by AntiVJ members — at least not regarding using mapping projections in his work, but AntiVJ association has given a good impulse to get to know the technical side of this art and made him contemplate how to do it on his own. It was necessary as there was no software meant for mapping at that time. Initially he created masques in Adobe Illustrator and Photoshop programs. VJ Wickiss speaks of his first projection mapping done with the help of a self-made application in Quartz Composer visual programming environment.

Projection mapping practice: Petri Ruikka

The Finnish artist Petri Ruikka tells me that he has come to VJing through the cinema and videos, as well as photography. Petri has to be told about at least because of the fact that it is exactly he who has propelled VJing culture in Northern countries on a mainstream level.
“I got stuck in the VJ world — everything started when my friend asked me to make projections for a certain event. It was in 2002, I did not know anything about the VJ culture of that time. I was somewhat related with electronic music sphere in Helsinki which made a good basis for such an activity,” the artist remembers.

During the first year Petri experimented with different computer programs and equipment, until together with his peer Henrik Axlund founded a VJ community Visual Systeemi. They have visited South America, Russia and a lot of other European countries with their performances. Petri is known in the Baltics and Northern countries as a VJ culture curator and organizer. He remembers: “Already in 2003 I was invited to “Pixelache” festival as a performance artist. In 2003 the focus on VJ culture started and I hope I was the one who contributed to the development of this subculture in its early period of existence. Since 2005 I have been invited to organize events for VJs. I took care of the 2007 meeting of VJs from the Northern countries and the Baltics which was at that time the biggest VJs’ event in the Northern countries.” Around 2008 Petri stopped VJing and got engaged in the commercial field. They both, together with Henrik set up their company Woland; their goal was to work in two directions — running commercial projects and working on video design projects.

In recent years Petri’s activities have touched upon another context of performance art — more concert visualization and video design application in the theatre and dance. Petri has visualized a few classical music concerts thus discovering a rather fascinating world.

As refers to putting mapping into practice Petri mentions two of his most important projects. The first one — “Arch of Neo” — is visual music, a dance movie “New Under the Sun” and MMST short dance video. “Arch of Neo” initially was meant as a real time audiovisual project based on cooperation. After the start-up and early development phase of it, the project transformed into a visual music movie where audio and performances had been separated from each other.

“My initial idea was that it should be a base project so that afterwards I can start working on material for real time performances and a few shorts. I was really interested in investigating a composition of dance movements and abstract animations. I wanted to launch a project where the projection would be seen on a dancer’s body. So we started working together with the choreographer and dancer Anniina Jääskeläinen.The dialogue was very interesting, especially when we were coordinating actual performance materials. We did a test filming with a projector, and then we worked for a full day in studio where we filmed two separate pieces of material — with and without projections. (The material without projections can be seen in the movie “New Under the Sun”). The technical solution of the projections was rather simple — we set up two projectors which covered the dancer’s movements from two different angles and filmed her from the angle where it was possible to avoid any picture in the background. We used a lot of animations and before the filming we rehearsed with synchronized in time motions, coordinating aesthetics and atmosphere. Later I worked a lot on this material to create the final imagery of the picture. All the additional animation was derived from Anniina and original animation movements. Then, Anniina’s choreography and final processing interacting, an interesting loop appeared as a special effect. Something, like a circle…” Petri recaptures.

One more project where projection mapping method was used was the video installation Awake in 2011. It was a commercial project; the contractor was Stockmann and Aalto University. “Every second year the Stockmann department store organizes fashion events where fashions of young Finnish designers are displayed. In 2011 they wanted to make an installation which would serve as a primary attraction in one of the shop windows. I worked on this project together with a designer Lamin Marong.

The theme of the event was “Awake” — the dazed world which exists in all of us somewhere between sleep and wakefulness. We decided to make an installation which would reveal itself from different angles and would be created by applying sequencing and editing between various physical installation layers. The motto of the installation was like this: “The world faced instant pictures, dreams faded from our memories and ambiguous words with strange character combined with particular flashing light sequence which woke you up from your dreams.”

Technically we designed with the help of a combination of four projectors and variable projection surfaces, as well as variable mapping surfaces. We mapped two composition types on the clothes put on two mannequins; it was a very challenging experiment as clothes are not static, it changes form with every movement. We fastened the final composition with a 2D masque and fabric, but it was almost impossible to make it accurate. The background for mannequins — the wall, was irregularly curved and we made a much simpler projection on it which contained much more narrative, thus figuratively describing the environment.

We made a kind of grid as a projection surface on one of the windows. It was done using projection tape, the expensive one which can be glued directly onto glass to make projections on it. The output of the light that was reflected from the surface was so strong that it almost seemed — there were small displays hidden in the shop window. Besides we had to solve our problems with light. As the event took place in May when it is light quite late in Helsinki but the installation best worked in the dark, we created two different cycles of moving pictures which were meant for real time — one for daytime, the other one for the night. One more challenge we had to face was the place of projections as the projection distance was too short. To make projections on mannequins we used two short-throw projectors which were placed next to the mannequins. To make projections on the window grid and the curved wall we used 6K lumen Panasonic projector. Videos were launched from two synchronized Qlab programs on two Mac Minis. The installation was on twenty-four hours a day for a month — it was an interesting durability test both, for the equipment and the software. Matti Pentikäinen, an artist I had cooperated with already before, including the project “Arch of Neo”, MMS and New Under the Sun, made the sound for the installation. It must be said that even considering the problem with daylight, the installation was a success. It was very difficult to shoot because of the various light sources, reflecting from surrounding buildings. Light variations created a lot of variable, falsely displayed surfaces on the installation.”

Petri says about projection mapping method: “I like applying mapping techniques to achieve good specific effects, like it happened with MMST video, and I like the directness that can be achieved using this method, especially directness of certain type of movement. On the whole I think that ability to animate surfaces or be able to create mobile counterpoints on moving and static surfaces is very exciting. With introduction of mapping spatial video has really exploded and a lot of interesting new media have appeared. I myself feel as if I have scraped some surface and found out that underneath there is a lot of other things to be discovered.”

Projection mapping practice: Laimonas Zakas

Laimonas Zakas is only 24 years old, but he has VJed at various electronic music events, festivals and theatre performances since 2008. At the moment he is studying for his Bachelor’s Degree at the Royal Academy of Arts in the Netherlands. He belongs to a Lithuanian VJ union Fullscreen VJs. He is also known for his project Glitchr which was made on the basis of social networks, applying unicode symbols and special applications. In 2011 Facebook closed the project but with the support of international fans it was reopened a couple of days later. In April, 2012 an exhibition Glitcr prints was opened as part of the multimedia art festival Centras in Kaunas, Lithuania and in August Laimonas had a solo exhibition at Jonas Mekas Visual Arts centre in Vilnius. During his working life Laimonas has been VJing mostly at clubs in Lithuania, as well as at the festival “Satta Outside” which has gained popularity in recent years, and such festivals as “Positivus”, “Tundra”, “Loftas Fest” and others.

Laimonas’s first mapping project was in 2009 and it was one of the first mapping projects in Lithuania. “We had seen a few projection mapping videos on the internet and decided to try to make our own one. At that time there was hardly any information about it on the internet and there were no programmes and no plug-ins. We made a masque on a facade photo using Photoshop, material using Premiere and After Effects meant for video processing and played it using special program for VJs Modul8.”

Laimonas confesses that he likes VJing and applying video mapping technique on primitive geometric sculptures — cubes, pyramids, etc. Last year he did business with a producer of electronic music 96wrld — they both together made an audio visual performance Voxels, the structure of which consisted of almost a hundred white boxes.

Laimonas applies both, classic methods — samples or animations, and a little programming (interactive, responsive to sound animation) in his work.

Projection mapping practise: Rimas Sakalauskas

Rimas Sakalauskas holds a Master’s Degree in animation from the Royal Collage of Art in London. Rimas is known as a video artist. He got recognition in the field of art in Vilnius and soon after in the whole Europe with his short reel “Synchronisation”. The artist started making films when he was 16 and continued his activities in the same style for the following decade. Relatively recently he shifted to projection mapping, and according to him this is just another aspect of working with visual material: “Mapping gives me a possibility to express myself in different meanings. I am happy to have a chance to try myself in this relatively new medium. Space and volume or generally accepted values — architecture and sculpture — have inspired me no less than mobile picture.” Lately Rimas is making videos for theatre performances. His future vision is connected with stage performances and concerts.

Rimas remembers that his beginnings in mapping projects were not too encouraging. Between studies he had come home to Lithuania and had been looking for a job to earn his living. So he had an opportunity either to work as a creative director for an advertising agency or make mobile picture graphics for an interactive media agency which was working on projection mapping projects at that time. Rimas decided that interactive media and projection mapping sounds more exciting than traditional advertising. After some time spent working with mapping techniques Rimas learned to implement his own projects. A good colleague of his Mykolas Natalevičius offered Rimas to make visualizations for his concert and six months later they presented and audiovisual project “Voice is a Drone”. It was Rimas’s mapping debut project. Before any new project Rimas investigates Youtube — even when it turns out to be demotivating, not motivating.

Rimas feels very enthusiastic about the possibility to combine programming with animation as it is a way to create absolutely new options and aesthetic approach in his work. However, today he is able to make linear 2D or 3D animations in classical Maya or After Effects programs. Rimas has tried to use effects but he is still the best at ordinary animation. In the future he wants to learn Processing or vvvv programming languages.

Conclusion

Laimonas adds that currently audio visual performance culture is very active in Lithuania. A lot of electronic music events and festivals invite to perform VJs and video artists. Last year in October, a considerable festival “Insanitus” took place in Kaunas where VJs and video artists from Lithuania and other countries made visual installations and did projection mapping on four stages. Projection mapping is becoming a part of club and festival culture in Lithuania. Petri, for his part, comments that within the last few years there have been several nice mapping projections in public space in Helsinki, as well. Projection mapping has been introduced into plays at the theatre, concerts and exhibition contexts. Seeing this new medium flourish in contemporary space, Petri still feels that there is enough space in Helsinki to implement new projects.

Original available in Latvian printed in Veto Magazine #25. Ask pdf version private.