Art, technology & our greater environment
Crystalizing the imperceptible with computers
Media art is a field of art that adopts an original approach : as Vera Molnar worded it nicely in her Éloge de l’ordinateur (dans les arts visuels), computers allow artists to establish processes and experiment with the results, rather than sanctifying the output, as if it came from a godly inspiration. In media art, the conclusion is not a definitive product : the creator is conducting a study, following patterns and methods, hoping to crystalize an original concept.
I believe this makes computers — and more generally, technology — a very adequate tool for producing art that looks at the world. You could call this extrospective art, in the sense that identity and self-reflection are mostly ignored, in favor of observable realities and shared experiences.
After all, that’s what technology is mostly built for : sensing, monitoring, recording. And much like paint brushes, I try to view computers as tools above all, rather than actual objects of study.
What I would like to discuss it here, is the encounter of such approach with our natural environment. The experience of wind, sound, or tides, are no mystery to most of us ; and we know their importance in the grand scheme of things, we feel their power too, yet we often lack perspective on them.
So with that said, how could we make use of the capabilities of our technology to emphasize the feeling of the greater things out there?
A couple of relevant artworks
Whether they make use simple electronics or actual modern computers, such pieces of art have been developed for a few decades. I’d like to present 3 here.
For the record, I personally haven’t experienced any of these in real life, so my interest in them stems more from their concept than their execution.
“Bell”, by Soichiro Mihara (2013)
“Bell” is a sound installation by Japanese artist Soichiro Mihara that was exhibited at the 2017 edition of Open Space at Tokyo’s NTT ICC art gallery.
The setup consists of only a few elements : under a glass dome, a bell is connected to a geiger counter. When the counter detects a radioactive particle, the bell chimes.
Through its shape and staging, the artwork references traditional wind bells and dream catchers, while making use of modern tools to make tangible a measurable phenomenon. In doing so, it creates awareness in the viewer, expectation, even fear.
“The sound of Ice Melting”, by Paul Kos (1970)
Paul Kos is an American conceptual artist whose work might not qualify as media art, but greatly made use of sound and video, quite early on.
In “The sound of Melting Ice”, multiple microphones are pointed at blocks of ice, capturing and amplifying their sound, as they slowly melt in the ambient temperature of the room.
The fact that this piece makes use of common stage microphones rather than a specialized array of sensor, is crucial : the goal is to give a voice to the ice blocks, and make them the center of an improvised orchestra.
“Wellenwanne”, by Carsten Nicolai (2001)
Carsten Nicolai is one of the most prominent media artists of our times.
While his work might seem cold and and intellectual, I find the viewpoints that he develops to be both humble and poetic.
His approach mainly consists of showcasing physical phenomenons — the way sound propagates, the reflection of light, the texture of materials, etc.
The piece “Wellenwanne” is a clear example of this : here, a tray of water is placed on speakers, and as the audio frequencies play, their sonic waves materialize and propagate on the surface.
Similarly to the rest of his works, Carsten Nicolai makes poetic versions of measurement tools. With “Wellenwanne”, the oscilloscope becomes an elegant observation device, powered by the laws of physics, transforming energy into shapes and patterns.
In all of this, computers
In many ways, these artworks remind me of Mono-ha — a conceptual art movement that began in the 70s, led by Japanese and Korean artists.
Mono-ha deals with perception, shapes, and above all, material : it exploits the tension between industrial and natural matter through carefully staged installation, sometimes ephemeral, always deeply minimalist.
I believe that the approach of Mono-ha provides a particularly relevant perspective when applied to computers : it highlights the difference in essence between the programmed silicon and the seemingly inanimate. The latter just is : it often appears as static to us, or just under the direct influence of the harsh elements, and the matter it is made of feels familiar. The former, while made of the same matter, moves and behaves with dedication, following the principles that it has been built for.
This creates a notable tension in their scale as well : on the one hand, our machines are efficiently built to acquire data and monitor arrays of sensors in real time ; on the other, glass, concrete, ice, all evolve over a radically different time scales. They reveal their dynamics over long periods of time, following chaotic patterns.
The resulting aesthetic of this confrontation itself is interesting, in my opinion ; it tells a story about what computers are, how they’re positioned in their environment, as if robots stopped their task and started smelling the flowers on their way to work.
The simplicity of the approach also feels like a breath of fresh air, in today’s art landscape : this isn’t about the Self and the expression of the ego, it’s an invitation for quietness and observation. And while it isn’t an explicit injunction to care for the environment, I do believe that such focus can help viewers create a bond with their surroundings, and watch the natural cycles.
It’s in the small things that the big changes reveal themselves.
A personal attempt
For the 2019 edition of Media Ambition Tokyo, I was given the opportunity to work on a piece of my own. I wanted to experiment with the ideas expressed above — the installation Study for a surrounding entropy is the result of this process.
The idea emerged when I first heard of Software Defined Radio (SDR).
Contrary to a typical FM radio receiver and its specific application, an SDR dongle is a piece of technology that allows you to capture raw radio data over a wide spectrum of frequencies, demodulate it, and decode it as you wish, all through software.
In media arts, I tend to consider the choice of the tools to be part of the overall coherence of a project. SDR happens to be just perfect here : with a simple 25$ USB dongle and a piece of antenna, you get access to an entire world of information, from mainstream radio stations to satellite images. The usefulness/simplicity ratio of the setup is huge, and this precisely reflects the philosophy that I want to convey.
Here, what interested me wasn’t the meaning of the captured data, but the bigger picture behind it : the electromagnetic stream of information does not only contain human telecommunication and interferences, but also atmospherical noise, emerging from random events happening on the planet and its surroundings.
This noise is what we would observe as static noise on a TV. And while it seems more of an inconvenience than a subject of study, I would argue that this is specifically what makes it intriguing : it is the ubiquitous signature of the chaotic state of our surroundings, at a given moment, in a given area. It’s random, its entropy is high by nature, and its chaos will only be disturbed by the predictiveness of man-made communications.
This is the observation that I wanted to put on display.
Technical setup
While I don’t believe that the technical choices that I made are particularly relevant for the end result, I would like to go through some of the the tools that were used, for the sake of an engineer’s curiosity.
The installation consisted of three main components : A capturing device, i.e. the USB dongle connected to its antenna, a vertical display, and a single speaker.
The Radio dongle
Using a python script, the electromagnetic spectrum is scanned at a slow pace : every 2 seconds, the SDR dongle tunes into a new frequency, estimates the randomness in its vicinity, and orchestrates the other components accordingly. The slow rhythm at which it operates is a relevant piece of information here : the goal is to mimic a natural clock, like a heartbeat, with which viewers can synchronize themselves.
This component acts as the core of the system, the maestro, sending cues and measurements to the others. I personally like to see it as a crystal ball : because the antenna is exposed, touching it will actually make you the antenna, and it will alter the signal.
Generative audio
The audio playing through the single speaker is a mix of the raw radio stream, as it is captured in real time, and some additional generated frequencies and percussions to emphasize the intended behavior.
The change of frequency for instance, as we compared it to a biological clock, is mapped to a soft tom sound, similar to a sound one could make with their mouth. The measured entropy, on the other hand, produces higher-pitched clicks that convey the way it successively bolts and plunges.
The raw audio isn’t processed, and occasionally lets us hear voices when tuned into a local radio, or encoded binary data.
Natural visuals
The generated visuals are the result of a simple particle system, updating more than a million particles in real time on the GPU, combined with a blur effect, that allows for a more intimate rendering of the harsh pixels.
The movement of the particles is dictated by a field of attractive and repulsive forces, evolving as the radio spectrum is scanned and measured : their strength is increased when a disturbance in the ambient entropy is caught, which as a result, excites the cloud of particles.
This was an interesting approach to experiment with, though my intent was never to make use of a specific rendering technique, but to find ways of creating a smooth, analog-like imagery.
In the past, I’ve also experimented with ray-marching, which provides an interesting insight into the very meaning of 3D objects, allowing to alter the rules of physics and implement analog-inspired visuals. This technique tends to require more computing resources though, and performs poorly for high-resolution renderings.
Taking a step back, it seems that the display here essentially does one thing : in converts the static TV noise into an organic shape. It looks like a microscope, and acts like an oscilloscope, providing a visual interpretation of a measurement.
And the venue of the Media Ambition Tokyo 2019 exhibition could have hardly been better : located at the 52nd floor of a skyscraper, facing the Tokyo Tower antenna, it was in a privileged position to capture a rich stream of data.
Thoughts & future improvements
From a technical aspect, the capturing device turned out to be barely sensitive to local fluctuation of the electromagnetic stream. Touching the antenna with a bare hand did create noticeable variations of intensity, but approaching a mobile device would only produce effects on specific parts of the spectrum. While interactivity was not a key component of the project from the start, it did create some confusion among the visitors.
As the title states, this project was mainly a study, meaning that it wasn’t intended to be a final artwork, but to provide a starting point for my own reflection on the topic. Ultimately, I’m starting to believe that computer screens and other virtual spaces aren’t that good of a medium. Most of the visual effects and sound textures that I wanted to create, do not benefit from being digital.
When they’re not the subject of the reflection itself, computers are mostly just convenient tools to make things work efficiently, but non-virtual solutions might as well be preferred. Here, the display could be replaced altogether with light diffraction mechanisms, and the speakers, by percussions and strings. The artworks that I referenced above do not use displays either, and I believe this serves them for the better. It simply removes the barrier of the virtual, and confronts viewers to raw objects.
Special thanks to Takayuki Miyatake (video), Tatsuya Motoki (hardware), Miyu Hosoi and the Qosmo team, for making this project happen.
Thank you for reading till the end. We are Qosmo, Inc. a collective of Artists, Designers, Engineers and Researchers. Read other Medium articles from Qosmo Lab and if you are so intrigued to find out more, get in touch with us from here. We are actively searching for new members, collaborators and clients who are passionate about pushing the boundaries of AI and creativity. Ciao!
