Building the world’s largest outdoor AI artwork
Our summer got unexpectedly hot. We had just finished an art exhibition at the Kunsthalle Tübingen, where we showed new visual phenomena on AI & expressionism as AI artist collective Lunar Ring. In late June we were approached by the state ministry of Baden-Württemberg, Germany, with an inquiry to create an interactive AI artwork for the Einheits-EXPO2020. The event serves to remember 30 years of German reunification by means of a 30 day long outdoor exhibition in Potsdam, the capital of Brandenburg, right next to Berlin. An interactive outdoor exhibition in public space that might be seen by about a hundred thousand visitors? Sounds great!
The idea was straightforward: create an interactive artwork that highlights Baden-Württemberg and it’s Cyber Valley Initiative as the epicenter for artificial intelligence on our continent. To do this properly, we had to think big and act fast, as the exhibition would already start in a mere five weeks. Summer plans? Goodbye! Free time? Maybe in winter. Too tempting to build something monumental to be set out in the public space.
Put simply, our plan involved capturing the scene at the front, feeding it to a supercomputer and teleporting the results into a gigantic display. Thereby, the work shows an altered mirror image of reality, all created by a live-dreaming AI. We wanted the observer to dive in and become part of an art piece that is generated by machine intelligence and live-streamed to the internet.
Skip all prototypes!
The idea born, all that remained now was to organize the hardware, write the software, to find a suitable name for the artwork and to throw everything over board that we knew about project management, due to time reasons. Name-wise, we decided for “Grenzauflösung”, translating to “dissolving borders”. It reflects both on the German reunion, while also implying dissolving borders between us and an ever-faster-evolving AI. Finding sponsors for the missing hardware was a bit more challenging, as the summer break and the ongoing corona pandemic didn’t make it any easier. However we got lucky, as LG Electronics’ Information Display business unit was ready to provide us with a beautiful array of their digital signage displays, which fit the special requirements of the exhibit’s environment, to form one giant video wall, spanning more than 200 inch (5.5 meters). NVIDIA, the leading computing company for AI, made available a DGX Station powered by 4x NVIDIA V100 GPUs. We were also happy to receive an excellent wide-angle camera from the leading producer of industry cameras, IDS Imaging. To bind everything together and help with all organizational and functional as well as technical aspects, we could rely on our strategic partnership with Bechtle AG, Germany’s top-notch IT system integrator. All that was lacking now was the AI software to make the dream come true.
How does it work?
Our traditional weapon of choice for turning ordinary pictures into reasonable artworks has been Neural Style Transfer. Possibly because this wizardry technology indeed has been conceived in our hometown Tübingen by Leon A. Gatys. In short, the idea is to recycle a pre-trained object detection network. The network has taught itself to see and categorize our visual world, via the magic process of optimization and has been subjected to millions of images. Over the course of training, the network implicitly understood what makes a cat a cat — and how to distinguish her from a dog or boat. However, when using the input image as drawing canvas, the object detection networks becomes the artistic itself. Perception becomes imagination. Pixels of the input image are adjusted, so they collectively produce specific patterns in the high-level representation of the object detection network. Crucially, it tries to mimic the patterns and relationship evoked from another image, the style image, by weaving in it’s structures and elements. Watching this process is mesmerizing and often reminds of crystallization. Below you can see an example, where we optimized the castle of Stuttgart to reflect properties of Kanoldt’s painting “Olevano”.
Unfortunately, however, the classical neural style transfer is unsuitable for creating an artwork that is interactive, as the runtime is about a minute, even for our most highly-bred algorithms. Thus we had to change gears and go for a short-cut solution without the need optimize for every image. Luckily there exists plentiful research under the umbrella fast neural style transfer. Instead of optimizing the image iteratively for many hundred iterations, on can use a single-shot forward pass of a specialized neural network, with the goal of imitating the “slow-but-beautiful” style transfer. This allows for interactive real-time application. After extensive comparisons and tests, we found the SANet approach proposed by Dae Young Park and Kwang Hee Lee to be most shiny for our purpose. It creatively weaves a new art work respecting the semantics of the input images in an interesting way.
Next we optimized the architecture and representation to get the most out of the four liquid-cooled V100 GPUs. To strengthen aspects of our work considering re-interpreting reality itself, we included additional features such as ghosting and augmented reality. Our final models were trained on different transformation styles, such as plants, flowers, pointillism, fractals, and many more.
Final assembly
With the help of a team of seasoned exhibition constructors and event managers, assembling the final system in Potsdam was smooth sailing. Our exhibit immediately caught attention, as it is located at the prominent avenue Hegelallee, where thousands of pedestrians and cyclists come by everyday. Seeing an artwork including themselves on a giant screen makes most people stop and wonder. The most rewarding and unexpected aspect of our exhibition was how reliably it brings a smile onto the faces of young and old. It elicits a cheerful and almost childlike behaviour, letting the observer melt into a piece of art that only exists for this very moment. Watch our live-stream here! — or come and visit us in Potsdam until the 4th October 2020!
Lunar Ring is an AI artist collective based in Tübingen, Germany. At the same time, we are running an AI/ML company, Colugo. Our mission is to transfer cutting-edge AI technology to our customers in Industry and MedTech. Creative artificial intelligence is at the core of what we do, connecting Lunar Ring and Colugo. For inquiries, please email lunar.ring.art@gmail.com
