Your Art’s Information Has A Shape
Part 2
During Joseph Nease Gallery’s somewhat recent “Catching Up / Resurfacing” show from April to June of 2023, “Frau Holle’s Werkhaus” (see video above) emerged as a captivating mixed-media installation, where the ancient art of hand-crafted crochet meets the cutting-edge realm of experimental video projections.
Digitized moving images (the grandchildren of the static camera) were key to this work.
Viewers were invited into a modern retelling of the classic German fairy tale, Frau Holle. Drawing from the rich tapestry of oral tradition, literature, and multimedia adaptations — including the notable translation by Jack Zipes in “The Original Folk and Fairy Tales of the Brothers Grimm: The Complete First Edition” (2014) — one of the most intriguing parts of this installation was its intertwining of fiber arts with AI. Artist Joellyn Rock tilted, as so many artists do, head first into machine learning as an extension of herself.
This fusion of traditional fiber arts with AI underscores a broader cultural shift towards the digital preservation and reinterpretation of our collective stories. As we witness the transformation of art forms like photography into essential tools for digital archiving, Rock’s work exemplifies how even the most time-honored narratives can be reimagined. Her approach mirrors the wider movement towards integrating technologies to both conserve and creatively expand upon our historical and cultural legacies.
As digitization has become a priority for archiving every sort of data, photography is suddenly integral to the process of rendering everything analog into a digital format. Whether we’re talking about the creation of point clouds for 3-dimensional objects, the recreation of 2D film as jpegs, or the cataloging of biological, archaeological, or geologic activity, the camera is needed.
Photography is currently the preferred gateway from the physical to the digitized world. There are of course alternatives, such as LIDAR for 3D scanning, the Radio-Frequency Identification (RFID) and Near Field Communication (NFC) tracking of everyday objects and spaces, or the Internet of Things (IoT) more broadly. But in 2024, these are far less omnipresent in the minds of most individuals (regardless of whether or not they use smart objects with RFID or NFC technologies in their home products).
With the current frenzy over digitization, everything that can be cataloged either already is or is in the queue to become a silicon simulacrum. This means every piece of famous 2-dimensional art has been “quantized” — packeted into a distinct package of pixels that our easily manipulated perception takes for the original image.
As an example, here is a representation of Claude Monet’s Haystack series.
The above artistic representation is a text-prompted machine generation, in some ways “creative” to the extent that we don’t know what it’s going to create before it’s done. But certainly created via machine learning, regardless of how ingenious or disingenuous we feel the end result is.
This indicates a new shift in the way visual art is represented online. More than just the end product itself shows us, for with generative art the end necessitates an ingestion of data at the beginning (in this case, Claude Monet’s complete Haystack series). The original works of artists are certainly still recreated through relational databases by the proper arrangement of pixels, colors, rows and columns. But that doesn’t explain the seemingly spontaneous generation that mimics original work.
Visual art has been transformed into something that allows machines to recreate it, and nobody has really cared much about it until platforms like MidJourney came into wide use. These models employ something called a vector database.
Vector databases represent novel building blocks of data organization and structure. The way pieces of information are arranged, related, and queried is taking on entirely new geometries — shapes that have far-reaching implications. We are only beginning to tap into the knowledge that can come from placing information elements in geometric relation and proximity within high-dimensional vector spaces.
As such, vector databases embody a distinctive convergence of intelligence and beauty — a nexus where artists like Joellyn Rock are pioneering new paths.
As mathematical morphologist Dr. Solomon Marcus presciently noted decades ago, anything ingested by a vector database can no longer be considered a single, well-defined object. Once it’s been embedded, it inhabits a multi-dimensional space with a complex vector structure. This means it actually has a shape. As explained in the video below, entire datasets now have morphologies associated with them. Shapes we can use to mathematically map out aesthetic territories.
What patterns, insights, and discoveries await as we explore the universe of associations captured within the rapidly evolving vector shape of information? The possibilities span from revolutionizing how we search for knowledge to using vector lenses to view truths about ourselves. The fruits it may bear for humanity remain excitingly unbounded.
To help illustrate how this works for visual data, below is a simplified version of the process using the same Claude Monet Haystacks representation from above.
Following the digital capture of an image’s pixels, each speck of color is meticulously cataloged, revealing their RGB (Red, Green, Blue) values alongside their spatial coordinates. This cataloging is not just a technical endeavor but a reimagining of art as mathematics.
At the core of the example above (Data Analysis via ChatGPT) was a conscious decision on how to represent depth, the Z-axis, which might reflect the intensity of a single color or the synthesis of multiple hues. This decision was non-trivial, as it defines a dimension of perception to the artwork (in the same way Raissa Venables’ photographic perspectives are methodically chosen before she combines them — read part 1 for reference), transforming a flat image into a dynamic landscape of peaks and valleys, colors, and shades. But it could have been any number of perceptual markers, from texture to hue, to anything else we can conceive of.
That’s the ultimate strength of a vector database — it allows for the interpretation of infinite dimensions of perception. Each generative model is different, because each model is defined by different perceptions or perspectives. This is why models like Stable Diffusion provide divergent results from Generative Adversarial Networks (GANs) like MidJourney, even when provided the same text prompts.
To go back to the example above (Data Analysis via ChatGPT), the construction of the 3D scatter plot from this data resembles painting with numbers, where each pixel is repositioned within a three-dimensional space, guided by its original location and the grayscale intensity — a metric of its luminance. This intensity is thoughtfully calculated from the RGB values, employing a formula that approximates the complexity of human vision.
The final product of this intricate process is a 3D scatter plot, akin to a digital sculpture, where the X and Y axes map the pixel’s original domain, and the Z-axis reveals its depth, quantified in shades of gray. The plot’s colors remain faithful to the original artwork, each point a tribute to the original’s vibrancy.
As fascinating as all of this is, it’s a watered-down version of how generative models actually work. To ensure clarity, we “downsampled” the parameters, in attempts to preserve the artwork’s essence without overwhelming the reader. Real models like MidJourney employ many more dimensions than we can easily conceive of.
Multidimensionality is key
It’s essential to note that real vector databases are truly multidimensional, encompassing far more information than mere pixel location and grayscale intensity could possibly convey. Some of which we understand, but other pieces of which are more or less dictated by what works (independent of our understanding it or not). As noted above, the coloration in the plot diagram is included for ease of human understanding. Though in a more sophisticated analysis, this sort of information could be conveyed along other vector database representations in a hyper-dimensional space to show how different works of art relate to each other.
This hyper-dimensional representation can capture a fuller spectrum of an artist’s body of work, offering a richer, more nuanced understanding than traditional two-dimensional or even three-dimensional visualizations can provide. This advanced approach allows for a deeper exploration of the artwork’s essence, bridging the gap between the tangible art of the past and the server-stored forms of our present.
Joellyn Rock’s innovative work with AI and generated images stands at the forefront of this fascinating intersection. Her approach to creating art is emblematic of how texts and images transcend their traditional boundaries, evolving into entities that are no longer singular or static but dynamic and multi-dimensional. By harnessing the power of AI, Rock transforms narrative and visual elements into complex vector structures, embedding them within a rich, multi-dimensional space that challenges our conventional perceptions of art.
This integration of vector databases into her artistic process allows for a deeper exploration of the themes and stories she seeks to tell, offering a nuanced layering of meanings that can be navigated and interpreted in multiple ways. The complexity of vector structures, with their ability to encode vast amounts of information in a compact form, mirrors the layered depth of Rock’s generated images. These images, rich with the interplay of intelligence and beauty, invite viewers to engage with them on multiple levels, uncovering new insights and interpretations with each interaction.
Rock’s pioneering work not only highlights the potential of AI and vector databases to revolutionize the art world but also serves as a compelling example of how technology can enhance the creative process. By blending the analytical precision of vector databases with the intuitive fluidity of artistic expression, she creates a bridge between two seemingly disparate worlds, revealing the inherent beauty in complexity and the intelligence behind creation. In doing so, Rock not only pushes the boundaries of what art can be but also opens up new avenues for exploration and understanding, marking a significant contribution to the ongoing dialogue between art, technology, and human perception.
