Holland Cotter wrote a brilliant piece for the NYT last month defending the experience of seeing art live, or “IRL” in millennial speak (In Real Life), implicitly condemning the recent hype around digitizing art on the web and doing mobile apps for museums. I’m CEO of Aura, a mobile app for art. You may assume I’m in the other camp. I’m not.
In fact at Aura, we’re passionate lovers of the heads-up slow art experience. We seem to be alone in this view amongst the field of 100s of mobile and web art companies out there. We think location-aware mobile apps and digital services have a destiny to make it easier to slow down and experience round art in the round with less time ignoring the art itself to read labels or futz with a clunky device. We think most app and web developers and supporters of app and digital art projects have totally lost sight of this or never believed it to begin with.
Why does this matter? How can this be? Isn’t a location-aware app by definition about using the app in presence of the art? Let me try to answer that.
Why #artIRL?
What an artwork looks like two-dimensionally is only a fraction of the experience of any artwork, even an ostensibly two-dimensional one. This has been much lamented by critics and art lovers alike since the dawn of photography. When I took old fashioned art history classes in college with overhead projectors and carousels of slides of famous artworks, professors urged us to go out to museums and experience and sketch all the artworks we could in the round.
What those two dimensions look like when reduced to pixels and shrunk to fit a web page or mobile screen is an even smaller fraction of that experience. In fact, reduced two-dimensional images are often incredibly misleading. Seeing one of the smaller Jasper Johns flags side by side with the massive Seurat Grand Jatte scaled to be the same size does no justice to the physical impact of both works. Thinking all of the impact of an artwork is conveyed in its image is a typical techie conceit that resurfaces generation after generation.
Bill Gates’ 1987 “vision” of all the world’s artworks on your screen at home has been reborn in the 2010s as the Google Art Project and as numerous digital art subscription services with dedicated screens like Electric Objects, Depict and Sedition. All these projects seem obsessed with the idea that high fidelity reproduction will somehow help you see all the detail as the artist intended — often to ludicrous extremes, such as the “Gigapixel” captures in Google Art Project which Frieze Magazine so beautifully deconstructed a while back.
That’s not to say these projects aren’t both entertaining and valuable — the old slide carousels served a purpose of at least making the pictorial content of artworks more accessible, and more easily memorable — but where they serve to convince people that they have truly experienced the artworks when they have only gotten a glimpse of them, that’s where I take issue.
The obvious problems with static 2D digitization of art are 3D works, motion-based works, sound works and immersive works and exhibits especially with multisensory dimensions. To these objections, the techies trot out 3D imaging/printing, digital video and VR. These are all great and valuable technologies but they don’t today and probably will not ever be the same as #artIRL.
Maybe there is some future VR (virtual reality) tech that will help with the scale, motion, sound, taste, smell, touch, detail, and immersion problems in some future reality, but today’s screens, whether on the wall or in your hand, are nowhere near #artIRL.
3D imaging/printing is great and I love hearing about projects where 3D printed reproductions can be touched by kids and the blind or remixed into other appropriated artworks. I also am a huge fan a lot of art being made today that natively uses 3D printing as part of the artist’s process. But until 3D printers can print actual wood, wax, feathers and the myriad other materials that artists and makers have used throughout history, experiencing 3D reproductions is still miles from experiencing original 3D #artIRL.
And the problems with two-dimensional static reproduction aren’t limited to scale, motion, detail and immersion. Here’s a partial list of what keeps me going to have art experiences in meatspace:
First, the patina, or aura, of an object’s history. Humans live in real space with real objects and there has always been something magical about objects that have been passed through the generations and ages or that have played a role in significant historical events. Just go to any natural history museum and watch young kids ooh and aah over dinosaur bones, or go to the antiquities gallery of your local museum and feel what it’s like to be in the presence of an ancient sarcophagus. Think about the wars fought over the authenticity of Biblical relics. Walter Benjamin most famously talked about the aura of the object in the age of mechanical reproduction. (Which you probably guessed is behind Aura’s name.) Humans care about their own history and the objects that played a part. Authenticity of an object has a visceral significance. Even a perfect 3D copy is hard pressed to evoke the same response from a human being as the original.
Other people and their reactions. We are social animals. The same artwork seen through the crowd at an art opening evokes a different reaction than one contemplated slowly, in silence. What we overhear others say near artworks is endlessly fascinating and influences our own perceptions. And let’s not forget that seeing art in a crowd of friends or strangers is a lot of fun (if you get over any intimidation factor.) The hordes at big art fairs like Art Basel Miami, popular local events like Oakland Art Murmur, are all about the social experience of being part of an exciting crowd of art patrons seeing and being seen seeing. Purists who decry this aspect of the art world are ignoring thousands of years of history of the social function of art patronage, from ancient and classical times through the Renaissance, Enlightenment and the present day. Social media harnesses and extends this fact but is still different than real world social presence.
Light and space. This relates to scale and 3D vs 2D but is worth separate mention. Environmental lighting reveals details of artworks in ways that would read completely differently in a static image. Walking up to an artwork through a large space and coming to closer and closer detail is a different human experience than zooming on a screen. For the vast majority of artworks and exhibits that were conceived for physical rather than digital consumption by their creators or curators, this can be a huge part of the intended experience. Or, even if you don’t care much about artistic or curatorial intent, it’s a huge part of the shared human experience with all the other viewers of the same work or exhibit through time.
Environments, nearby artworks. Curation has become a bigger and bigger part of art experiences in the last century. Curators are attaining the status of artists, and many artists also act as curators. Most curated exhibits are about creating a story or experience through the juxtaposition and placement of different works. Many art lovers will find over the years that they will see the same works as part of different exhibits with different curatorial points of view and walk away with different impressions. Even casual art viewers tend to experience works in exhibits from a comparative point of view, deciding which work is their favorite in a given gallery and explaining to their companions why they like one work over another. Digital curation, with the help of new tools and services like Instagram, ArtStack and Curiator, is on the rise and is certainly valid, but again it’s a new medium and not a replacement for the old. Like films, catalogs and other forms of art presentation that have appeared over the last few centuries, digital will become yet one more platform amongst many.
OK, so by now you should be convinced that #artIRL is at least different than digital. Hopefully you also have a feeling it can be more fun and rewarding. Now let’s talk about why #artIRL doesn’t just speak for itself and needs technology and media for a full experience.
Why art needs context
The nature of real-world art experiences is ephemeral, even if the artwork endures for decades, centuries, or even millennia. The crowds change or dissipate. The exhibition closes or is rehung. The artwork moves, changes hands, is loaned out, or goes into storage. The artwork decays or is destroyed. Even you, the viewer, grow older and your perception is changed by your life experience. People have been carrying sketchbooks for centuries to try to capture the memory of their art experiences. Today they take home postcards and posters and take photos. These artifacts aren’t meant to reproduce the real world art experience, but to serve as a memory aid.
The artwork itself also doesn’t tell its whole story. That’s why we have wall labels in museums and galleries. It doesn’t say who made it, when it was made, what the artist’s thinking was, what the critical reception of the work was or what role it played in history. Yet it’s hard to remember the details we read on a wall label after the fact. We may scribble notes, or lug home exhibition catalogs. Most of the time we just accept our short memory. The inability to remember the names of what we saw makes most of us shy to discuss our art experiences with others after the fact, contributing to an overall lack of confidence around art.
Other aspects of an artwork may or may not be obvious: what it’s made of, how it was made, what color it is, how to describe its style, what the subject matter is. These aspects may even be misleading or obscure — such as Deborah Butterfield’s horses made of cast bronze but looking like wood, or a figure of a woman that turns out to be a specific historical figure such as in this painting of Mrs. Darrow (whom I presume, but don’t know, from the date and origin must have been the famous defense attorney Clarence Darrow’s wife — anyone who can comment please do!) These aspects may be on wall labels, but it’s hard to read longer text while wandering around an exhibit. Curators rightly lament ugly labels distracting from the presentation of art, and the visitors who spend their time in the presence of great artworks reading text and ignoring the artwork.
Wall labels are usually also pretty brief if present, omitting many potentially interesting details, and often making references that they don’t have the space to explain, such as to technical art terms, movements, historical figures and events. A longtime friend, accomplished and known musician and writer who is not educated in art recently honestly asked me “What’s gouache?” while at a museum reading a label. Good question.
So mobile apps are solving the problem, right?
A lot of people today have the idea that mobile apps may enhance real-world art experiences, but we think their vision is usually technology-cenric rather than art- or user-centric. We also think their assumptions about both technology and social behaviors are wrong. The hype is way ahead of reality here.
The majority of location-aware app projects are trying to replace the wall label by automatically calling up similar or extended content related to a specific artwork in the presence of that artwork. We think that is extremely limited and somewhat misguided. Even if it worked perfectly (which none yet appear to do — if you have actually used one that does, please write me), such an app would just replace the distraction of a wall label to the side of an artwork with a heads down experience of reading text on one’s device.
Truth is, most art apps aren’t yet location-aware, or, even if they have location capabilities in a floorplan map feature, the artwork listings aren’t tied to location.
Then, most apps that have been produced, both the location-aware ones and the catalog pre-visit ones, have a preloaded subset of known artworks. Limitations of app architecture and the logistics of digitizing content usually mean that only a fraction of the artworks in a given museum are in that museum’s app. The same holds true for the few aggregator apps like STQRY that have subsets of a select group of paying participating museums included.
So the user is forced to switch between wall labels and the app as they wander through exhibits (more cognitive distraction from the art), or just take a “highlights tour,” missing most of the museum’s bounty. And what about art in smaller galleries, art fairs, museums and artist studio exhibits lacking the capability or budget for bespoke apps?
Now let’s look at location-awareness. There is a lot of buzz around a variety of cutting edge technologies that promise to deliver content tailored to exactly the specific artwork a user is viewing. At Aura, our technical cofounder @deeje (Deeje Cooley) and his team have done extensive R&D around all of them and we are pretty confident that none work the way that anyone expects. The few real implementations involve tradeoffs that significantly impact the user experience and result in more heads-down, frustrating, distracting user experiences while in the presence of art. The many glowing news articles and scholarly papers about various startups and museum apps rarely talk about anything beyond a testing phase or hearsay.
This isn’t the article for a detailed technology analysis, but in general we are confident that, at best, a combination of GPS, indoor positioning, wifi and other triangulation can produce a geo-fenced shortlist of nearby works, leaving the user and the user interface to somehow select a specific work to bookmark or read about. (iBeacons deserve special note — their signals are way too strong to pinpoint a specific work in a crowded museum, and vast differences of scale of artworks from miniatures to multi room environments make “nearby” a very elastic concept.)
Sadly, the user interfaces have also been poor, the shortlists in densely hung venues are not short, and the actual time to get a location fix and generate a list of nearby works has been slow. The best location-aware apps from a user experience perspective still rely on QR codes or the user entering a numeric code from a wall label — much like the last generation of the 50-year-old audio tour devices.
Yet then we still have both labels and the app, with users engaging in a non-natural behavior to match what they are seeing IRL with what’s on their device.
OK, so then what should a location-aware mobile app do for #artIRL?
We think that the conception of the primary role of the mobile device needs to move away from pushing didactic content at you while in presence of the art like wall labels or an audio tour, to recording the art you encountered and where you encountered it as you freely wander through different art venues in the world. You then end up with your own personal catalog of all of your own art experiences, with the ability to recall what you saw, share your experiences with friends, and enjoy exploring more fun details and connections after the fact. Instead of information, whether on a wall label, through a headset, or on a screen, competing with the in presence experience of the art itself, it extends the ephemeral experience of art through time.
We think location capabilities should used first in service of knowing exactly where you were when you saw a given artwork. Location awareness joins timestamps, image capture/recognition and text recognition as tools to match different people’s records of the same artwork, so that knowledge supplied by one user — possibly the curator, possibly the artist, possibly an amateur expert — will be found by other users encountering and the same work. The knowledge linked to a given artwork is an accumulation of all voices and perspectives, not just the “official” view, with reputation, ranking and presentation making for a coherent experience. Every user’s experience records contribute passively to the global knowledge of what art has been where in the world; while users who choose to share reactions or add their knowledge contribute more actively.
The goal is to free the art curious person to wander the world’s museums, galleries, artist studios, back alleys, homes and public spaces discovering and fully experiencing as much art as they can in its myriad forms, with the confidence that what they see will be automatically retained, recognized and associated for them to recall, share and further explore throughout their lifetimes.
Such an app can also act as a guide to discover new art to see and follow. Yet unlike tour or catalog apps, it’s not working from a limited set of featured works, artists, galleries or museums. Nor is it restricted to a limited taxonomy of genres or styles of artworks that the user is asked to “like”. The accumulation of all of its users’ encounters with artworks form the database of artworks to recommend to other users. Users’ own past history of what they’ve remembered, how they’ve reacted to it, and the highly specific topics they’ve followed in exploring associations from what they’ve seen before drive what is recommended to them.
A little about Aura
Aura v1 is the first step toward such an app. It’s a personal art notebook.
It enables you to remember art you see anywhere you go. It starts out empty on purpose, like a physical notebook, waiting for you to fill it with your #artIRL experiences.
It uses your phone camera to capture images of the art and any wall labels. It lets you keep multiple images for a single artwork together. It transcribes label photos into text notes that you can easily re-read, tag, edit and search. If it recognizes someone’s name, typically an artist, it links your memory to a page on Aura.com with descriptions of that artist drawn from Wikipedia and links to the full Wikipedia article. It knows when and where you took each memory. It makes it easy to share your art memories with friends and followers on other social media like Twitter and Facebook and directly via email and SMS.
Most users get into a habit of quickly capturing a photo or two of each artwork and another of the label, just as they were doing with just their camera app before Aura. Then they put the device down and spend most of their time in the presence of the art looking at the art. Not the wall label or their screen. They are confident that Aura has captured the label for them, so they can learn more about the artist, title, medium, and any curatorial notes later. For users needing large fonts, or in crowded exhibits, Aura’s transcribed notes are a lot easier to read and consume. In fact, reviewing art memories in Aura after an exhibit visit, either alone or with friends, has become a fun and enlightening new activity. Often they realize connections between works within and across different exhibits and venues that weren’t obvious in the moment. Links to artists from Aura memories further their art knowledge. Sometimes those discoveries make them want to come back to the museum or gallery for another IRL look.
It seems simple, but it’s got a lot going on under the surface that will let Aura do more and more magical things over time. Early users tell us it’s surprisingly powerful and has already had a tremendous impact on their experiences of art. These users range from casual museumgoers trying to retain more of their art experiences and share them with friends, to art professionals keeping track of what they see for clients and looking at their clients’ own Aura memories and notes.
The memories that users are taking with Aura v1 are also the foundation of knowing what art was seen where when. Aura automatically creates a record of a “thing” when two or more user memories images, location and metadata match closely. When this happens, users making a new memory will see what others have already shared about that thing — which may be the official curator, if they have decided to use Aura as their platform for mobile enhancement.
You can see the things that Aura knows of nearest you from any browser at www.aura.com/nearby. Based on what is on the wall labels, and, eventually, capabilities for users to add their own reactions and observations, Aura will begin to know a lot about the characteristics of artworks. This will enable us to add discovery and recommendations. You’ll be able to come to a new city and find where artworks are by your favorite artists, dealing with subjects that we (and you) learned that you are interested in, or related to topics you follow.
There’s a lot more coming, but all of it will start with and lead to more and more rewarding #artIRL experiences.
Note: If you agree with this line of thinking, and yet you post your art experiences to social media, I’d encourage you to start using the established tags #slowart and the tag I’m suggesting, #artIRL, so we of like mind can find eachother.
