Augmented reality. How to design active interaction with art
Worldwide, reality-augmenting technologies are increasingly being used by museums and cultural institutions that want to attract new audiences and play with the potential of new media. These technologies are still in their infancy, both in terms of hardware capabilities and the potential for using the medium itself. Most of the projects carried out so far base their impact on the effect of “attractions”, stunning with new technology, as George Melies’ did with his early cinema experiments, using film tricks such as the disappearance of actors, image blending or slow-motion effects. I wrote on some examples of both Virtual Reality (VR) and Augmented Reality (AR) applications used in cultural institutions, adding a handful of critical forecasts in Augmented and Virtual Realities. Post-reality forecasts.
The question I was asking myself was how we can use the new media to enrich our visitors’ experience so that the fundamental value of interacting with work of art does not disappear from our eyes?
Remembering that technology is only a problem-solving tool and not an aim in itself, I started by properly recognizing the pains of National Museum in Warsaw visitors.
One of the problems we diagnosed during the research with users was the feeling of lack of competence in the perception of art. Museums often expect an already educated viewer, often focusing on history enthusiasts or devoted admirers of visual arts.
Instead of bringing together, art often separates:
I know that the museum is for CULTURAL people who are familiar with art and knowledgeable, but….but there could be a few sentences, some information, about what the Middle Age was and why those artworks were created.
People visiting museums are very much eager to learn, they declare their willingness to learn and acquire new knowledge. Just by closely observing visitors, we will quickly realize that they positively evaluate the experience when they manage to recall and combine given content with their previous knowledge. If, while attending the exhibition, we come across only signatures informing about the author of the work, the dates of its creation and technique? We leave the museum with a sense of insufficiency and lack of appropriate knowledge.
We confronted this pain point with the participants of Hack Art — the hackathon organized by the Museum, which took place at the end of May 2018 in the National Museum in Warsaw. Using cutting edge technology, an interdisciplinary team of five, addressed the problem posed by the Museum: how can we give our viewers access to expert, yet easily digestible knowledge about pieces of art, which would bring them closer to the context of their creation, explain the meanings and offer interesting stories? Is it possible to implement such an idea without interfering directly in the way exhibitions are presented? In response, they created a working prototype of Mind the Art — application based on AR (Augmented Reality), which gave them the third prize.
Let’s imagine that you enter the Gallery of 19th Century Art in the National Museum in Warsaw — where over 300 Polish and European masterpieces are gathered, with such highlights as Jan Matejko’s “Battle of Grunwald”, works by Gierymski, Boznańska or Malczewski. You are passing a beautiful portrait of a woman — it’s Alina Bondy Glassowa painted by Konrad Krzyżanowski. Who was she? Why was she portrayed by the author? In what style is the painting done?
What if you could take your own smartphone out of the pocket, launch an app, a simple interface would invite you to find more about the details of the painting with a few taps. Curated information would encourage to interact more closely with the work of art, paying attention to fascinating details. We don’t have to take notes, no need to try to take decent photos of the painting without a flashlight or look for it among the thousands of poor quality reproductions on the Internet. You simply add it to your private collection of favorite works and download a great quality digital version and make it your new wallpaper or screensaver. Possible? Absolutely!
The application uses AR Core technology, which seamlessly blends the digital information points and physical painting from the NMW collection in real time on. How does it work? Camera in our smartphone works like a scanner, while photographing an image, the application recognizes it without the use of any actual QR codes and displays markers on the painting, which hides extra pieces of content prepared for visitors. Using our own device significantly lowers our entrance threshold: we do not need to rent a classic audioguide and learn how to use it, and most importantly, we choose which content is interesting for us.
I asked a group of international students of the 1st year of New Media Art at the Polish-Japanese University of Information Technology for help in creating the interface design and think of additional functionalities of the application.
During the entire semester, together with an experienced UX and AR instructor Marcin Wichrowski, we went through all stages of user-oriented design: from research, identification of target groups, many iterations of the design, prototyping and finally user tests in the space of the Museum.
Our goal was to design such an interaction with the application that would 1) maximize the use of time and attention of the user; 2) satisfy the curiosity of the visitor; 3) reduce the distance between the visitor and artworks, 4) work an interactive alternative to audio guides. The priority was to put the artwork, not the technology at the center of the experience.
First, students chose the target groups to which the application will be addressed:
- Visitors 55+
The specificity of these audiences is that they usually have more time and willingness to visit cultural institutions. They are highly motivated to learn: many of them attend third-age universities or educational events.However, they will not always be tech-savvy, so they need a clear and simple interface.
- Teenagers (10–18 years old)
This target group usually visits museums as part of school activities, family trips or events. They do not want to be educated in the traditional way but are interested in interactive technologies. They usually own a smartphone, use many applications, love any form of gamification i.e. challenges, badges and scoring systems and opportunities to compare themselves with others. - Parents
Although they may not have so much time to use the app themselves visiting the museum with a family, they are interested in valuable education tools so will be very interested to show the application to their kids if they can see that it is useful.
How to convince visitors to install the app?
- After connecting to open Wi-Fi in the Museum user will receive a push notification encouraging to install the application, which will not take more than a dozen or so seconds to download.
- Installing the app will result in extra benefits for the user. For example, a user can purchase cheaper tickets via the application.
- The youngest users will be able to collect points while using the application, which they will be able to exchange for prizes: e.g. stickers with the greatest masterpieces of the Museum.
Additional benefits:
The application will provide access to a clear map of the Museum.
In the next versions, it could be integrated with automatic tracking of location, which will allow for easy navigation inside the Museum, finding specific works of art and avoiding feeling lost.
During the semester, two prototypes of the application were created. Both teams opted for a minimalistic user interface and simplicity of use.
After opening and logging in to the application we get acquainted with the instructions for scanning images and unlocking the AR function:
After scanning the image, Augmented Reality mode is starting, placing virtual markers on the painting in the Museum, and can be unlocked by simple taping.
After creating an account, we can add the viewed painting to our private collection, share it through social media, and finally, we get a link to Digital MNW — a repository of digitized images which one can download in good quality. We can also keep track of our progress in learning about the collection and earn points for passing the next stages.
In addition, thanks to the Museum’s map, we can easily see the building’s space and get to other galleries, find elevators for the disabled or restrooms.
Users who have tested the application with us appreciate it first of all:
- concept
- easy to use
- minimalism
- the possibility to return to your favorite artworks later
- no overload with technology
While we are at the development stage, our hope is that its users will not only be able to expand their knowledge with a device they know and gradually develop a habit of looking at art with curiosity but above all, feel more comfortable in the Museum. From the institution point of view, it opens the possibility to invite visitors to actively experience art and make people stop being afraid to visit the Museum.
We will soon publish an article on AR Core technology itself and app technical aspects.
