Towards a better storytelling of cultural heritage through XR
Our partnership as an XR-dedicated company with French Ministry of Culture and Communication
Every year since 2012, the French Ministry for Culture and Communication launches a call for projects called “Innovative Digital Services”. It represents a plan for funding solutions wich facilitate access to culture, heritage, creation and cultural democratization.
Minsar (ex Minsight) had the honor of being chosen for the 2016 edition, by building a partnership with three prestigious French institutions: the National Monuments Centre, the Museum of Decorative Arts of Paris, and the National Museum of Archeology.
That is how, for two years, we have been thinking together about the future of digital engagement with Cultural Heritage. In this article, I will present a sum up of our past, present and future reflexions. To read the complete report written for the French Ministry of Culture, go to this link.
2016–2018 : Clearing the field
Pierrefonds Castle
Our very first public experimentation took place on November the 18th and 19th 2017, in the Salle des Preuses of Pierrefonds Castle. The experience was to be created and lived through a Microsoft Hololens device.The goal of that experimentation was to provide visitors with some keys to better understand that prestigious place. For instance, we wanted to offer holographic restitutions of Napoleon the Third’s collection of arms and armors. Indeed, that collection was removed from the castle at the end of the 19th century, never to be returned.
We ran into a few problems. First, we could not, at the time, obtain any authorization from the Invalides Museum, which keeps the armor collection, to scan one of them. As a result, we decided tu use high-resolution photographs of the armors, and we installed those photographs on the wall. Even though the 3D effect was absent, we were pleasantly surprised by the effect which remained convincing. By looking at those armors suddenly springing from the wall, we could experience that feeling of magic associated with XR technologies.
The second major problem lied in the fact that in the middle of the Salle des Preuses stands a long, rectangular display case, which used to host a horse armor. As a matter of fact, we quickly realized that when the infrared rays of the Hololens went through that display case, its spatial mapping was broken and twisted. That is why we decided, for this early experimentation, to limit our experience to the first half of the room.
On the D-Day, the experience was created only a few moments before the arrival of the public. In that regard, we reached one of Minsar’s goals, which was to give the opportunity to create a 3D experience directly through the headset, quickly and easily. As for the public, they were delighted, and the experience was a great success with about 97% of satisfaction.
Museum of Decorative Arts
With the Museum of Decorative Arts, our first experimentation took place on Paris Museum Night in May 2016. The goal of the project was to present a specific work of art, Rhinocrétaire II by François-Xavier Lalanne, through different virtual spaces, and give the opportunity to the visitor to interact with it. To do so, the work of art was reproduced in 3D thanks to photogrammetry.
The experimentation first was proposed in Virtual Reality, then in Mixed Reality. Technically, it represented the first cultural experience on HoloLens in the world. The device was settled in front of the work of art, and visitors were invited to put on the HMD. This way, they could wander through the new environment in which the Rhinocretaire II was virtually installed. They could read information about the artist, as well as interact with the work of art itself. Indeed, the Rhinocrétaire II is actually a writing desk, which means it can be opened. However, its opened state is never showed to the public.
There again, the public was delighted. Besides, they could express their creativity and impressions on a guest book after the experience.
With the Museum of Decorative Arts, we were also very interested in studying the process of creation of a scenography with Minsar by the museum staff, rather than its “mere” reception by visitors. To that end, we organized a conference about the possibilities provided by Minsar, notably for guide-lecturers, on June the 18th 2018. For instance, we showed them the perspectives of using our software to support their own storytelling: handling the lights in real time, taking the visitors into a 360 video, or restituting the context of the object the guide-lecturer is presenting.
During the conference, we had the confirmation that a certain amount of people was still wary regarding digital tools. A great part of the conference was thus dedicated to reassurance and discussion, notably to show that the digital tool did not aim at replacing physical cultural heritage, and that it was simply another way to engage with it.
National Museum of Archeology
With the National Museum of Archeology, we started by organizing a workshop in the royal chapel of Saint-Germain en Laye, on June the 12th 2017. There, we made some experiments about the possibilities of scenarization within the chapel, through Microsoft HoloLens. However, the important light variations induced by the tall gothic windows created the same problem we had at Pierrefonds Castle: it broke the spatial mapping. That is why we decided, with the National Museum of Archeology, to start our experiments in Virtual Reality. As a result, out project in the Sainte-Chapelle of Saint-Germain en Laye will take place in 2019.
End of 2018 and towards 2019: the next major features in Minsar
Cross-platform
Minsar’s main horizon is to democritize the creation of XR experiences. To that end, the application cannot afford to be limited to only one device. Even though we believe in the necessity to create a 3D experirence in a 3D, immersive interface, it seemed obvious that Minsar also needed to be supported on more affordable devices such as smartphones. As a result, it is now possible not only to create an experience on iOs, but also to publish it through Minsar Distribute, and to share the link to the world. Anyone having an AR-Ready iOs or Android can now open the experience on their device. It is a major progress in Minsar’s development: the process of creation with our software is now complete. Moreover, in 2019, we aim at supporting Virtual Reality as well as the Magic Leap One. Indeed, that newcomer in the world of Mixed Reality seduced us with its resolutely creative and imaginative spirit.
360°
We believe that the possibility to import 360° videos or photographs is another very important feature. Indeed, these elements represent the opportunity of offering a compromise between Augmented and Virtual reality. In that regard, it is currently one of the only ways to actually combine the different levels of reality. Howver, importing 360 elements in real-time represents a great technical challenge, especially for Hololens. That is why it will be one of our main lines of research for 2019.
Interaction
At Pierrefonds Castle, we clearly witnessed the desire of the public to interact with the virtual objects. By the time, the only interaction we could provide was the triggering of some audio commentaries when the visitor walked over a virtual sound symbol. Of course, one of our major concerns for 2019 will be to develop interaction through the use of triggers, but also the use of built-in animations as well as imported from any 3D animation software (Blender or Maya for instance). Besides, we would like to open our experiences to the possibility of being reinterpreted, rearranged by the visitors themselves.
Collaboration
Another very important preoccupation for 2019 will be the development of collaboration both in the creation of an experience and its visualization. Indeed, if we truly want to translate the process of creation of an exhibition or a cultural event from a physical to a digital dimension, that concept is crucial. As a matter of fact, building an exhibition gathers many different persons : scenographists, curators, artists… With the collaboration feature, the scenographist will be able to collaborate in realtime, and on the same experience, with the curator or with the artist, even if they are somewhere else physically. That example, though simple, opens huge perspectives of creation.
Creative features
Speaking of perspectives of creation, we would like to greatly enrich the features of creation. Several lines of research are planned in that regard: possibility to handle and change the lighting and ambiance of the experience in realtime, better render of shaders, procedural generation features as well as visual effects such as fire, snow, rain, dust etc.
2019: New horizons
We are bursting with ideas and perspectives for 2019, and cannot wait to get to work! On the menu: we will take you to the heart of the Marais district, to unveil the mysteries of the Hôtel de Sully, and home to the National Centre Museum. We will also improve and enrich our former experience in Pierrefonds Castle, with more scenaristic possibilities. Then we will invite you to dive into the extraordinarily rich history of the National Museum of Archeology. From Henri the 2nd to the royal chapel built by Saint Louis himself, we will also take you for a ride in less known rooms of the majestic museum, such as the Salle Piette. At last, with the Museum of Decorative Arts, we will help ENSAAMA students thinking and elaborating a new museography for the Marsan Pavillon, as well as try and invent new ways of representing digitized 2D works of art.
