Pushing the Digital Boundaries
Behind the Scenes of Revealing Krishna
By Digital Project Managers Haley Kedziora and Anna Faxon
Amid the curators, conservators, and educators, an art museum is not the first place most people expect to find digital project managers. But at an institution known for pushing technological boundaries, there’s room for not one but two such staff members, sharing the responsibility of connecting visitors to the CMA’s collection with meaningful digital experiences.
We deal with both the big picture and the minutiae in projects throughout the museum, our roles bridging all the components. With needs shifting depending on the unique goals of each special exhibition, we report directly to the CDIO and oversee projects across the Digital Innovation and Technology Services department— which includes, video producers, developers, and AV integrators — and across the museum. For Revealing Krishna: Journey to Cambodia’s Sacred Mountain, not only did we work with talented people across departments, we also brought in talent from outside, including experience design firm Dome and mixed-reality partners Interactive Commons at CWRU.
While each step of a project is focused on a result, we work in the medium of process. Behind the refined immersive experiences in the exhibition is a story of concepting, discussing, and testing.
The making of each of the four digital experiences in the exhibition has been a multiyear endeavor. In a series of vignettes we will highlight some of our favorite moments that are otherwise tucked neatly behind the scenes.
Photogrammetry
The exhibition’s centerpiece is the CMA’s newly restored masterwork, Krishna Lifting Mount Govardhan, one of eight monumental sculptures originally from a mountain in Cambodia called Phnom Da.
Four of the eight are physically on view in Revealing Krishna —including a second Krishna sculpture now in the National Museum of Cambodia (NMC) in Phnom Penh — but all of them are featured in the exhibition’s digital elements. In the “Gods of Phnom Da” interactive, life-size projections of the eight Hindu god sculptures rotate and zoom to reveal iconographic details. While in “the Story of the Cleveland Krishna” HoloLens experience, stone fragments rearrange in real time to show the conservation history of the Cleveland and Phnom Penh Krishnas.
The eight sculptures are kept in museums across three continents. Although all eight sculptures were unable to travel to Cleveland, due fragility or size, it was crucial to have visually consistent imagery for the digital concepts. So our team traveled to them instead.
In 2019 the Digital/Technology team at the CMA worked with The Farm 51, a company based in Poland, to capture 3-D models of all eight gods and the fragments of the two Krishna sculptures using photogrammetry and LiDAR (3-D laser scanning).
Photogrammetry involves taking hundreds of photographs of a single object from every angle. (Learn more in our blog on photogrammetry). A precise lighting and camera setup is needed to accurately represent the surface of these 1,500 year old sculptures. In fall 2019, the team traveled to the National Museum of Cambodia (NMC) in Phnom Penh to begin creating models of the six sculptures housed there, working into the night with artificial lights to capture every physical detail of the artworks.
Then, in January 2020, members of the Farm 51 traveled to Cleveland to produce 3-D models of the Cleveland Krishna and the various fragments kept in the CMA’s conservation lab.
One month later, the photographers went to Paris to capture the images for the final model needed for the exhibition, the Harihara in the collection of the Museé Guimet. With this final model, the set was complete. At this point, we started working on what would become the interactive animations for the “Gods of Phnom Da” in the third digital gallery of the exhibition.
With each of the sculptures fully documented from all angles, Dale Utt III (True Edge Archive) constructed high-resolution 3-D models, giving the animations a level of accuracy that only the many hundreds of photos and LiDAR scans could offer.
Prototypes on Prototypes
Bringing the digital experiences from concept to reality required many prototypes, particularly for the projection-based interactives.
In March 2020, we were in the museum wading through a projected Cambodian canal as we reviewed footage in a nearly full-scale prototype of “Journey to Phnom Da,” the exhibition’s first digital element. With this prototype, we worked with UX (user experience) designers, videographers, and sound designers to lock down the final video sequence and test our proposed projector set-up, which required precise blending of two overlapping projection fields on two parallel walls, plus a central wall to give the illusion of one seamless waterway.
After testing the “Journey” prototype and then packing it away for a year due to the onset of the COVID-19 pandemic, we began a new prototype in spring 2021. Initially, we were skeptical about doing another one, since setting up is no small task. It requires three 22-foot-long empty walls — a true rarity at the museum where most are covered in art. Thankfully, the timing worked out and we set up in the ArtLens Gallery while the space was closed to the public as a pandemic precaution. This time, we were able to further troubleshoot and refine the experience. We were glad we did. We discovered new wrinkles to iron out, saving us some headaches during the installation process.
We also created multiple prototypes for “Gods of Phnom Da,” to review the content, and the look of the screen and to test the functionality of the software forming the interactive experience.
We began pre-pandemic with two projection screens hung in a corner of the design team’s basement production studio. There, we reviewed content from our interpretation team, confirming that each angle of the model was true to the sculpture and that each caption accurately gave context for the gods’ iconography. Once we were able to return to the offices, we picked up where we left off. This time, the goal was to make sure that the software running the experience functioned as expected. The entire “Gods” experience is centered on two concepts: the unique iconography that appears after stepping into an “activation zone” and a full-body view of each god simultaneous.
While this prototype didn’t reflect the scale of the exhibition, we could test the experience itself and were able to verify that the software works. After ArtLens Gallery reopened to the public, we moved our prototyping back to the basement of the museum. With three full-sized screens to put the finishing touches on the content, we were able to finalize the content of the interactive.
Beyond the content and software, we made adjustments to the hardware and worked through new issues. For example, the projectors allowed too much light to extend past the sides of its corresponding screen, which created distracting bright spots. We needed a solution that aligned the projected area with the edge of the screen, but after much trial and error, we realized that the product we needed didn’t exist. Our team went to work making a rough prototype lens cover complete with movable shutters to allow for adjustments. The Audio Visual (AV) integrators then created elegant 3-D printed lens covers with a secure fit that could withstand the heat of the projectors, and adjust to the right size for the projection.
Finding the Voice of Krishna
The HoloLens experience has been a key part of Revealing Krishna since the early concepting phase. The CMA’s cross-departmental project team has been working with Interactive Commons and official technology partner, Microsoft, on early development and prototypes for this concept since 2019. The experience is a dynamic balancing act of timing and cooperation while a group of visitors wearing HoloLens headsets enter a 10-minute experience and move around six stations.
For a complex timed experience, we needed a narrative track to guide visitors, with a script to concisely tell the intercontinental story of Krishna, keep visitors moving through the space, and simultaneously spark joy and wonder.
The script began with curator Sonya Rhie Mace, and Jenn DePrizio, who leads the museum’s interpretation staff, alongside the rest of the HoloLens experience team. After multiple iterations, the complex story was there, but we needed to pivot from an institutional narrative to an immersive, cinematic telling. We brought in a film scriptwriter, Nick Fitzhugh, to rewrite the HoloLens experience script with a fresh voice. However, there was a conundrum: with a cross-continental story spanning seventh-century Cambodia, 20th-century Europe, and contemporary conservation labs, who was the right voice to tell this story?
It was during this writing and revision process that we thought: rather than a disembodied, documentary voice, what if the narrator was Krishna himself? This key change brought the experience to life in new ways, allowing the team to infuse the whimsy and playfulness of the child god into the narrative. The scriptwriter worked collaboratively with Mace and the project team to ensure that the script was accurate and respectful to the sculptures and their stories, while allowing the narrator’s personality to shine.
We looked for the voice of our Krishna, with everyone agreeing it would be Cambodian. There were many other factors to consider though, including making certain that the speaker’s age, clarity, pronunciation, and intonation, fit Krishna (an eight-year old at the time of the myth of Krishna Lifting Mount Govardhan). We needed someone with fluent English, Khmer, Sanskrit, and French pronunciation, who was able to embody the personality of Krishna: playful, cheeky, bemused, yet omnipotent.
From many taped auditions sent by Cambodian students, the team found Emi, a 13-year-old in Cambodia who hit every mark. We worked with a friend of the museum, KHIN Pothai, who had arranged the on-location video and photography shoots the previous year, to coordinate Emi’s audio recordings in Siem Reap.
The HoloLens experience was not the only digital element to include voice talent. For the immersive timeline in the final gallery, the museum was able to work with author Loung Ung and director, actor, and humanitarian Angelina Jolie, who generously lent their voices to bring the centuries-long story of the gods of Phnom Da to life.
On the Canals and in the Caves
As you enter Revealing Krishna, you are transported to seventh century Cambodia via the canals used for travel through the floodplains surrounding Phnom Da. The footage in this immersive hallway travels to Phnom Da from Angkor Borei, the city just to the north, and mirrors the journey pilgrims would have taken to reach the sacred mountain.
In fact, the same canals remain and are used today, but travel is much faster by motorboat.
To capture the footage for “Journey,” the creative team spent days zooming through the canals —capturing the voyage between Angkor Borei and Phnom Da multiple times and periodically slowing down to record the sounds of local animals and lapping water. The video footage was captured using a three camera rig which records from the front and sides of the boat.
In the finale of the HoloLens experience, visitors explore a 3-D rendering of the actual cave temple on Phnom Da where Krishna Lifting Mount Govardhan originally stood. Visitors see candlelight flickering on the dark, polished surface of the sculpture, thanks to an artist’s reconstruction. The cave temple itself — both interior and exterior — is represented in breathtaking detail, down to the stonework around the cave entrance. This is not just a reimagined cave, instead it’s an exact, full-scale model of the original cave temple.
To gather the necessary material for the model of the cave, the team traveled to Phnom Da to take photos and scans. After a short scramble up hill and through the stone doorway, the whole team became involved in documenting, meticulously measuring and recording details. The detail-oriented nature of this work paid off in the end, with our resulting model being so realistic that we had to go back and tidy up some of the virtual debris in the final version.
Meanwhile, back in Cleveland the museum partnered with Interactive Commons at CWRU to create the immersive mixed-reality portion of the exhibition. We will never forget our first time entering the 3D model of the cave at the Interactive Commons’s office: it was a rough-draft model, but stepping into Cambodia was like nothing we had ever experienced. That was back in August 2019, which gives a sense of how long this project has been in development. Since then, the team has spent countless hours refining the model and all the digital elements, from the technical (figuring out spatial anchoring and bug fixes) to visual refinements to the lighting of the sculpture and the cave walls.
Revealing Krishna is an exhibition truly greater than the sum of its many parts. Ushering each concept to completion has taken a great deal of time, talent, and care. We hope the efforts of the entire team’s labor, visible and invisible, show up in a seamless and immersive experience for visitors, intertwined with the awe of the sculptures on view. The detail-oriented, sometimes-tedious, sometimes-thrilling work in the years of editing, tweaking, and testing have resulted in moments of pure captivation.
Want to learn more about the making of the exhibition? Find a list of programs and resources below:
- Visions and Revisions: Behind the Scenes of Revealing Krishna
Sunday, December 12 at 2:00 p.m.
Hear how curators, conservators, engineers, physicists, materials scientists, and leaders in the field of technology and digital imaging worked together to understand and convey the form, history, and context of the museum’s monumental sandstone sculpture Krishna Lifting Mount Govardhan.
Some museumgoers may remember Krishna Lifting Mount Govardhan, on view for decades at the CMA, as standing securely on complete stone legs and feet, with his arm raised into thin air. Today, after seven-plus years of research, international collaboration, and conservation treatment, the sculpture looks very different.
- Learn more about the digital process on the CMA website
- Listen to Microsoft’s latest podcast on Revealing Krishna, discussing the collaboration between Sonya Rhie Mace and Jane Alexander as well as their amazing use of mixed-reality to bring the story to life.
Take a journey to Cambodia in the Cleveland Museum of Art’s exhibition Revealing Krishna: Journey to Cambodia’s Sacred Mountain for an unforgettable exploration of art and digital experiences. On view though January 30, 2022. Become a member and see it for FREE! #CMAKrishna
