The Making of “HoloScenes”
Envisioning the future of shared holograms
This post is meant to give you some insight into the process used to create the “HoloScenes” series on Holographic Visualization.
As part of a Visualized 2016 workshop on the role of visualization in society, one of the participants asserted that our community of data visualization professionals don’t often share how exactly they go about doing their work across domains with the myriad of skills required. I do think we share the what fairly often (examples, code snippets, etc), but perhaps not the how often enough. That comment stuck with me, so here’s my contribution to relaying my thought process and technique for creating part one of my new Holographic Visualization series “HoloScenes”.
Upon some reflection, I now realize that project percolated for almost a year, but went relatively quickly once I began to execute. Interesting how your subconscious directs your actions when it knows what you really want to do.
Motivation
Kinda wish there was some brilliantly conceived plan behind this series, but the truth is it just came to me in a moment of reflection. And, I just wanted to explore the holographic design space and push my own abilities to find the right kind of experiences that would really shine vs. those trying too hard to fit in. Getting that natural feel turned out to be difficult — don’t think all of the pieces feel natural enough. Time to iterate or move on.
Scouting Locations
I knew that I wanted to portray a breakthrough experience using holograms, so I started by scouting locations for the series to be “filmed on location”. It turned out the locations were all over the place — Seattle, Seoul (South Korea), NYC, Vancouver BC (Canada), Silicon Valley, and near Chicago. But, the fact that I captured those locations for my future work were somewhat random. I had no idea what I was looking for, but when I saw it — magic happened. The scene presented itself to me in a high degree of clarity. I visualized how I’d use a particular place as the setting for a conceptual piece. Wasn’t sure exactly how that would look of course, but I could feel it was at least a good start for what I was after.
There were many photos I took with my phone camera that didn’t work out — and others that are still on the digital shelf, waiting for that spark of inspiration to realize them. One production note on these — you’ll see below I tried to leave the original photo alone as much as possible for the pieces.
HiFi vs. Sketchy
The obvious thing for me to do was go with high fidelity comps since the backdrops were all photographs. Although, I did decide early to do something different with the people in the scene. I have always liked the look of architectural pieces with silhouettes of people as stand-ins for depicting the size of the structure. I borrowed that approach for the HoloScenes series, but with a twist — the characters in the scene are silhouettes to both (1) allow the viewer to put whoever they choose into the scene through their own imagination, and (2) use it as a visual device to clearly differentiate between real people present in the scene and holographic avatars that are in the space, engaging with the real people in some way. I’ll circle back to some of the issues with finding the right poses to place in the scene.
Here’s a peak into my process and challenges along the way…
“Guest Speaker” — behind the scenes
This piece focuses on the challenge of finding out what’s happening and how to catch it when you’re at an industry conference. Applies to many other scenarios as well, but this one is pretty common.
Motivation
This scene from inside a lobby just called out for people to be sitting talking about something. In this case, it seemed like a great example of when to use table top holograms to help navigate the hotel to find the room where the most interesting speaker would be.
Key aspects
Notice how the hologram was possibly placed on the lobby table top by the conference organizers so that any attendees walking by throughout the day could just sit for a second and checkout what was happening next.
You’ll also see how the text of the vertical display was designed to only face the front side of the virtual display (text appears backwards as you the observer are standing behind it). Other types of displays would flip the text to always be visible.
Using a 3D scanned model of the guest speaker (Graham Holo) rather than their photograph was a nice touch that I totally overlooked initially.
Also, when well done, holograms are fully integrated into their environments and are aware of their placement. The coffee cup sitting on the glass table top is not overdrawn but properly occluded by the rendering.
Setting
Filmed on location in Silicon Valley, USA.
“Next Quarter” — behind the scenes
Working with remote teammates, customers, and clients is pretty commonplace these days, so what could we do to improve that awkward dynamic that’s so frustrating? (yes, I know — fix the conferencing software).
Motivation
This scene from inside a corporate headquarters is found everywhere — a nice location is staked out for collaboration, but often goes unused for anything meaningful. It’s in an open space which is highly advantageous for working together, but seldom used in that way. What a waste of space and money.
Key aspects
The dynamic here is that one of the people in the meeting is remote (Kimiko, shown as the blue translucent silhouette) and has something important to present and discuss, so she has brought with her a display surface to make her own points rather than rely on one within the physical space. Conversely, the other attendee is quite real and present in this physical space (shown as the black silhouette). This duality is an important aspect of collaborating with holograms and holographic projection of avatars. It’s quite possible the inequality of remote meeting can’t be fully bridged, but this feels closer.
One aspect not covered explicitly in this exploration is the depiction of gaze by the remote person. That’s often helpful in knowing what they are referring to and even engaged at the moment. For another time.
Using the circular table in the space for displaying the products and their production problems is figured out by the meeting software. It just looked for a conference table or other flat table surface to project onto. The size of the objects is fit to the available space and is intentionally respectful of the attendees physical size.
Putting the current focus of the meeting (problems with production of particular parts) directly onto the hologram is a fundamental design principle for holographic or volumetric interaction design — use callouts and associated panels to label if they won’t fit directly on the object, but don’t have the detail too far away, like on an adjacent panel (a mistake I made when first rendering this scene).
Setting
Filmed on location above Times Square, New York City.
“In the Data” — behind the scenes
The focus of my current work is Holographic Visualization of data and information. This is a whole new frontier that needs lots of experimentation and envisioneering to see what’s possible and would actually be helpful to augment the tools and techniques we already use.
The key for me was to try putting people inside the data itself, within the context of their work.
Motivation
I had privilege to visit The National Center for Supercomputing Applications (NCSA) earlier this year — a real thrill for me being that the Blue Waters facility is filled with Cray machines and is the largest unclassified supercomputer in the country. The scientific visualization and simulation they can do here is unbelievable.
Seeing this scene before me, I immediately imagined being able to see inside the processes and jobs running on the myriad of machines to help the staff monitor what’s happening when things go right (or wrong in this case).
Key aspects
First, notice that the holographic data visualization is room scale (and by the way, the actual room this was taken in is about the size of a football field). Utilizing room scale holograms for exploration is incredibly inspiring. I have experimented a bit and am constantly blown away by the visceral nature of being in it vs. observing it on a screen. Room scale data is the future. Trust me on this one.
Being in the data makes selection and exploration essential. You will want to isolate particular aspects and expand them to see what’s important about it. In this example, the IT Pro is examining a set of processes that appear to have errors. The next logical part of this is to take action on it — don’t forget to design that next step in, or you just get frustrated observers instead of resolution.
Note that both people in the scene are quite real (denoted by black silhouettes) and have chosen to walk into the center of the data representation itself. This is quite common in holographic visualization — moving throughout the space, ducking down to get better view, circling around a particular point in the room to see an object from different angles.
This is a geeky thing, but these are real Cray supercomputers in an awesome place for anyone involved in computers or software to experience. The scenario here is quite possible for the staff of Blue Waters.
Setting
Filmed on location at the University of Illinois, Urbana-Champaign.
“Break Time” — behind the scenes
What would it be like to see what your co-workers were doing on their breaks if you could see into their mobiles? In this scenario, you get to see how people will not only interact with all types of media for their own enjoyment, but involve other people in catching up on what’s hot, gossip, or even making decisions on pressing issues around the house.
Motivation
In many ways, this was the most important of the explorations in this series — it tries to depict both personal and shared experiences in a public space. But, the difference here is that we see and possible hear things that are typically private or hushed in these kinds of places.
There’s alot to portray here, and thus the image is busier than I’d like.
Key aspects
Making the invisible or private into something shared and public is an interesting space to explore. It happens when we’re in public spaces and happen to catch a glance of someone’s mobile screen or overhear their inappropriate phone conversation. But, what if we started to not care so much if other people saw the media we were consuming or even creating for that matter. You might get a busy space as shown in this exploration.
Really loved showing how two friends would want to explore something together in full fidelity and size rather than squinting at a zoomed in version on their own mobiles.
Another really interesting interaction to explore is how we talk about things that are important to our friends and family. These people are trying to decide what to cook for dinner, with one of them being remote (depicted as the blue translucent avatar). Being able to gesture and mark up content like a photo remotely is possible today, but perhaps more engaging and personal when done at full body size as if they were actually present in the room.
Setting
Filmed on location in Seoul, South Korea.
“Gallery” — behind the scenes
Being able to experience great artwork, sculpture, architecture, or artifacts in full scale as if you were next to them is a fantastic use of virtual reality. We also have an opportunity to augment existing spaces with these same things and be able to explore them within the real world.
Motivation
A personal passion of mine is trying to find the best way to showcase the incredibly intricate artwork of my late sister Sharon Pell-Lie. Her artwork really calls out to be seen in person because of the incredible detail found in the mixed media pieces. What if you could experience her work full size even though the originals have been sold off or housed in private collections?
Key aspects
Using the existing space is key to this exploration. It just so happened that the alcove cutouts in this building are perfectly shaped and lit for paintings. There’s a tremendous opportunity to find existing spaces and architectural features to present creative artifacts in.
A key element of some pieces of art is the detail. In this example, sweeping out a selection rectangle on the painting using an air gesture results in a detail panel appearing that can be resized and positioned as the person likes. Opens up all kinds of possibilities for deeper exploration and understanding.
Another great use of the holographic medium is the ability for it to help us meet the artist, writer, or creator of the piece. Using a 3D scan or a reconstructed 3D model from photographs helps the viewer to make a more personal connection with the creator and their backstory.
When figuring out how to showcase artwork in a limited space the notion of navigating a large collection must be addressed. One simple solution is the tried and true method of putting navigation flippers into the scene to allow quick previous / next movement. Ugly, but effective. I know.
Setting
Filmed on location in Vancouver, B.C. Canada.
“Live from Somewhere” — behind the scenes
The opportunity to transform public spaces within hotels is incredibly exciting to both adventure seeking guests and management looking to provide more reasons to hang around the hotel bar. These spaces are often quite beautifully designed and decorated, but frequently empty or lonely feeling. We can fix that.
Motivation
I’m a huge fan of live music. Always have been. But recently it’s harder to get to see enough shows live due to lack of time, cost, and travel required. Wouldn’t it be great if we could be at live performances more often — but even better, what if the performers took advantage of this new holographic visualization technology to augment their performances?
Key aspects
In this example, an electronic music DJ is “piped in” live to the hotel lobby, complete with dramatic surround sound and spirited holographic projections that dance to the music in real time to entertain the guests. You could imagine even joining in and dancing next to the electronic fire with your loved ones before heading out for the evening.
Using the existing space for satellite pieces of art or imagery is done automatically by the holographic concert system, providing more than a single screen or location within the room to look at.
Setting
Filmed on location near Seattle, USA.
“Ready Player One” — behind the scenes
Enjoying a good book is still one of the most relaxing pursuits I can think of. Our imaginations help us bring the story to life as we turn the pages. It’s also fun to see when a motion picture is released of the book if your own image of the characters and scenes is similar to the director’s. What if reading a book was more like a adding scenes from a movie to the book as you read?
Motivation
I read the incredibly entertaining book Ready Player One by Ernest Cline out on my deck by the fire and could not have enjoyed it any more — except that I kept wanting to see what some of the characters in the book actually looked like without having to jump out to a web browser which would ruin my book bliss.
Key aspects
The ability to use the reader’s gaze on the page to trigger animations, sound bites, or additional video is something should be explored more — I was more interested in how it would either enhance or detract from the experience of enjoying a good book.
Using real elements within the space that you are reading the book is another area that should be utilized when designing these types of experiences. Notice how Ultraman’s ray bounces off my iPad.
Setting
Filmed on location in my backyard in Woodinville, Washington USA.
More work to come, but hope that gives you a some things to explore yourself and a glimpse into my process for creating these design explorations.
ABOUT THE ARTIST
Mike Pell is leading Design for The Microsoft Garage, an innovation accelerator turning employee’s wild ideas into reality daily, worldwide. Bold, insightful and uncompromising, Mike is recognized as a thought leader in the field of Holographic Visualization and Smart Information.
Related posts
“Datascrapers” a data art series by M. Pell
Find more like this at
http://www.futuristic.com
All real products and services depicted in this artwork is copyright of their respective owners. All rights reserved.
All artwork Copyright © 2016 Mike Pell, All Rights Reserved.
