Wisdom From The Women Leading The VR, AR & Mixed Reality Industries, With Naomi Augustine of Mixed Realities

An Interview With Fotis Georgiadis

Focusing on solving problems — A tried and true lesson I stuck with that always gave me remarkable results. Looking for a problem to solve provides you with a clear goal, or north star, and helps you stay focused. What I also learned is that people love solving problems. I found it easy to rally a team around it as well as get buy-in from other leaders. This was always my starting point when I created a new project, pitched an idea or design, or built a new team.

The Virtual Reality, Augmented Reality & Mixed Reality Industries are so exciting. What is coming around the corner? How will these improve our lives? What are the concerns we should keep an eye out for? Aside from entertainment, how can VR or AR help work or other parts of life? To address this, as a part of our interview series called “Women Leading The VR, AR & Mixed Reality Industries”, we had the pleasure of interviewing Naomi Augustine.

Naomi Augustine grew up in Japan and has a bachelor’s in physics from the University of California, Berkeley. Her career spans across aerospace, gaming, and augmented reality. She is currently the co-founder of QXR Studios, specializing in augmented and extended reality (AR/XR) storytelling.

​Her alter ego is a science fiction writer. Her first book, Mixed Realities: 7 Stories That Will Make You Question the Universe is a collection of stories that poke at the squishy parts of our universe, human understanding, and our relationship with technology.

Thank you so much for doing this with us! Before we dig in, our readers would like to get to know you a bit. Can you tell us a bit about your backstory and how you grew up?

I grew up in Japan and Hawaii and learned English as a second language. My single parent mother raised me, and it was just the 2 of us island hopping since I was 5 years old. We were poor and moved often, but we were a happy household. My fondest memories were of living on the Big Island where I spent much of my time after school swimming in the ocean. Despite having little, my mother supported my education and numerous hobbies the best she could — such as science, art, and music. I gravitated towards these hobbies because we did not have a computer, so I spent most of my childhood drawing, writing, reading, and playing the clarinet.

Is there a particular book, film, or podcast that made a significant impact on you? Can you share a story or explain why it resonated with you so much?

When I was around 13 years old, I read Stephen Hawkings “A Brief History of Time” and decided I wanted to be a physicist. Up to that point, I was primarily interested in astronomy, but this book introduced me to black holes, relativity, the big bang, and a pile of questions surrounding the mysteries of the universe. There was so much we didn’t understand, and I wanted to find the answers. I went into a book reading frenzy after that, diving into similar books such as Kip Thorne’s “Black Holes and Time Warps” and George Smoot’s “Wrinkles in Time”. I spent my high school afternoons in the library, on the floor of the physics section reading as much as I could understand. I accidentally skipped 2 grade levels in math and science because of it and eventually got accepted to UC Berkeley with a scholarship. Because of that book, my life changed forever.

Is there a particular story that inspired you to pursue a career in the X Reality industry? We’d love to hear it.

The story is a little bittersweet, but I wouldn’t be here if it weren’t for these series of events. People often wonder why I ended up in the XR industry and not in physics. Life doesn’t always go as planned and I’m still working on celebrating my accomplishments instead of regretting the paths not taken. After graduating from college, I did get to work in physics for a few years in aerospace. I lost that job in 2009 during the recession and could not re-enter. I ended up in the gaming industry for a while, doing test engineering and eventually managing engineering teams. 5 years later, I woke up one morning feeling a deep sadness about my lost physics career. Later that day at my desk, I came across a video of Microsoft demo-ing their latest breakthrough augmented reality technology (Hololens). There was a hologram of the game Minecraft on a real table, and I started googling more about AR and who was working on it. I had unconsciously decided that day to go into AR to steer my career towards something more physics related. I loved games but wanted to do something more cutting edge. AR felt like the perfect cross between physics and game development — optics and 3D content creation.

During my google search, I stumbled upon Magic Leap and their job listings. They were looking for an engineering manager, so I submitted my resume through their website. They called me the next day and a few months later, I moved to Florida to begin my new career in AR. That was 7 years ago.

Can you share the most interesting story that has happened to you since you began this fascinating career?

It’s hard to choose 1 interesting story because every day was interesting working in AR. I might have to go with the making of the “whale in gym” project because of its colorful history and the sheer size of the content. If you’re familiar with Magic Leap, then you have most definitely heard about the “splashing whale” marketing video that circulated around the internet in 2016. (https://www.youtube.com/watch?v=GbpqwUUfMAQ)

The video was controversial because it wasn’t clear if this was a real demo of technology or not. It was just a marketing video since the company was deep in the trenches of hardware and software development at the time it was released. When the technology was operational, we decided to create the actual whale application. I was asked to take point on it as an R&D lead and producer for the experience. We worked with the talented XR and visual effects studio, Magnopus (https://www.magnopus.com/), to build an exceptionally large and realistic model of a humpback whale splashing into the ground. But that was not enough, we had to make it splash in an actual gym to recreate the original video. Our team partnered with the University of Miami and occupied their gymnasium for a couple of weeks. To really beef up the experience, we added heavy duty fans and a subwoofer system to literally blow away the faculty and students who came to see it. The whole thing turned into a hybrid movie set and Disney Park experience type thing — which was extremely exciting.

I really enjoyed running around performing tests, like measuring the distance between where the users will be standing and where the whale was going to splash and working with my team to calculate the timing of the fans and splashing sound effects. The whole thing was triggered through the network, so we could synchronize the whale splash to multiple users at the same time. I got a glimpse of the future of an AR show control system, where any content could be projected to a stadium full of people for the Superbowl, musical performance, or a city-wide AR alert system.

Whale-sized AR whale

Photo credit: Brett Shipes & Naomi Augustine-Yee 2019

(shot through Magic Leap headset, location University of Miami)

Photo credit: Naomi Augustine-Yee 2019

(location University of Miami)

Can you share a story about the funniest mistake you made when you were first starting? Can you tell us what lesson you learned from that?

Sorry I am thinking and thinking but cannot find an appropriate story!

None of us can achieve success without some help along the way. Is there a particular person who you are grateful towards who helped get you to where you are? Can you share a story about that?

Many people helped me on my journey, and I wish I could talk about all of them. Two recent moments particularly stand out and I’m grateful to have worked with and met these wonderful people. One of the lessons I learned is the importance of finding a mentor and being kind and respectable to each person you meet. Each person, regardless of their seniority or level of experience, has gone through their trials and has their own struggles. I’ve learned to simultaneously ask for help and offer help, in order to form great professional relationships.

I’m grateful to John Gaeta, an executive at Magic Leap and award-winning film VFX inventor, who gave me a seat at the table on multiple high-profile projects. He ensured I was invited to all the executive meetings, introduced me to key people, and publicly gave me credit for my work. He encouraged me to take on speaking engagements, gave me interesting strategic and technical problems to solve and taught me the value of using the “3 act structure” for crafting proposals, presentations, and pitches. He elevated my career from a technical producer to a lead of R&D, then to becoming one of the leading industry voices for city-scaled AR applications.

When the Magic Leap adventure ended during the pandemic, I felt a profound sense of professional and personal loss. I was struggling with my sense of self-worth and was prepared to throw in the towel. However, 3 wonderful former colleagues reached out to me and asked me about co-founding a new studio. I got a text from Prudence Fenton, an award-winning film producer that read, “Have you ever considered art directing?” “Never,” I said. I didn’t think I had the chops for something that artistic. I was only good at making charts and decks. “You should, because you’re a natural,” she replied. She and 2 others, Graeme Devine and Andy Lanning, also both industry veterans, urged me to go for it and here I am doing creative work for the first time in my career as a founder.

The 4 of us founded QXR Studios in 2020 — https://www.qxrstudios.com/

I don’t know what I did to deserve such kindness. It has something to do with showing kindness myself and refusing to quit when things get tough. I hope to help others in the same manner as they helped me.

Are you working on any exciting new projects now? How do you think that will help people?

My first science fiction book just came out on February 8th. It’s titled “Mixed Realities” and it is a collection of 7 loosely related novellas about the nature of the universe. If you count early drafts and notes, I’ve technically been working on these stories for 15 years, though most of the real writing happened in the last 4 years. But it’s not an ordinary book. I wanted to try something a little experimental. This is a book that contains illustrated QR codes that the reader can scan with their phone and hear a musical track during key moments of the story. I wanted to capture the feeling of watching a film or TV series in a book format, to give the reader that extra dimension of immersion.

The book’s audience is college students and young professionals, particularly those interested in science and technology, particularly women. Many of my lead characters are female, who sit in the driver’s seat of the plot and overcome challenges using their smarts and creativity. These are the kinds of stories I loved reading growing up, so I hope to pass on the kind of inspiration I experienced when I was younger. The book’s themes are scientific, technological, and philosophical and focus on optimistic stories about friendships, friendships between humans and AI, family, and comradery.

Ok super. Thank you for all that. Let’s now shift to the main focus of our interview. The VR, AR and MR industries seem so exciting right now. What are the 3 things that most excite you about the industry? Can you explain or give an example?

There are so many innovations coming out of the XR industry, and the applications are very wide. They will touch on every sector of our lives — entertainment, healthcare, transportation, communication, and social connection. My examples will be focused more on AR and MR, since this is my area of specialization.

1. A new type of Presence (Spatial Telepresence) — The #1 thing I am excited about is “spatial telepresence” using avatars or volumetric videos. XR is all about “3D-ness” or seeing “spatial” content, whether you see it inside a fully immersive VR environment or overlaid in the real world. Connecting with others halfway across the world as an avatar or volumetric video, feels like the next thing to being there in person. Except, I just saved myself an expensive plane ticket, the carbon footprint created by jet fuel, jet lag, and about 20 hours total of flying time. In the future, this will feel like teleportation and this kind of new “presence” will have applications in the way we work, play, and stay connected with one another.
2. Lightweight AR utility apps — One of the design lessons I learned from working in AR is that less is more. In the early days, it was difficult to resist the urge to create large AR content or applications filled with a lot of holograms. This led to the cluttering of the user’s visual field and obstruction of seeing much of the physical world — which can be dangerous in certain situations. In AR, the physical world takes priority, and the content is enhanced. All those useful utility apps on your phone will eventually go away and end up inside your everyday objects and the world outdoors. These lightweight AR utility apps will provide you with the information you need on the spot without obstructing your view — such as google search, checking traffic, or sending a message.
3. Digital twinning and multiple copies of the world — In combination with lidar scanning, geo-spatial location, cloud technology, and augmented reality, we can anchor holograms to real physical locations based on context. For example, scanning a city block will result in the creation of its digital twin. We can attach all sorts of meta-data to that digital twin, such as coordinates, list of businesses, traffic, local temperature, etc. and store it on the cloud. An AR visualization created to represent temperature, for example, could be displayed to a user walking through this city block. One digital twin can be copied many times (infinite!) and relevant information can be grouped and stored together. Traffic data can be grouped and stored in the traffic digital twin, weather data in the weather digital, and so on. These invisible layers of information are stacked on top of each, and a user could “tune in” or “change the channel” to the appropriate layer based on what they need.

(optional photo use below)

Image credit: Magic Leap 2019

What are the 3 things that concern you about the VR, AR and MR industries? Can you explain? What can be done to address those concerns?

1. Accessibility — I have concerns about “tech for the rich”. It’s an accessibility problem that arises when new tech is expensive and only available in urban or more developed regions of the world. When only certain regions of the world can use technology, the rest of the world is left behind. This is true for internet access, quality infrastructure, and even clean water. I often felt uncomfortable speaking about the future use cases of AR outside of the US, depicting concepts and drawings of smiling people looking at fantastical AR visualizations overlaid over their clean city. Countries struggling for clean water, basic housing, and safety have no interest in this. It made me want to focus on AR use cases that highlight problem areas and are used as a communication tool. AR can be used to visualize complex data in 3D space as well as run complex simulations to arrive at solutions quickly.
2. Formfactor — I am unsure if glasses are the way…at least for full phone replacement and all-day use. I just can’t picture society wearing things on their face for extended periods of time, it’s a lot to ask. Even contact lenses are intrusive so I’m skeptical about that too. I really have my eye on the next technological leap from visual projection systems, which feel like it could offer spatial information to us without the burden of putting stuff on or in our bodies. Putting my UX hat on, I’m always going to lean on the more natural solution. XR can be offloaded to the physical structures outside and in our environments.
3. Disconnecting from the physical world and increased distractions — XR, particularly VR, can offer worlds that are more appealing than reality. In the same way we have the tendency to escape to television, video games, or our phones, when our lives are stressful, I worry that VR will make this temptation even greater. Just like our phone, AR/MR has the potential to further increase the bombardment of information. Instead, this time the information will be in front of us and not tucked away inside our small devices. I worry that we are not ready to handle such a large change.

In summary, all 3 of my concerns center around XR not being designed around problems and people enough. We’re rushing into developing this new technology because it is new and novel. If we don’t slow down and put more focus on the end goal, then we will be scrambling to fix many unforeseen problems down the road. (similar to our plastic and social media addiction problem) We need greater focus around security, safety, and inclusion.

The entertainment aspects of VR, AR and MR are apparent. Can you share with our readers how these industries can help us at work?

I’m particularly excited about XR presence technology, or “spatial telepresence”, as it is called sometimes. What I like most about this concept is that it has the potential to reduce travel (carbon footprint reduction and saving time) and give us the next best thing to in person contact, when being in person is challenging. My enthusiasm for this technology has further increased since entering the pandemic area, after being cut off from my colleagues.

At the start of the pandemic in 2020, I had the privilege of working with the United Nations Development Programme to test out some of these XR presence technologies. My team was highly distributed, stretched across California, Colombia, Zimbabwe, New York, Croatia, and Turkey. We kicked off this program in Jan 2020 and met in person one time before lockdown kicked in. We decided to dog food the project and purchase a few VR and AR headsets. Our team partnered with Spatial (https://spatial.io/), a leading XR remote space developer, and we started having meetings inside the application. It was amazing to see a semi-realistic avatar version of my colleague in Croatia floating beside me in my Los Angeles home. We communicated via voice in real-time and set up our virtual room with presentations, images, graphs, videos, and maps to be shown to our stakeholders.

Having to execute this pilot program during the pandemic turned out to be advantageous for us because the need was noticeably clear. As technology matures, I can see this becoming an even more valuable tool to connect teams across the world, especially during tough times.

(optional photo use below)

Image credit: Naomi Augustine-Yee & UNDP 2020

Are there other ways that VR, AR and MR can improve our lives? Can you explain?

Another use case I am excited about with AR/MR is real-time language translation. I would love to wear a pair of AR glasses that can display translated text of the speaker into my field of view in real-time when I am traveling to a foreign country or meeting with international colleagues, clients, or friends. If we were both wearing AR glasses then the communication barrier would be greatly reduced, and our sense of connection increased. This is something I would gladly wear glasses all day for, as the benefits outweigh the discomfort.

AR language translation and telepresence are both concepts that I chose to write about in the first chapter in my book “Mixed Realities”.

Let’s zoom out a bit and talk in broader terms. Are you currently satisfied with the status quo regarding women in STEM? If not, what specific changes do you think are needed to change the status quo?

No. I can’t deny that there have been improvements here and there but it’s 2022 and I expected us to be further along. In my honest opinion, the tech industry has a lot of work to do for women. I feel like I have been fortunate to have been given the opportunities that I have, but my journey was still filled with difficulties because of my gender.

This is a complex problem that unfortunately needs to be addressed at several levels simultaneously — the fabric of society, deliberate behavioral corrections at institutions/corporate settings, education, and parenting level. All 4 aspects need to change together in order to make significant improvements. I particularly want women’s accomplishments highlighted more in the science and tech industry so that they can serve as inspirations for younger women who are trying to find their place in the world.

What are the “myths” that you would like to dispel about working in your industry? Can you explain what you mean?

Myth 1 — The only technical job is programming: I do not have a CS degree. When I switched from aerospace to gaming and AR, I initially felt the shock of programming being the dominant technical skill in the field. I took my physics and EE background and applied it to software and learned some coding along the way. The result was systems engineering, a valuable discipline needed to build something large or complex, like a satellite, an online multi-user application, or an AR headset. (I have worked on all 3) The AR industry also has an array of other STEM roles that aren’t centered around programming — optics, wafer process/fabrication, mechanical engineering, data science, deep learning research, hardware and industrial design, systems architecture, systems testing, and more.

Myth 2 — There’s no room for creativity in tech: Tech cannot progress without creativity. Creativity in tech is called innovation — identifying a problem/need then producing a solution, designing a new architecture, performing R&D, and iterating through trial and error. I often found myself tapping into my creative brain more than my tech brain during my experience. In quality assurance, you can hire 30 QA testers to click on buttons to find defects, but if you can build 1 automated solution then you just designed a new and better system. In AR, hundreds of engineers were building optics and CV algorithms, but creativity was needed to understand which features were more important in the context of a real-life application. Understanding a problem requires you to envision yourself in an array of scenarios — its part imagination and part analysis. It’s imagining a user journey, it’s kind of like storytelling and constructing a world in your mind. I often worked with artists to have a series of concept images drawn to help visualize how an AR application could be used. This in turn informed content creators with artistic direction on building 3D models for the application itself. That was my favorite part about working in AR — I got to use both my creative and tech brain.

What are your “5 Leadership Lessons I Learned from My Experience as a Woman in Tech” and why? (Please share a story or example for each.)

1. Focusing on solving problems — A tried and true lesson I stuck with that always gave me remarkable results. Looking for a problem to solve provides you with a clear goal, or north star, and helps you stay focused. What I also learned is that people love solving problems. I found it easy to rally a team around it as well as get buy-in from other leaders. This was always my starting point when I created a new project, pitched an idea or design, or built a new team.
2. Saying No — I learned the art of saying “no” after burning out from saying “yes” too many times. It’s an easy trap to fall into, especially early in your career because you want to show off how competent you are. When you say “yes” to too many things, your time and energy are spread thin, and the quality of your work suffers in each category. Women are especially vulnerable to taking on more work than they should, so I found that establishing clear boundaries is critical. “No, I will not be the note taker in this meeting.”
3. Ask for help and show a willingness to learn — This is a terrific way to build and strengthen your relationship with colleagues and learn something new along the way. This was especially true working in AR, where many industries and experts came together from different backgrounds, often clashing. For example, the word “design” meant something different depending on which department you were interacting with. It was easy to feel frustration when a common language hadn’t been established yet. If I didn’t understand something, I set aside my pride and asked for help. It is perfectly okay to say “I don’t understand this, can you please show me?” Overtime, your knowledge pool grows and you can pass the information along to others.
4. Be kind and diplomatic, especially to difficult people — A bit related to the point above, being kind and diplomatic helps you build and strengthen your relationships with others. It’s no secret that there are a lot of arguments happening inside tech companies, where it seemed more important to be right even if it meant burning bridges. I have observed that those who chose this path, found themselves without allies because they were seen as uncollaborative. In the end, I discovered that more opportunities presented themselves when I conducted myself in a kind and diplomatic manner. I became someone everyone wanted to work with, even the difficult people. I used these skills to connect people together and establish a collaborative cross-department relationship.
5. Seeing everyone as “knowledge blobs” when finding yourself in a room full of men. I was often the only woman in the room. When I noticed this, I became self-conscious and timid, often not saying a single word in these meetings. I had created an imaginary handicap for myself, believing I somehow didn’t deserve to be there because of my gender. In order to overcome this, I imagined everybody in the room, including myself, as amorphous blobs of knowledge. Being one of the many intelligent blobs sitting in the room, I was able to speak up with more confidence about my area of expertise. I was invited to be there for a reason so it helped me own that. (This trick also works when you are the most junior person in the room)

You are a person of profound influence. If you could inspire a movement that would bring the most amount of good to the most amount of people, what would that be? You never know what your idea can trigger. :-)

*Working together as a species*

This phrase is rather loaded, as it contains a long list of things I wish to improve about our world and society. Tackling environmental issues like climate change, inequality across all the “-isms”, wealth gaps, greed, animal cruelty, politics, wars, the list goes on. In a nutshell, I’d like humans to get better at working together as species. To focus more on our similarities and common survival goals rather than obsessing over our differences. The way I’m trying to be heard is by writing novels and incorporating this theme into my stories.

We are very blessed that very prominent leaders read this column. Is there a person in the world, or in the US with whom you would like to have a private breakfast or lunch, and why? He or she might just see this if we tag them :-)

Sean Carroll. Sean is one of my favorite physics communicators and his book “From Eternity to Here” inspired several of the stories in my book “Mixed Realities”. After reading his book, my head was swimming with thoughts about entropy, Boltzmann brains, multiverses, the connection between information and life, and I felt compelled to write some stories. I’d love to give him my book and chat about the nature of the universe. Maybe even discuss creating an AR visualization of some physics concepts.

Thank you for these fantastic insights. We greatly appreciate the time you spent on this.