Beyond Broadcast and Social Media: Game Engines are the New Reality Engines
By Scott Broock & Mike Seymour
Part one of several articles on the case for a new cloud based interactive media company built on top of today’s game engine technology
Author’s Note: Within a day of this article, Andreessen Horowitz announced a $150 million investment in Roblox, raising Roblox’s valuation to over $5 billion. Roblox is a massive online social community and game platform where 115 million kids and teens come together each month to play and create.
It’s worth reading what Andreessen wrote to explain the importance of the deal to their portfolio.
“People have been talking about the “metaverse” for a very long time, ever since sci-fi writer Neal Stephenson first coined the concept of a global, virtual, persistent space shared by many. While pundits have been distracted by the readiness debates and questions over VR vs. AR, the foundations of a global metaverse have been quietly built in the background… in Roblox.”
The message could not be more clear for legacy media. Social gaming is eating the world.
Games are Eating the World
Have you ever dreamed of your own personal oasis, a place where you can be or do what you want with others, free from the overload of today’s media? It may not be as far off as you think.
Hollywood is spending billions of dollars on a global bet that streaming television and video will endure lucratively in the challenging next phase of the internet. But there is a risk that these large bets — packaged as bite-sized, direct-to-consumer, on-demand, ad-supported or subscription-based services — are just content traps.
In a world awash with information and eye candy, “social gaming” is rapidly evolving as the largest existential threat to legacy media’s claim on engaged consumer attention. Trust and manipulation issues are also causing people to look beyond incumbent social media platforms such as Twitter, YouTube, Instagram and Facebook.
Beyond the marketing buzzwords and enormous ad spends, edge computing, advanced graphics processing, machine learning (AI) and 5G are truly creating the perfect storm for a new real-time network that delivers a rich medium for interactivity, creativity and communication. This new network will transform today’s game engines into reality engines for collaboration and emotional participation.
Which partly explains why Netflix CEO Reed Hastings has been widely quoted as saying that he’s more concerned with the competitive risk of massively networked, interactive experiences like Epic Games’s Fortnite than Hulu or HBO Max or any other traditional broadcaster.
Blueprints for the Metaverse
Epic Games’ CEO Tim Sweeney predicted in 2017 that we were three years away from the beginning steps of the ‘Metaverse’ — an homage to the virtual reality of human-controlled avatars in the 1992 sci-fi masterpiece Snow Crash. Loosely stated, the Metaverse is a term that has morphed since the publication of Snow Crash into a catch all, pop culture reference to a gaming universe where characters play and mingle socially in a mirror-world of reality. Think Wreck it Ralph meets Ready Player One. Sweeney is not alone, but as the head of one of the most impactful technology companies in this space, his vision has deep implications — and for good reason.
His vision of the Metaverse is more than the extended universe of Hollywood character IP that many have made it out to be. As Sweeney stated last summer at SIGGRAPH and expanded upon at this month’s DICE conference, it is as much a rich communication medium as it is an entertainment experience.
Sweeney’s solution to — and reconceptualization of — today’s siloed media portals is an open source visual ecosystem for play, socialization and discovery that can be experienced any time on any device in the physical and the digital world. Interoperability will drive greater engagement and foster an open economy for creators to monetize virtual goods and services, and the portability of social graphs between games and experiences will create an open environment for discovery. The Metaverse as a new sticky destination can encompass, and be backwards compatible with, legacy media.
In totality, the promise of this Metaverse is far more than just its content — it’s an entirely new network from today’s streaming video and social media hubs, and that’s a very big deal.
Engines of Creation
In the The Hive is the New Network, VCs Arjun Sethi and Andy Artz argue that the first phase of the internet was to connect the world. The second phase was to bring real world relationships online. The next phase of our connectivity will be the reduction of friction between people and information to create a smarter, evolved network that is bigger than the sum of its parts. A view similarly held by VC and former CEO of AOL, Steve Case, who contends in The Third Wave: An Entrepreneur’s Vision of the Future that the internet will be defined by deep cooperative partnerships instead of siloed gatekeepers.
This is precisely the arc that we’ve watched immersive entertainment take over the last decade. It has evolved over time to address the shortcomings of the previous incarnations of mass media. It is not that the prior media and social platforms were uninteresting, rather it’s the inevitable rise of new models that better address the shortcomings of entrenched incumbents to address users’ desires for more connected and involving experiences.
Game engines have moved rapidly beyond games to be interactive high-fidelity graphics and environment engines. They provide the foundations upon which the true next generation of interactive experiences such as Sweeney’s Metaverse can be built. Previously reserved for high-end visual effects houses and game development studios, these tools have become increasingly affordable, accessible and pervasive enough to work on a global scale with the necessary fidelity for a new generation of creators. This critical tipping point has been hard for many legacy media executives and technologists to see, as they try to understand the coming Metaverse as a powerful new medium.
This new medium is distinctly different from previous asynchronous mediums in that it provides real-time authentic interaction. Posting on social media today is still effectively a personal ‘broadcast model’ of one to many — post snapshots and wait to see if anyone engages with it later. As a matter of human nature, people are seeking even greater interaction and control of their identities.
People decry their loss of privacy and freedom in legacy business models that, as Sweeney has pointed out, are based on adversarial customer models. Their business approach suffers from a ‘mind virus’ of wanting to ‘own’ customers. In this legacy model, old media seeks eyeballs or attention and it seems willing to promote or do anything, no matter how unbalanced, to achieve inherently narrow metrics. Clicks, Views, Likes and Ratings all avoid any sense of championing rich and meaningful engagement. Such vanity metrics bias to internet echo chambers of the banal.
Rather than seek to trap attention and own the customer experience through siloed or walled app ecosystems, new interactive content built with game engines can be accessed through instantaneous links, eliminating the friction of downloads. Snap, WeChat, Twitch, Mixer and browser-based portals all become direct gateways to a meta-platform of shared experience across all devices. By expanding the pool of compatible devices to every connected device that can connect to the internet, the odds of content discovery, feedback and innovation increase. A strong answer to the gating problems that continue to hold back Virtual Reality.
Which is why cloud gaming is so fascinating. Nominally, it is about replacing home consoles with remote servers that can deliver complex game play to any device, any time through a broadband connection. The reality, however, is that the technology needed to make cloud gaming work is being built out to serve the next generation of decentralized mass communication.
Data centers have become the next battleground for Amazon, Microsoft and Google as they compete to push data services and information closer to where we live, work and play.
Within those data centers, AMD, Intel and NVIDIA are waging their own battle to provide scalable GPU rendering and AI computing for immersive simulations that can be accessed at home, on the go and in large social venues through low latency connections including 5G.
On top of this physical infrastructure, game engines will crack open the tools of visual effects and character production in much the same way that Final Cut Pro and Premiere ushered in an era of user generated content in the early 2000’s. The difference this time around is that the output is three dimensional, interactive and scalable.
At the top of this interactive infrastructure is you and the visual identity that you can choose as your representation to the world.
Consider that star Fornite player Ninja with his colorful hair, and DJ Marshmello with a bucket shaped helmet, wear their gaming personalities in a fluid state of identity that fits their celebrity in the real world and their extended selves in the digital world. Fortnite is a mirror-world virtual stage where they perform with their audience of millions, not to them.
We have seen that when Epic Games creates a skin of these celebrities for use in Fortnite, they elevate these iconic characters for TikTokers to emulate and power through dance competitions, and enhanced personal interaction.
And as Sweeney has stated, his goal is to simplify and extend the tools of Fortnite so that anyone can host a concert like Marshmello, any time that they want.
The Metaverse isn’t just a next generation playground, it is a next generation network for the celebration of flexible identities. A celebration that can be jointly experienced and participated in by users without the fear of failure — the number one obstacle to innovation. In many respects, the use of an avatar or digital identity is the next iterative jump from anonymous forums which themselves were a jump from ghost writers and pen names.
Building your own Social Universe
Slipping into a configurable skin that reflects a disguised talent, free of judgment, liberates people to be the hero of their own realities. It also allows the pursuit of social capital without sacrificing your true identity or trading private personal freedom. Imagine being an influencer without loss of privacy concerns. Be who you want to be and protect your personal identity for those you trust.
In Japan and China performers are achieving social consequence as Vtubers. These real-time animated characters and holograms use facial AI and motion capture to perform live and interact with their audiences in real-time. Anime performances can be instantly thanked and applauded, with immediate and often individualized onscreen tributes. This phenomenon is fast spreading throughout Asia and is just now reaching the US.
The next logical step is a character customization tool that lets you, like these digital celebrities, become a character with your own style, personality and social profile. In doing so you would preserve your verified, but private identity. A secure marketplace for validating and sharing virtual merchandise, props or outfits for your avatar can add an economic layer for monetization that incentivizes trade and collaboration with others, creating strong network effects.
With the annual Game Developers Conference (GDC) and NVIDIA GPU Conference (GTC) right around the corner in March, it’s a strong bet that tools of this nature will be announced by a range of companies with a vested interest in expanding the digital human space to an eager community of developers and creators.
Living an Animated Life
It is instructive to look at the way that today’s youth are experiencing music, style and art in their media to understand what comes next in the world of social media and status. Animation styles are now more than ever reflecting diversity and original artistic expression. Sony Pictures Animation’s triumphant Spider-Man: Into the Spider-Verse opened a new dimension to representation in the crowded world of live action superheroes by stylistically fusing an older medium with the visual language and color palette of anime and the sensibility of today’s gaming culture.
Riot Games has mastered the art of turning their animated League of Legends Champions into hyper stylized pop stars that attract several hundred million views on YouTube, and real world DJ Marshmello performed a rave in Fortnite that drew several million concurrent players as audience members. Most recently, anime fan and Grammy winner Billie Eilish teamed up with famed Japanese artist Takashi Murakami for the tour de force music video you should see me in a crown that features Eilish as a hyper kinetic, motion captured avatar.
It isn’t just that we are getting a new medium to play games that look like cartoons and vice versa, it is the way that games and animation are changing our identity and mode of communication that’s defining the new medium of the coming Metaverse.
Which echoes strongly the last time music, style and art collided with a new youthful medium: MTV.
The Next MTV
Cable television had just started to penetrate America’s neighborhoods in 1981 when it found its killer app. A “collide-oscope” of music and in your face short form content that kicked off with a now iconic man on the moon opening and the debut of The Buggles’ Video Killed the Radio Star. America, and soon the rest of the world, couldn’t get enough. MTV exploded onto the scene and defined a generation in much the same way we are watching global gaming engage the youth of today.
The difference is that today’s mixture of rebellion is interactive, immersive and virtual.
As Hollywood turns to game engine technology and virtual worlds to make movies like The Lion King and television like The Mandalorian, the opportunity for narrative innovation will grow.
Digital assets and environments are building blocks that can be mixed, matched and reconfigured on the fly as traditional video — interactive shows and extended reality for the next waves of augmented and virtual reality.
Viewed as a hybrid of shows and social media, this next generation of interactive entertainment can also borrow from the safeguards of film and television’s past to ensure that sponsors reach the correct audience with the right inclusive brand positioning.
Characters can have embedded code — a digital DNA if you will — that authenticates the age of their users for COPPA compliance, and interactive shows can be marked with clear ratings that are family and advertiser-friendly.
All of which is welcome news for families underserved and oversaturated by the wild west of programmatic advertising and insufficient moderation by today’s incumbent social platforms.
McLuhan and the Metaverse
When the late Canadian scholar Marshall McLuhan coined the phrase “the medium is the message” 70 years ago, people had a hard time understanding its meaning. From today’s perspective, it seems simpler to decode and increasingly relevant.
According to McLuhan, all of humankind’s technologies are extensions of our bodies and minds. Each advance in technology stretches our sensory perceptions of the world in new directions, creating a new environment — or medium — that reshapes communication and society.
The printing press led to radio which led to television which led to the internet and so on. Handed an iPhone today, McLuhan would have likely argued that mobile devices are nothing more than extensions of our brains, carried in the palms of our hands, for the purpose of declaring identity in a global village of social media.
McLuhan predicted that the electric age of television and networked computers would merge into a fundamentally different environment than those that had developed over longer periods in the past, creating a connected world of always-on entertainment. Gaming, personal computing and telecommunications are converging to create that new participatory medium.
The Metaverse that Tim Sweeney is imagining takes McLuhan’s thesis to the next level. In today’s world of Fortnite, TikTok and soon 5G, the Metaverse is now the message.
As we accelerate up the steepening curve of innovation and our communication networks evolve into a high speed, low latency connective tissue between our devices and our daily lives, the line between artificial and real will continue to blur — as will the line between gaming, entertainment and identity.
The world of interdependence that McLuhan anticipated — and that Sweeney is now committed to realizing through the reality engine of the Metaverse — will soon change society and human communication.
Scott Broock is the former Executive Vice President of Strategy and Innovation for Illumination Entertainment, Global VR Evangelist at YouTube, and Vice President of VR Content and Deal Development at Cinematic VR Pioneer, Jaunt. He has worked at the intersection of gaming, telecommunications, TV and film for almost two decades. Broock’s current venture, Totem Networks LLC, is funding and developing avatar-first interactive global experiences for streaming media, cloud gaming and location based entertainment.
Mike Seymour is a lead researcher in the Motus Lab at The University of Sydney. His research explores using artificial intelligence and interactive photoreal faces in new forms of Human Computer Interfaces (HCI). He is the co-founder of fxguide and fxphd which chronicles and educates the film, television and gaming world on state of the art visual effects, virtual production and digital humans. Seymour recently was the chair of SIGGRAPH ASIA’s “Real-Time Live” session featuring the latest advances in motion capture, render and interactivity.