Future of Immersive Experiences
A Bharat Innovation Fund Perspective
By: Rohan Choukkar
Immersive experiences have traversed a long and interesting path with continuous efforts and innovations being made to blur the lines between the virtual and real worlds. One of the earliest successful attempts at creating a virtual reality experience was the Sensorama prototype developed in 1960 (patented in 1962) by cinematographer, Morton Heilig, to fully immerse the viewer into the film. The Sensorama was an arcade-style theatre cabinet that included stereo speakers, fans, stereoscopic 3D display, smell generators, vibrating chair, etc. to stimulate more than senses of sound and sight.
The following decades have seen several attempts, innovations and inventions to craft modified realities in the form of virtual reality (VR), augmented reality (AR) and mixed reality (MR) and immersive experiences that are much more realistic and futuristic. VR transports the user to an artificial environment using headsets/ head-mounted displays, PC/ console and haptic controllers. It is often considered to be best suited for gaming and entertainment owing to the higher levels of immersion VR offers. AR augments the real environment of the user by digitally overlaying information/ insights/ content in real-time over existing physical elements. Ever played the game of Pokémon Go or used Snapchat filters? These are the most widely known examples of AR. Owing to the lower possibility of immersion and not completely cutting the user off the real environment, AR is considered best suited for professional contexts. MR tethers the virtual elements and environment to the real environment (instead of simply overlaying elements) by using next-gen sensing and imaging technologies. Users can literally have one foot in the real world and one foot in the virtual world! It is known to have big possibilities in teaching/ training, sports and entertainment, remote working, healthcare and engineering/ construction owing to its ability to present 3D virtual objects/ images in real-world settings. For instance, Microsoft’s HoloLens was leveraged by Case Western Reserve University in Ohio, USA and Cleveland Clinic to develop a unique MR-based app to reimagine how anatomy is taught to medical students and is a groundbreaking innovation in the space.
Throughout this 6-decade period, VR has been touted to be the next big thing and continuous work has gone into crafting modified realities. However, it is only in the past few years that cutting-edge tech innovations, compelling content and immersive experiences in specialized use cases across a variety of industries have come close to fruition. This is owing to a series of revolutionary technological changes that are leading the industry to the cusp of a breakthrough that will change the face of the existing business models and will steer the ecosystem towards maturity.
Up until 2018, a vast majority of investments and developer/investor interests were concentrated on developing and innovating top-notch, cutting-edge hardware such as headsets, head-mounted devices, etc., fundamental infrastructure and device hardware, optics, glasses, displays, etc. which have created a solid foundation for the sector to develop. Considering the large quantum of sci-fi culture, academic imaginings, etc. that have been built around artificial reality for several decades, gaming, entertainment and related content creation have a stronghold in the ecosystem and the market. Case in point — there are over 6000 AR/ VR content studios in the US alone!
Over the past few years, however, innovations and path-breaking use cases in a wide range of domains have been discovered/ invented/ imagined. For instance, a doctor/ surgeon leveraging gaming technology, ultra HD 4K displays, next-gen imaging, etc. enabled by XR (umbrella term for VR, MR, AR) to peer into a patient’s body with minimum/ no incision, getting detailed organ maps and predicting risks/ hitherto unforeseeable circumstances and planning ahead to promote greater precision, lesser complications and faster recoveries among patients. Similar simulations enable better predictions and preparations in military training and operations, engineering and automotive industries. XR with its gamification capabilities, cutting-edge innovations and development frameworks is steadily making inroads into verticals such as real estate, branding, retail, education, etc. for use cases such as design and development, experiential learning, simulation, unique and highly customizable visualization, etc.
All thanks to the accelerated pace of technological and computing advancements, the proliferation of smartphones (especially AR-enabled ones) and high-speed internet (especially the upcoming 5G), the space has become more democratized and has enabled us to discover and develop path-breaking use cases over and beyond gaming and entertainment-oriented content.
Traditionally, VR has played a dominant role in the ecosystem with huge investments flowing into startups and companies attempting to create innovative use cases with VR. However, we have observed that the interest in VR is slowing down and AR & MR are gaining dominance in the past 5 years. This shift is attributed to the 3 factors. Firstly, VR relies on hardware and equipment such as laptops/ computers, heavy VR headsets, etc. and power supply to function, thereby, increasing the cost factor. Secondly, VR hardware is not portable and scores low on the locomotive capabilities for developers and users, increasing their hassle costs, unlike AR/MR capabilities that are available on portable smart devices (phones, tablets, glasses, etc.). Thirdly, the barriers to entry with respect to AR development are lower because dedicated AR frameworks, app-based and web-server-based, for different smart device operating systems that are enabling effective plane surface detection, superior algorithms for edge surface detection and dedicated AR support within standard OS have reduced the need for third-party development tools, thereby reducing costs of development. So, even startups and companies with small teams can engage in AR/ MR content creation, app development and other use cases. There is a need for massive advances on the VR front if it is to become mainstream in the long term.
The emergence of these XR stacks such as Google’s ARCore for Android and Apple’s ARKit for iOS have created a solid stack that can run on 50+ smart device models. These dedicated app-based AR development frameworks enable developers to develop XR-based apps that can run on smart devices. Despite their many benefits, app-based XR stack does have certain frictions such as development and optimization of apps for different devices and objects, need for higher bandwidth, continuous updating for apps for new content creation, etc. WebXR, on the other hand, is a standard interface for a range of devices and does not require developers to create separate apps. This browser-based approach too has certain frictions such as the need for rendering on cloud and recognition across devices for the content to show on them, higher bandwidth requirements, etc. Both mobile AR and WebXR are in their infancy stages and it is only a matter of time that they will forge ahead. However, the emergence of WebXR and mobile AR have certainly democratized the space.
The emergence of 5G internet is considered important for the development and fast-paced growth of the XR space because it provides a unified platform for a wide range of devices/objects/ machines (not just mobile broadband) and delivers multi-GB peak rates, reduced latency, higher efficiency and improved performance. With heavy 3D assets and multiple connected devices and objects (such as glasses/ headsets, etc.), 5G internet will eliminate a majority of frictions related to mobile AR and WebXR and thereby, elevate XR.
The horizontal stack in the XR ecosystem will most likely be owned by major players such as Apple, Google, Facebook, Microsoft, etc. who are pumping in substantial investments into the development of a myriad of hardware and cutting-edge innovations and dominant unicorns like Magic Leap and Blippar. We, at Bharat Innovation Fund, believe that the vertical play may be the path that startups and smaller players in the XR space opt for as the vertical stack has not been fully explored and disrupted by the dominant global players. This is especially true for India which has hitherto played only a miniscule role in the XR space and much of the work done is focused on content creation and backend work in model conversion, catalogue digitization, etc. Emerging players in the space such as PlayShifu are going for the vertical play.
We see the XR space in India changing in the near future due to the democratization of XR, 5G rollout, edge computing advances and greater vertical stack disruptions. We strongly believe that we are on the cusp of a major breakthrough in the AR/VR/MR space in India and this revolution will be led by next-gen startups and venture capitalists investing in fueling these startups.
About the Author:
Rohan Choukkar is an Associate at Bharat Innovation Fund.
