A brief history of immersive technologies
Natalie Mouyal takes a look back and forward to what’s next in VR and AR
The recent movie and bestselling novel, Ready Player One, tells the story of a hidden quest within a virtual reality (VR) platform, where characters escape from the hardship of their real-life city slums. It is not far removed from the visions of the first pioneers in virtual reality.
While the term “virtual reality” did not enter mainstream until the mid-1980s, the concept already existed in popular imagination. Initial exploration into 3D and a 360° field of view, simulated experiences, virtual worlds and head-mounted displays (HMD) had emerged by the late 1990s. However, it was not until the 21st century that the virtual reality experience became more established with the advent of smart phones, powerful computer graphics, motion controllers and computer interfaces that track gestures.
The theoretical beginnings of VR
Virtual reality began with works of fiction. Writers transported their characters to imaginary worlds that, based on today’s technology, appear to be prophetic. In one of the first short stories to imagine a virtual reality, Pygmalion’s Spectacles, published in 1935, characters wear goggles to experience “a movie that gives one sight and sound…taste, smell, even touch. […] You are in the story, you speak to the shadows, and the shadows reply, and instead of being on a screen, the story is all about you, and you are in it”.
Other classics that stirred popular imagination included the television series Welt am Draht (1973) directed by Rainer Fassbinder, the movie Tron (1982) which transports a computer programmer inside a mainframe computer, The Lawnmower Man (1992) and The Matrix (1999).
However, it was the essay The Ultimate Display, written by Professor Ivan Sutherland in 1965, that provided one of the first scientific visions of virtual reality. It describes an “ultimate display” connected to a computer to create a virtual world that provides audible and tactile feedback and the ability to interact with objects. Many of the hypothetical suggestions made in the essay have since come to fruition, including gestural interfaces, eye-tracking, haptics, augmented reality (AR) and voice recognition.
Development of VR technologies
Initial attempts to create a virtual reality were based on photography and film. With the stereoscope developed by Charles Wheatstone in 1838, a viewer could perceive a 3D object by having each eye look at different 2D images. This technology remains relevant for AR applications such as Google Cardboard which superimposes stereoscopic images onto a smartphone screen.
In the late 1950s, the cinematographer Morton Heilig developed the Sensorama, which sought to fully immerse the viewer in a short multisensory film. A mechanical device that resembled an arcade game, the Sensorama provided the viewer with moving, 3D images, scents, vibrations and tactile sensations such as wind.
By the late 1960s, it had become possible to integrate electronics into virtual reality environments. In 1968, Ivan Sutherland, with his student Bob Sproull, developed the “Sword of Damocles” which is now considered to be the first HMD system. While the user interface and images were primitive, the system displayed output from a computer programme and could track the user’s eyes and head position.
The US government, through its Department of Defense, National Science Foundation and National Aeronautics and Space Administration (NASA), has funded many projects essential to the development of virtual reality technologies. Areas of research have included computer graphics, networked environments and simulation.
Edward Link developed one of the first commercial flight simulators in 1929 which was later used extensively by the US military to train its pilots during World War II. While the first flight simulators relied upon mechanical systems to provide pilots with feedback, the introduction of electronic systems greatly enhanced the user’s sense of reality. To train soldiers in dangerous situations, Charles Comeau and James Bryan developed the Headsight in 1961. This linked a remote camera to the movement of the user’s head.
In the mid-1980s, the engineer Thomas Furness designed a helmet for military pilots that projected information in an immersive and 3D space that pilots could view and hear in real time. His work, which began in the 1960s, included motion tracking, 3D sound, and the use of speech and gesture as sources for user input. To further increase the sensory environment for pilots, Louis Rosenberg developed the virtual fixture platform in 1992 which provides 3D immersive reality.
Haptic feedback began with the use of telerobotics by NASA and the nuclear industry. A master arm controlled a remote — slave — arm, usually in order to operate a space vehicle or handle irradiated material. The haptic systems developed by the University of North Carolina at Chapel Hill, under the auspices of the GROPE project, have also provided an important insight into the use of master control arms for virtual reality applications.
Consumer interest
The video gaming industry has helped to propel the technologies used for virtual reality and inspire popular imagination. Consumers, buoyed by visions of virtual reality from books and movies, began accessing technologies in the 1980s and 1990s. Gear developed to provide haptic feedback became commercially available from the company VPL Research, whose team included such VR pioneers as Jaron Lanier and Tom Zimmerman. Its equipment included the Data Glove, EyePhone and Data Suit.
By the 1990s, the gaming industry had begun offering virtual reality headsets and games such as Sega’s VR headset and Genesis console and Nintendo’s Virtual Boy. The company Virtuality developed a network of arcade machines that allowed gaming in a multiplayer immersive environment. However, by the late 1990s, the hype surrounding virtual reality had faded into disappointment and mass consumer demand diminished. Despite these setbacks, virtual reality continued to prosper within the gaming industry with the establishment of virtual communities and massively multiplayer, online role playing games (MMORG).
Current VR applications
VR and AR applications are now flourishing beyond the gaming or home entertainment industries. From education and training to manufacturing and health care, these applications are becoming increasingly prevalent.
In education, students can interact with objects within a 3D environment. Medical students can be trained while watching live-streamed, 3D surgeries from anywhere in the world. Visitors can walk virtually through heritage sites or, in the case of the Kremer Museum, visit a virtual museum collection. First aid responders are immersed in seemingly real-life disaster scenarios while miners learn to recognize risks while walking through virtual mines.
Manufacturers benefit from virtual and AR applications. Interactive 3D modelling tools used by car and rail manufacturers allow designers to view and test their products before production begins. HMDs provide employees with additional sources of data that are useful for inventory management or warehouse navigation. Construction crews can gather data to visualize real-time conditions and make adjustments as necessary.
In healthcare, virtual reality applications provide therapy to children with autism and help treat post-traumatic stress disorder. Surgeons use visual tools in preparation for an operation. Applications allow doctors to perceive the experiences of a patient with hearing and visual impairments.
What’s next?
While Ready Player One gives us a glimpse of the possible future for virtual reality, that future remains elusive. More recently, the most successful applications have been able to provide an AR, whether that’s finding Pokemons or tracking inventory. However, it is not yet clear how we will decide to merge virtual and real environments, much less why or which interfaces will be successfully adopted. Works of fiction may provide the best source of inspiration.
NM
A version of this story appeared in issue number three of e-tech in 2018.
VR and AR applications rely on such components as screens, processors, motion sensors, gyroscopes, cameras and images which are linked to hardware such as headsets. IEC technical committees (TCs) and subcommittees (SC) are responsible for producing International Standards that make the hardware and software used in these technologies possible.
The Joint Technical Committee of IEC and ISO, ISO/IEC JTC 1, provides standardization in information technology. Specifically, ISO/IEC JTC 1/SC 24 develops Standards relating to image processing, computer graphics and virtual reality.
IEC TC 47 and IEC SC 47F are responsible for the standardization of sensors and microelectromechanical systems. IEC TC 100 produces Standards which contribute to the quality and performance of audio, video and multimedia systems and equipment while IEC TC 110 covers electronic display devices including touchscreens.
`
