Development of Early VR

Nat DeMenthon
Jul 31, 2020 · 8 min read

As seen through the lens of historical CHI videos.

Sutherland sits at an antiquated computer that has 4 views of an outlined 3D model of a chair.
1964; Computer Sketchpad; MIT Lincoln Lab, Sutherland

In 2018, the Association for Computing Machinery (ACM) decided to make its video archives available online. ACM is the organizer of influential conferences such as SIGGRAPH and the Computer-Human Interface (CHI) conferences, where many of the groundbreaking innovations of the past 60 years in human-computer interfaces, computer graphics, virtual reality, and augmented reality were shown to the world for the first time. The researchers would show live demos when possible, but otherwise would often present videos of their lab demos during their talks. These videos were not in digital format, but recorded on analog magnetic tapes in VHS format and played at low resolution with a videocassette recorder (VCR) on the cathode-ray tube of an analog TV. Catherine Plaisant and I took the task of digitizing these analog videos and preparing them to be viewable on the ACM Digital Library, as part of the Historical CHI Video Project.

Between class project obligations, trying to wrangle hundreds of copyright permissions, and digitizing all of the VHS tapes, we’re happy to announce that these pieces of history will soon be accessible to view through the ACM Digital Library. This is a massive treasure trove of 300+ demos which were originally presented at the annual ACM CHI conferences from 1983–2002, and they will soon be viewable online alongside their original papers. Some of the demos are already on Youtube, but we re-digitized them from their original tapes in higher quality for this project. We will announce once the videos are posted.

As a young graduate student working on the project, these demos were very eye-opening. My perspective has been severely limited by what I’ve seen released in a commercial sphere — we see Apple products released and touted as new and groundbreaking, when in reality such technology has been in development for years, even decades before it hits the market. I was amazed to see things like virtual reality and 3D modeling in such a sophisticated state back in the 80s and 90s.

VR has conceptually existed since the 1800s with the Stereoscope, which was a view-finding device that used two parallax views of a scene to create the illusion of depth.[1] The first iteration was created by Sir Charles Wheatstone in 1832, but stereoscopes are still in use today with devices such as the Google Cardboard.

A wooden stereoscope with eyeholes to look through points at a book with two similar images in parallax.
1861; Stereoscope by Oliver Wendell Holmes. This isn’t the first design for the stereoscope, but it was the most popular 19th century design.

Though the foundations of VR have been around since the 1800s, 3D modeling is essential for creating the fully immersive experiences we know today. The first demo for 3D modeling and CAD was from 1964 of Ivan Sutherland’s Sketchpad. We haven’t been able to acquire posting permissions from WGBH who produced the video, but you can view the demo below:

As one of the oldest videos we digitized (as part of CHI 1983), I was floored by how old 3D modeling and CAD is. There’s even a predecessor touch screen in use here, 4 years before the first computer mouse was unveiled by Doug Engelbart. The “touch screen” is actually a light pen, which was invented in 1952. The light pen system includes a photosensitive tip which senses the light changes of individual pixels, and then passes pen button input to the computer. [2]

Sutherland was also responsible for the first modern VR headset in 1968, “The Sword of Damocles,” a name poking fun at the slightly concerning metal contraption hanging over the user’s head. This machine detected head movement to explore a 3D model in a virtual space. The weight required the headset to be harnessed to the ceiling, so it definitely wasn’t in a commercial stage quite yet. [3]

Sutherland wears the headset; it is hooked from his head to a large crossbar on the ceiling.
Sutherland wears the headset; it is hooked from his head to a large crossbar on the ceiling.
1968; Sword of Damocles; Sutherland

In a CHI 1983 presentation, David Kasik of Boeing presented 3D modeling using the Tiger System, closer to what we see today. 3D rendering and modeling technology advanced tangentially to VR headsets, and created a more immersive experience in virtual reality spaces. In this case, 3D modeling is used as a tool for blueprinting Boeing airplanes and airplane parts. The demo can be viewed here: https://open-video.org/details.php?videoid=8003

A simple red 3D model of an airplane part with smooth shading.
A simple red 3D model of an airplane part with smooth shading.
1983; Tiger System Demonstration; Boeing, Kasik

The Oculus Rift was the first time virtual reality entered my consciousness as a viable medium. I come from a family of developers, so we managed to get a developer kit version back in 2013. At the time, there were only a few small games and apps available (Chicken Simulator was probably the most entertaining thing on the store, and I played it many more times than I should have), but it expanded my views on how similar games and apps could be developed. My only other data point for VR technology was the Nintendo Virtual Boy from 1995, which was considered a commercial and usability failure. My family rented one from Blockbuster when it was still around, and the slow frame rate gave us motion sickness. I thought that this was just the level of sophistication that could be achieved at the time.

A red headset with a stand to place on a table, and an attached game controller.
A red headset with a stand to place on a table, and an attached game controller.

I was very wrong. NASA presented something that looked and worked like the Oculus back in 1990 — the VIEW. Thanks to the DataGlove, the interactions are quite similar to the current Oculus Quest, with natural hand movement rather than hand-held controllers. The VIEW was used for simulations to train astronauts.[4] For now, you can see the VIEW demo on Youtube here: https://www.youtube.com/watch?v=TY8CyUQOncc

A woman wears the headset and DataGloves. The headset looks heavier and bulkier than modern ones.
A woman wears the headset and DataGloves. The headset looks heavier and bulkier than modern ones.
1990, VIEW: The Virtual Interface Environment Workstation, NASA Ames

I can imagine the usability issues I had with my 2013 Oculus Rift would be worse with this headset (motion sickness, low resolution display, eye strain, lagging head tracking, heavy weight) but seeing such a sophisticated product 23 years before getting to try it out myself is pretty inspiring.

Someone sitting in front of a computer wearing the dataglove, which is mirrored with a 3D model on the screen.
Someone sitting in front of a computer wearing the dataglove, which is mirrored with a 3D model on the screen.
1987; DataGlove; Thomas G. Zimmerman, Jaron Lanier, Chuck Blanchard, Steve Bryson, Young Harvill (VPL Research)

The DataGlove also has its own equally interesting demo posted to ACM Digital Library, from CHI 1987. It used fiber optic cables to determine the angle and position of fingers with relative accuracy, and interpreted hand gestures as controls. It allowed the user to grab and interact with objects in a 3D virtual environment. The demo can be viewed here:

A Nintendo product based on the DataGlove, called the “Power Glove” was later developed for commercial use by VPL Research and Mattel. You can see it mentioned briefly in the demo, but it is a simpler version of the DataGlove that interacts with 2D environments. The Power Glove was released as an accessory to the Nintendo Entertainment System (NES) but it didn’t see much commercial success due to difficulty of use and few games made specifically for the input system. [5]

The Power Glove has a complex-looking controller on the wrist and the Dataglove’s fiber optic cables are hidden.
The Power Glove has a complex-looking controller on the wrist and the Dataglove’s fiber optic cables are hidden.
1989 Version of the Power Glove made for the Nintendo Entertainment System

In “Be There Here” from the CHI 1992 Special Video program, Brenda Laurel presents important applications of VR technology, and features needed for VR to fully immerse and engage the senses. The segment emphasizes the importance of fast head tracking, binaural sound that adjusts according to head motion and position, and high resolution displays. Brenda Laurel is considered one of the pioneers of VR and founded the VR company Telepresence with Scott Fisher in 1988. The video can be viewed here:
https://vimeo.com/27346162

The image reads “You do not understand anything until you understand it in more than one way. Marvin Minsky.”
The image reads “You do not understand anything until you understand it in more than one way. Marvin Minsky.”
1992; Screenshot from the “Be There Here”; Telepresence Research, Brenda Laurel

As seen with the Virtual Boy and Power Glove, there were still quite a few usability concerns facing commercial attempts at VR in the 90’s. “Towards Usable VR” in 1999 by Robert Lindeman examines possible input devices that make VR interfaces easier to use. One of their tests involves paddle-like handheld windows, and a single finger tapping controller. The demo isn’t on Youtube, but can be downloaded here: https://open-video.org/details.php?videoid=4966

A person wearing a headset points a finger to a paddle-like interface, held in their other hand.
A person wearing a headset points a finger to a paddle-like interface, held in their other hand.
1999; Towards usable VR; Lindeman, R. W., Sibert, J. L., & Hahn, J. K.

We have a few videos in the collection showing exciting early developments in augmented reality as well. One such video is “Multiple-Computer User Interfaces: “Beyond the Desktop” Direct Manipulation Environments” from 2000. It looks quite similar to interactions in the MagicLeap headset, with users collaboratively instantiating magazine pictures of furniture into 3d spaces:

A catalog sits on a table with circled furniture; corresponding 3D models of the furniture are projected on the table.
A catalog sits on a table with circled furniture; corresponding 3D models of the furniture are projected on the table.
2000; Multiple-computer user interfaces; Rekimoto, J.

Virtual reality and augmented reality are advancing faster than ever, and are breaking ground as an even more usable medium — VR is now very inexpensive with the Google Cardboard, and many of us use facets of augmented reality every day with iPhone games and Instagram filters. Even more exciting are the applications to simulations and 3D design using more sophisticated headsets, such as the Microsoft HoloLens 2.

I highly recommend every HCI student check out these videos for a more well-rounded perspective of HCI history, especially those students new to the field. Our textbooks barely scratch the surface of where this tech started, and are pretty limited to commercially tested products. There’s a lot for everyone; data visualization, augmented reality, children’s UX, 3D modeling, collaboration, touchscreens, accessibility, and so much more. I hope these videos inspire the next generation of interaction engineers to push forward with experimental technology. Some of this work is just now hitting the market in a massive way, 20 or 30 years later.

References

  1. Hillis, K. (1999). Precursive Cultural and Material Technologies Informing Contemporary Virtual Reality. In Digital Sensations: Space, Identity, and Embodiment in Virtual Reality (pp. 30–59). University of Minnesota Press. Retrieved June 28, 2020, from www.jstor.org/stable/10.5749/j.cttts6mg.7
  2. Light pen. (n.d.). Retrieved June 26, 2020, from http://www.computerlanguage.com/results.php?definition=light+pen.
  3. Parisi, D. (2018). Interface 4.: Human–Machine Tactile Communication. In Archaeologies of Touch: Interfacing with Haptics from Electricity to Computing (pp. 213–264). Minneapolis; London: University of Minnesota Press. Retrieved June 28, 2020, from www.jstor.org/stable/10.5749/j.ctt20mvgvz.9
  4. Rosson, L. (2014, April 15). The Virtual Interface Environment Workstation (VIEW), 1990. Retrieved June 26, 2020, from https://www.nasa.gov/ames/spinoff/new_continent_of_ideas/
  5. Backwards Compatible — The Power Glove. (2008, May 19). Retrieved June 28, 2020, from http://www.abc.net.au/tv/goodgame/stories/s2248843.htm

Here are links to some relevant papers exploring 3D, animation, VR, and AR. Each of these will have their corresponding videos uploaded to ACM Digital Library. Videos from before 1991 currently do not have corresponding papers on ACM DL:

Sparks of Innovation: Stories from the HCIL

Research at the Human-Computer Interaction Laboratory at…