Development of Early VR
As seen through the lens of historical CHI videos.
In 2018, the Association for Computing Machinery (ACM) decided to make its video archives available online. ACM is the organizer of influential conferences such as SIGGRAPH and the Computer-Human Interface (CHI) conferences, where many of the groundbreaking innovations of the past 60 years in human-computer interfaces, computer graphics, virtual reality, and augmented reality were shown to the world for the first time. The researchers would show live demos when possible, but otherwise would often present videos of their lab demos during their talks. These videos were not in digital format, but recorded on analog magnetic tapes in VHS format and played at low resolution with a videocassette recorder (VCR) on the cathode-ray tube of an analog TV. Catherine Plaisant and I took the task of digitizing these analog videos and preparing them to be viewable on the ACM Digital Library, as part of the Historical CHI Video Project.
Between class project obligations, trying to wrangle hundreds of copyright permissions, and digitizing all of the VHS tapes, we’re happy to announce that these pieces of history will soon be accessible to view through the ACM Digital Library. This is a massive treasure trove of 300+ demos which were originally presented at the annual ACM CHI conferences from 1983–2002, and they will soon be viewable online alongside their original papers. Some of the demos are already on Youtube, but we re-digitized them from their original tapes in higher quality for this project. We will announce once the videos are posted.
As a young graduate student working on the project, these demos were very eye-opening. My perspective has been severely limited by what I’ve seen released in a commercial sphere — we see Apple products released and touted as new and groundbreaking, when in reality such technology has been in development for years, even decades before it hits the market. I was amazed to see things like virtual reality and 3D modeling in such a sophisticated state back in the 80s and 90s.
VR has conceptually existed since the 1800s with the Stereoscope, which was a view-finding device that used two parallax views of a scene to create the illusion of depth.[1] The first iteration was created by Sir Charles Wheatstone in 1832, but stereoscopes are still in use today with devices such as the Google Cardboard.
Though the foundations of VR have been around since the 1800s, 3D modeling is essential for creating the fully immersive experiences we know today. The first demo for 3D modeling and CAD was from 1964 of Ivan Sutherland’s Sketchpad. We haven’t been able to acquire posting permissions from WGBH who produced the video, but you can view the demo below:
As one of the oldest videos we digitized (as part of CHI 1983), I was floored by how old 3D modeling and CAD is. There’s even a predecessor touch screen in use here, 4 years before the first computer mouse was unveiled by Doug Engelbart. The “touch screen” is actually a light pen, which was invented in 1952. The light pen system includes a photosensitive tip which senses the light changes of individual pixels, and then passes pen button input to the computer. [2]
Sutherland was also responsible for the first modern VR headset in 1968, “The Sword of Damocles,” a name poking fun at the slightly concerning metal contraption hanging over the user’s head. This machine detected head movement to explore a 3D model in a virtual space. The weight required the headset to be harnessed to the ceiling, so it definitely wasn’t in a commercial stage quite yet. [3]
In a CHI 1983 presentation, David Kasik of Boeing presented 3D modeling using the Tiger System, closer to what we see today. 3D rendering and modeling technology advanced tangentially to VR headsets, and created a more immersive experience in virtual reality spaces. In this case, 3D modeling is used as a tool for blueprinting Boeing airplanes and airplane parts. The demo can be viewed here: https://open-video.org/details.php?videoid=8003
The Oculus Rift was the first time virtual reality entered my consciousness as a viable medium. I come from a family of developers, so we managed to get a developer kit version back in 2013. At the time, there were only a few small games and apps available (Chicken Simulator was probably the most entertaining thing on the store, and I played it many more times than I should have), but it expanded my views on how similar games and apps could be developed. My only other data point for VR technology was the Nintendo Virtual Boy from 1995, which was considered a commercial and usability failure. My family rented one from Blockbuster when it was still around, and the slow frame rate gave us motion sickness. I thought that this was just the level of sophistication that could be achieved at the time.
I was very wrong. NASA presented something that looked and worked like the Oculus back in 1990 — the VIEW. Thanks to the DataGlove, the interactions are quite similar to the current Oculus Quest, with natural hand movement rather than hand-held controllers. The VIEW was used for simulations to train astronauts.[4] For now, you can see the VIEW demo on Youtube here: https://www.youtube.com/watch?v=TY8CyUQOncc
I can imagine the usability issues I had with my 2013 Oculus Rift would be worse with this headset (motion sickness, low resolution display, eye strain, lagging head tracking, heavy weight) but seeing such a sophisticated product 23 years before getting to try it out myself is pretty inspiring.
The DataGlove also has its own equally interesting demo posted to ACM Digital Library, from CHI 1987. It used fiber optic cables to determine the angle and position of fingers with relative accuracy, and interpreted hand gestures as controls. It allowed the user to grab and interact with objects in a 3D virtual environment. The demo can be viewed here:
A Nintendo product based on the DataGlove, called the “Power Glove” was later developed for commercial use by VPL Research and Mattel. You can see it mentioned briefly in the demo, but it is a simpler version of the DataGlove that interacts with 2D environments. The Power Glove was released as an accessory to the Nintendo Entertainment System (NES) but it didn’t see much commercial success due to difficulty of use and few games made specifically for the input system. [5]
In “Be There Here” from the CHI 1992 Special Video program, Brenda Laurel presents important applications of VR technology, and features needed for VR to fully immerse and engage the senses. The segment emphasizes the importance of fast head tracking, binaural sound that adjusts according to head motion and position, and high resolution displays. Brenda Laurel is considered one of the pioneers of VR and founded the VR company Telepresence with Scott Fisher in 1988. The video can be viewed here:
https://vimeo.com/27346162
As seen with the Virtual Boy and Power Glove, there were still quite a few usability concerns facing commercial attempts at VR in the 90’s. “Towards Usable VR” in 1999 by Robert Lindeman examines possible input devices that make VR interfaces easier to use. One of their tests involves paddle-like handheld windows, and a single finger tapping controller. The demo isn’t on Youtube, but can be downloaded here: https://open-video.org/details.php?videoid=4966
We have a few videos in the collection showing exciting early developments in augmented reality as well. One such video is “Multiple-Computer User Interfaces: “Beyond the Desktop” Direct Manipulation Environments” from 2000. It looks quite similar to interactions in the MagicLeap headset, with users collaboratively instantiating magazine pictures of furniture into 3d spaces:
Virtual reality and augmented reality are advancing faster than ever, and are breaking ground as an even more usable medium — VR is now very inexpensive with the Google Cardboard, and many of us use facets of augmented reality every day with iPhone games and Instagram filters. Even more exciting are the applications to simulations and 3D design using more sophisticated headsets, such as the Microsoft HoloLens 2.
I highly recommend every HCI student check out these videos for a more well-rounded perspective of HCI history, especially those students new to the field. Our textbooks barely scratch the surface of where this tech started, and are pretty limited to commercially tested products. There’s a lot for everyone; data visualization, augmented reality, children’s UX, 3D modeling, collaboration, touchscreens, accessibility, and so much more. I hope these videos inspire the next generation of interaction engineers to push forward with experimental technology. Some of this work is just now hitting the market in a massive way, 20 or 30 years later.
References
- Hillis, K. (1999). Precursive Cultural and Material Technologies Informing Contemporary Virtual Reality. In Digital Sensations: Space, Identity, and Embodiment in Virtual Reality (pp. 30–59). University of Minnesota Press. Retrieved June 28, 2020, from www.jstor.org/stable/10.5749/j.cttts6mg.7
- Light pen. (n.d.). Retrieved June 26, 2020, from http://www.computerlanguage.com/results.php?definition=light+pen.
- Parisi, D. (2018). Interface 4.: Human–Machine Tactile Communication. In Archaeologies of Touch: Interfacing with Haptics from Electricity to Computing (pp. 213–264). Minneapolis; London: University of Minnesota Press. Retrieved June 28, 2020, from www.jstor.org/stable/10.5749/j.ctt20mvgvz.9
- Rosson, L. (2014, April 15). The Virtual Interface Environment Workstation (VIEW), 1990. Retrieved June 26, 2020, from https://www.nasa.gov/ames/spinoff/new_continent_of_ideas/
- Backwards Compatible — The Power Glove. (2008, May 19). Retrieved June 28, 2020, from http://www.abc.net.au/tv/goodgame/stories/s2248843.htm
Here are links to some relevant papers exploring 3D, animation, VR, and AR. Each of these will have their corresponding videos uploaded to ACM Digital Library. Videos from before 1991 currently do not have corresponding papers on ACM DL:
- Towards Usable VR: an empirical study of user interfaces for immersive virtual environments
- Multiple-computer user interfaces: “beyond the desktop” direct manipulation environments
- Exploring 3D navigation: combining speed-coupled flying with orbiting
- Navigation methods for an augmented reality system
- Welbo: an embodied conversational agent living in mixed reality space
- GeoSCAPE: designing a reconstructive tool for field archaeological excavation
- Interaction in a collaborative augmented reality environment
- Information visualization using 3D interactive animation
- Rapid controlled movement through virtual 3D workspaces (videotape)
- LyberWorld: a 3D graphical user interface for fulltext retrieval
- TIME: three-dimensional input, modification and evaluation
- Improvisational animation
- The Rockin’Mouse: integral 3D manipulation on a plane
- The cubic mouse: a new device for three-dimensional input
- The Task Gallery: a 3D window manager
- Illuminating clay: a 3-D tangible interface for landscape analysis
- Groupspace: a 3D workspace supporting user awareness
- Creating principal 3D curves with digital tape drawing
