Eighteen Years Later, a Reflection on User Interfaces in Minority Report
In Steven Spielberg’s Minority Report, the DC police force uses futuristic technology to apprehend murderers before they kill their victims. The department, known as the “PreCrime” unit, relies on special psychic humans, or “precogs”, to predict the crimes, but uses a complex citizen identification system for much of tracking and detective work. The officers interact with the system a number of ways, including a full body gestural interface, modular touch screen devices somewhat evocative of modern tablets, and even what appear to be smart watches. The movie is set in 2054, but was released in 2002. Eighteen years later, looking back on the technology, the flashy gestural interface still feels futuristic, but some of the other technologies come off a bit dated.
What enabling technologies must exist in that cinematic universe to enable the interactions you see? Could we build something similar today?
The signature interface in Minority Report is a full body gesture-based system. In the opening few minutes of the film, the protagonist, John Anderton, is shown standing in front of a large, transparent screen, gesturing with his arms to manipulate various images and videos on the screen. He wears sleek gloves with glowing fingertips, presumably so that the interface can track his movements. Anderton uses this system to sort through the psychic predictions of the precogs and cross reference with other sources to determine crucial details of the crime, such as the exact location of the murder or the identities of the victims.
As a 2020 viewer, one of the first questions you may have while watching is whether or not we could build this interface today. I think the answer to that question depends on how closely you wish to match the experience in the movie. I found a reasonably close imitation through another Medium article about the realities and drawbacks of the Minority Report interface. Back in 2008, MIT Media Labs used Oblong’s g-speak platform with a ceiling full of cameras and spotted gloves to create the gesture based interface shown in the video below.
But what if we want to ditch the gloves? A quick web search shows that there are tech companies developing devices to recognize hand gestures performed by bare hands. One of the leading products on the market appears to be the Leap Motion Controller, built by the bay area based company Ultraleap, which includes a small motion sensor and software development kit.
Of course, Google has also put their hat into the gesture recognition ring in the form of Project Soli. This product is a tiny chip which uses radar technology to track motions of hands on a sub-millimeter scale.
So, it seems the sensor technology to recognize hand gestures is available, or at least close. However, I can’t find any evidence of people using either of those devices to create a Minority Report inspired interface beyond the device demonstration videos.
Imagine interacting with that interface. What is challenging about it?
While the gesture based interface in the PreCrime headquarters is great for dramatic effect and futuristic flair, it may not be a realistic interface for crime work for reasons of time and energy efficiency. First of all, many of the gestures required for this interface involve broad, sweeping movements with both arms. While this is fine for the limited use shown in the movie, I suspect sustained use for more than a few minutes could be rather tiring.
The user wears what appear to be special gloves for tracking purposes, and these gloves have little glowing dots on the fingers, implying each finger can be tracked separately. A more practical gesture-based approach might be hand and finger gestures, such as pinching to zoom or pressing together different combinations of fingers to select items, but this provides less visual drama for the audience than the bigger, less energy efficient full arm gestures.
However, one could argue that the use case for the gestural interface is just the short spurts of time before the officers run off to stop the murder, and therefore energy use isn’t such a big deal. There are some depictions of desktop computers in the PreCrime headquarters, both touch screen and keyboard controlled, so perhaps those are the interface of choice for sustained use.
Still, according to the plot, the officers are often on a tight time crunch when solving murders, arriving only seconds before the crime occurs. The PreCrime unit uses the full body gestural interface to narrow down suspect and location details before heading out in the field. Many of the movements shown involve a full range of motion, such as flinging both arms across the body to dismiss a picture off the screen. I suspect it would be faster to, say, swipe an inch on a touch screen or press a nearby button on a panel than to fling both arms two feet through space, and in a situation where a second could be the difference between life and death, it seems foolish not to optimize for time.
Did technology develop in the way the filmmakers envisioned? If not, what’s different?
A minor piece of technology that catches my eye as a 2020 viewer are the watches the PreCrime officers wear. These watches count down the time until a murder occurs, and if you look closely, they also display the case number. This seems like the kind of functionality you might get from a modern smart watch, but the Minority Report watches are round, with lots of tick marks and the minute numbers engraved around the outside. This looks dated in comparison to today’s sleek, touch screen smart watch designs. Additionally, it appears that all the watches do is count down the murder; the officers are never shown interacting with the watches to bring up any other features. So while it was clever of the filmmakers to predict watches with increased functionality, in hindsight they could have dreamed bigger with their designs.
Another supposedly futuristic technology that already seems dated are the little transparent slides that seem to be used to store video. In one scene, John Anderton decides to watch some home videos of his son. He pulls out a case of little transparent slides, each about the size of a playing card, and selects one to insert into a slot in his entertainment system. In a time when most movies were played off of DVDs, or even VHS tapes, the move to a similar system with smaller, sleeker storage devices must have made sense. Now, in the era of cloud storage and streaming, it just looks silly.
Perhaps the biggest technological prediction miss is the lack of phones. I don’t recall seeing a single cell phone in the movie, much less the equivalent of a modern smartphone. While there are touch screen tablet type devices, they are larger than your average smartphone, more comparable to an iPad, and it does not appear that anyone ever takes them out of the office. In order to communicate with each other, the officers talk vaguely into their sleeves. If further information is needed on site, instead of pulling out a smartphone-type device to review footage, the officer in the field contacts someone in the office to look it up on a computer there. When is the last time you called someone to Google something for you? Mobile computation is an integral part of many people’s daily life in 2020, and it is remarkably absent from Minority Report’s depiction of 2054.
