Privacy in the Metaverse might be Impossible (new research study)
A new paper from the University of California Berkeley reveals that privacy may be impossible in the metaverse without innovative new safeguards to protect users. Led by graduate researcher Vivek Nair, the recently released study was conducted at the Center on Responsible Decentralized Intelligence (RDI) and involved the largest dataset of user interactions in virtual reality that has ever been analyzed for privacy risks. What makes the results so surprising is how little data is needed to uniquely identify users , potentially eliminating any chance of anonymity in virtual worlds.
As background, most researchers and policymakers who study metaverse privacy, focus on the many cameras and microphones in modern VR headsets that capture detailed information about the user’s facial features, vocal qualities and eye motions, along with ambient information about the user’s home or office. Some researchers even worry about emerging technologies like EEG sensors that can detect unique brain activity through the scalp. While these rich data-streams pose serious privacy risks in the metaverse, turning them all off may not provide anonymity.
That’s because the most basic data-stream needed to interact with a virtual world, simple motion data, may be all that’s required to uniquely identify a user within a large population. And by “simple motion data,” I mean the three most basic data-points tracked by virtual reality systems — one point on the user’s head and one on each hand. Researchers often refer to this as “telemetry data” and it represents the minimal dataset required to allow a user to interact naturally in a virtual environment.
This brings me to the new Berkeley study entitled “Unique Identification of 50,000+ Virtual Reality Users from Head & Hand Motion Data.” The research, which analyzed over 2.5 million VR data recordings (fully anonymized) from over 50,000 players of the popular Beat Saber app, found that individual users could be uniquely identified with over 94% accuracy using only 100 seconds of motion data. Even more surprising was that half of all users could be uniquely identified with only 2 seconds of motion data. Achieving this level of accuracy required innovative AI techniques, but again, the data used was extremely sparse — just three spatial points for each user tracked over time.
In other words, any time a user puts on a mixed reality headset, grabs the hand controllers, and starts interacting in a virtual or augmented world, they are leaving behind a trail of digital fingerprints that can uniquely identify them. Of course, this begs the question — how do these digital fingerprints compare to real-world fingerprints in their ability to identify users?
If you ask people on the street, they’ll tell you that no two fingerprints in the world are the same. This may or may not be true, but honestly it doesn’t matter. What’s important is how accurately you can identify an individual from a fingerprint that was left at a crime scene or input to a finger scanner. It turns out that fingerprints, whether lifted from a physical location or captured by the scanner on your phone, are not as uniquely identifiable as most people assume.
Let’s consider the act of pressing your finger to a scanner. According to the National Institute of Standards and Technology (NIST) the desired benchmark for fingerprint scanners is a unique matching with an accuracy of 1 out of 100,000 people. That said, real-world testing by NIST and others have found that true accuracy of most fingerprint devices may be less than 1 out of 1,500. Still, that makes it extremely unlikely that a criminal who steals your phone will be able to use their finger to gain access.
On the other hand, the Berkeley study suggests that when a VR user swings a virtual saber at an object flying towards them, the motion data they leave behind may be more uniquely identifiable than their actual real-world fingerprint. This poses a very serious privacy risk as it potentially eliminates anonymity in the metaverse. In addition, this same motion data can be used to accurately infer a number of specific personal characteristics about the user, including their height, handedness, and gender. And when combined with other data that’s commonly tracked in virtual and augmented environments, this motion-based fingerprinting method is likely to yield even more accurate identifications.
I asked Nair to comment on my comparison above between traditional fingerprint accuracy and the use of motion data as “digital fingerprints” in virtual and augmented environments. He described the danger this way:
“Moving around in a virtual world while streaming basic motion data would be like browsing the internet while sharing your fingerprints with every website you visit. However, unlike web-browsing, which does not require anyone to share their fingerprints, the streaming of motion data is a fundamental part of how the metaverse currently works.” — Vivek Nair
To give you a sense of how insidious motion-based fingerprinting could be, consider the metaverse of the near future — a time when users routinely go shopping in virtual and augmented worlds. Whether browsing products in a virtual store or visualizing how new furniture might look in their real apartment using mixed reality eyewear, users are likely to perform common physical motions such as grabbing virtual objects off virtual shelves or taking a few steps back to get a good look at a piece of virtual furniture. The Berkeley study suggests that these common motions could be as unique to each of us as fingerprints. If that’s the case, these “motion prints,” as we might call them, would mean that casual shoppers wouldn’t be able to visit a virtual store without being uniquely identifiable.
So, how do we solve this inherent privacy problem?
One approach is to obscure the motion data before it is streamed from the user’s hardware to any external servers. Unfortunately, obscuring data means introducing noise. This could protect privacy but it would also reduce the precision of dexterous physical motions, thereby compromising user performance in Beat Saber or any other application that requires physical skill. For many users, it may not be worth the tradeoff.
An alternate approach is to enact sensible regulation that would prevent metaverse platforms from storing and analyzing human motion data over time. Such regulation would help protect the public but is difficult to enforce and could face pushback from the industry. For these reasons, researchers at Berkeley are exploring sophisticated defensive techniques that they hope will obscure the unique characteristics of physical motions without degrading dexterity in virtual and augmented worlds.
As an outspoken advocate for consumer protections in the metaverse, I strongly the field to explore all approaches in parallel, including both technical and policy solutions. Protecting personal privacy is not just important for users, it’s important for the industry at large. After all, if users don’t feel safe in the metaverse, they may be reluctant to make virtual and augmented environments a significant part of their digital lives.
Dr. Louis Rosenberg is an early pioneer of virtual and augmented reality. His work began over thirty years ago in VR labs at Stanford and NASA. In 1992 he developed the first functional mixed reality system at Air Force Research Laboratory. In 1993 he founded the early VR company Immersion Corp which he brought public on NASDAQ in 1999. In 2004 he founded the early AR company Outland Research. He received his PhD from Stanford University, was a tenured professor at California State University, and has been awarded over 300 patents for VR, AR, and AI technologies.
