Loading…
0:00
14:12

In early April, MIT research assistant Arnav Kapur, 24, uploaded a short video on YouTube. The clip showed him moving around campus in various settings wearing a white plastic contraption wrapped around the right side of his face.

As he walked by rows of bikes parked next to mounds of melting snow, his lips were closed while his inner thoughts flashed as words on the screen. “Time?” it read. A male voice responded, “10:35 a.m.” In the next scene, Kapur was shopping in a bodega. The prices of the things he tossed into his shopping cart — toilet paper, an Italian wrap, canned peaches — appeared on the screen. “Total $10.07,” the male voice responded. In the final scene, Kapur moved a cursor around a video console, ostensibly with his mind.

Kapur came to MIT’s Media Lab from New Delhi in 2016 to build wearable devices that seamlessly integrate technology into our 24/7 experience. No more reaching for cellphones. No more staring at screens. No more eyes down. No more tuning out to plug in.

Improbably, AlterEgo, the soundless, voiceless, earbud-less device he’d been working on for the last two years had become adept enough at reading his thoughts that he could use it to order an Uber without saying a word.

“We wanted to capture interactions that are as close to thinking in your head as possible.”

In its current incarnation, Kapur’s device — developed in collaboration with his brother Shreyas (an MIT undergrad), a few fellow grad students in the Fluid Interfaces department, and lead A.I. guru Professor Pattie Maes — is a 3D-printed wearable device outfitted with electromagnetic sensors that hugs one side of your jaw and, via Bluetooth, connects you with your what Maes calls your computer brain — the Internet’s massive web of information most of us access via smartphones some 80 times a day.

It’s radical for the simple reason that it is noninvasive — no implants required — and can process silent human communication with an exceptionally high degree of accuracy. Eventually, Kapur promises, this contraption will be practically invisible to other people.


A few months after the video came out, Kapur sat down for an interview with Medium in a small fifth-floor Media Lab office, which he shares with other researchers. He’s clean-shaven, neatly dressed, and grad-student thin; his brown eyes alternating between sleepy and searingly intense — an impressive trick. Among the computer parts, books, and other detritus scattered around the room sits a pink ukulele. Not his, he says.

Kapur’s natural inclination is to talk long, but since his invention has been drawing media attention, he’s clearly been working on his soundbites. “I’m very passionate about A.I.,” he says. “I think the future of human society is about us collaborating with machines.”

Since the introduction of the smartphone, 2.5 billion people already turn to their computer brain when they need to drive somewhere or cook something or communicate with other humans or forget the capital of Missouri. Cognitive augmentation through technology has become central to daily life. Organic brain, computer brain. They’re already working together, says Kapur, just not as well as they could.

Because of the way our devices are designed, however, they distract us more than help us. To consult with the infinite world at our fingertips, we have to give our devices our full attention. Screens demand eye contact. Phones require earbuds. They pull us out of the physical world and into theirs.

Kapur wants to perfect a device that allows users to communicate with A.I. as effortlessly as one’s left brain talks to one’s right brain, so that humans can integrate the power of the Internet into their thinking at every level. Once the technology becomes a natural extension of the body, Kapur believes, we will be free to become better at being human.

“This is how we’re going to live our lives,” he says.

When conceptualizing AlterEgo, Kapur based his design guidelines on a few fixed principles. The device couldn’t be invasive because he considers that inconvenient and not scalable. Interacting with it had to feel natural as well as be invisible to others, so the device had to be able to pick up silent cues. Painfully aware of the ways tech can get co-opted, he also wanted user-control baked into the design so that the device would only detect volitional, rather than subconscious, signals. In other words, it should only read your thoughts when you want it to.

You must want to communicate with your computer brain in order to interact with it.

Other tech pioneers have developed human-to-computer conversational interfaces with some success, but there are always caveats. To interact with Siri and Alexa, one must talk directly to a machine, which feels unnatural and isn’t private. Hampering adoption of this technology is the creeping concern that we don’t know exactly who’s listening to what when these devices are around.

Kapur needed a new way around the problem. What if a computer could read our thoughts?


As a researcher who “dabbles across disciplines” (he’s tried and failed to write a short website bio because he doesn’t want to be “put in a box”), Kapur began to think of the human body not as a limitation but as a conduit. He saw the brain as the power source driving a complex electrical neural network that controls our thoughts and movements. When the brain wants to, say, move a finger, it sends an electrical impulse down the arm to the correct digit and the muscle responds accordingly. Sensors can pick up those electrical signals. One just needs to know where and how to tap in.

Kapur knew that when we read to ourselves, our inner articulatory muscles move, subconsciously forming the words we’re seeing. “When one speaks aloud, the brain sends electrical instructions to more than 100 muscles in your speech system,” he explains. Internal vocalization — what we do when we read silently to ourselves — is a highly attenuated version of this process, wherein only the inner speech muscles are neurologically triggered. We developed this habit when we were taught to read — sounding out letters then speaking each word aloud. It’s a habit that’s also a liability — speed-reading courses often focus on eliminating word formation as we scan a page of text.

First observed in the mid-19th century, this neurological signalling is the only known physical expression of a mental activity.

Kapur wondered whether sensors could detect the physical manifestations of this internal conversation — tiny electrical charges firing from the brain — on the skin of the face, even if the muscles involved were located deep in the mouth and throat. Even if they weren’t exactly moving.

The original design of AlterEgo’s armature pinned a grid of 30 sensors to a subject’s face and jaw so that they could pick up the neuromuscular action when the experimenter used his or her inner voice to communicate. Proprietary software was calibrated to analyze the signals and turn them into distinct words.

There was just one problem: In the beginning, AlterEgo’s sensors detected nothing.

Kapur had built the hardware and the software and hoped for the best. But the myoelectrical signals from this silent speech were very weak. It would have been easy to rethink the whole thing at that point. “But,” he says, “we wanted to capture interactions as close to thinking in your head as possible.”

Kapur moved the sensors to different regions of the face, increased their sensitivity, and reworked the software. Still nothing.

One night, Kapur and his brother were testing the device in their Cambridge apartment. Kapur was wearing the device and Shreyas was monitoring the computer screen. They’d rigged the device to track signals in real time so that Shreyas could note the exact moment it picked up something, if anything.

It was getting late. Kapur had been speaking silently into the device for a couple of hours — having programmed it to understand just two words: yes and no — without any meaningful results.

Then Shreyas thought he saw something. A blip on the screen.

“We didn’t believe it,” Kapur says. He turned his back on his brother and repeated the action. “We kept seeing one bump in the signal and thought it was some artifact in the wires. We were really sure this was some sort of noise in the system.”

Were they actually seeing something?

After testing and retesting for the next hour, Kapur was convinced that they’d made contact.

“That was a crazy moment,” he says. They celebrated with a pizza the next day.


It took Kapur and his collaborators two years to develop the hardware and software for AlterEgo, designing the device so that it could be worn with ease, refining its sensors and target locations to downsize the package into something less visually intrusive. Eschewing earbuds, which he believes disrupt normal human behavior, he developed an aural feedback system through bone conduction; the device whispers answers to queries like a genius guardian angel.

Once the device started picking up myoelectrical pulses, Kapur focused on developing a data set to train AlterEgo to recognize signal signatures for various words. It was a laborious process — someone had to sit in a lab wearing the device and silently speak specific words until the computer mastered them.

So far, AlterEgo has a vocabulary of 100 words, including numbers from 1 to 9, and commands like add, subtract, reply, call.

Because the YouTube video made it look like AlterEgo was reading Kapur’s mind, there was some public hand-wringing. “It’s really very scary that our thoughts are no longer private,” wrote one concerned commenter on an article about the technology. “Technology like this can be used by the real life Thought Police.”

Kapur and the A.I. expert Maes are exquisitely sensitive to ethics concerns like these. Kapur believes that as a creator, he can subvert nefarious uses by building safeguards into the design. Kapur insists that AlterEgo can’t actually read your mind and will never be able to do so. He very deliberately developed it to only respond to volitional signaling — conscious communication. You must want to communicate with your computer brain in order to interact with it. That differentiates AlterEgo from, say, Google Glass. The device doesn’t have a camera, for example, because Kapur doesn’t want his wearables collecting any more information than you expressly give them.

“A.I. itself isn’t bad, but we still have discussions about possible abuses of the technology,” he says. “So we try to build the tech to fit the principles that we developed. That’s why we designed AlterEgo from the ground-up — we’ve thought about it in a certain way from the beginning so that it will be used the way we designed it.”

Kapur, who has worked on a few projects with Harvard Medical School, is especially driven to use the technology to help those who are impaired. Alzheimer’s patients, outfitted with the device, could have a memory assist, for instance. And because it detects micro neural signals, it could help those who are physically challenged — deaf or mute people, stroke victims, or people with ALS, stutters, or autism — communicate with the world.

To really make AlterEgo really functional, Kapur still needs to program the thing to detect a vocabulary bigger than 100 words. And he’ll need enough data to ensure that the device works on a full range of different heads and silent speakers. That said, he thinks the existing technology is good enough that at some point, it will be able to synthesize information and extrapolate new words from their context.


In the modern, gleaming white Media Lab offices, it’s easy to get seduced by a vision of a bright sparkling future in which we fluidly think with both of our brains — the one we were born with and the computer to which we’ve willingly tethered ourselves.

Maes offers a litany of ways a fully integrated and seamless A.I. system could transform us if its software were designed to augment instead of amuse. Technology could help us realize our dreams, Maes says. (Her well-earned reputation as a techno-utopian guru is part of what draws ambitious students like Kapur to MIT.) AlterEgo could teach us foreign languages by fluidly guiding us through our environment in a non-native tongue. It could be a social lubricant, reminding us of people’s names and other key data points when we greet them.

“I think the future of human society is about us collaborating with machines.”

Then, as if on cue, she takes an unexpected sharp turn away from Kapur’s pure, A.I.-human mind-meld concept. If hooked up to biofeedback information such as heart rate, sweat biomarkers, and body temperature, she says, the device could anticipate behavior and send us subtle signals to encourage us to reach pre-programmed goals. It could detect when we’re nodding off at work and emit a stimulating peppermint scent. It could reprogram behavior, emitting the stench of rotten eggs when we reach for that third cupcake. It could detect when we’re anxious and deliver soothing messages, inaudible to others, to talk us down.

It’s a significantly different future — more monetizable, perhaps, and directed toward behavioral engineering — than the one envisioned by her student. If we could integrate A.I. and all the information on the web into our conscious thought, Maes seems to suggest, then we could finally shed 10 pounds. But maybe that’s really what we want after all.

It’s not hard to see how Kapur’s invention could, within a few years, become a billion-dollar idea, with implications for the defense industry and tech giants like Facebook and Amazon. Just who owns the intellectual property behind AlterEgo is less clear. Kapur talks around the question. He says that if he left MIT, he could take it with him. But that’s not his plan, for now. He’s driven to remain in an academic setting, tinkering away on an invention he believes will improve human life rather than simply selling out to the highest bidder. This is his baby, and he wants to see it through to the end.

But what if someone copies his tech, builds their own version, and creates the next unicorn tech company without him?

“I really don’t know how to answer that,” he says with a shrug, his expression placid, his thoughts unreadable.