Researchers have created an AI that can convert brain activity into text
The machine-learning algorithm could eventually help patients in communication who are unable to speak or type
We might still be far off from the time when computers can actually read our minds as we saw in the Hollywood Scifi thriller “Transcendence,” but we have already taken the first steps in that direction. Elon Musk started a company called Neuralink (2016), which is working on a long-term goal of developing a brain-computer interface (BCI) called Neural Lace.
The company recently announced the next phase where fine threads, thinner than the human hair will be implanted in a human brain to detect the activity of neurons. Neuralink plans to start human trials in the second quarter of 2020. While this process is invasive in nature, scientists have been working on parallel models where brain activity could be read by non-invasive means.
Taking this a step further, researchers from the University of California, San Francisco have developed an AI system that could provide the basis of a speech prosthesis eventually. The model involved recruiting four participants with electrode arrays implanted in their brains to monitor epileptic seizures.
“We are not there yet but we think this could be the basis of a speech prosthesis.” ~ Dr. Joseph Makin, co-author
They were then asked to read aloud a set of 50 sentences while the team of researchers tracked their neural activity during this process. The data collected henceforth was fed to a machine-learning algorithm. This AI system then converted the resulting brain activity into a string of numbers. The second part of the system converted these numbers into a sequence of words.
Initially, the system generated nonsense sentences. As it trained by comparing each sequence of words with the sentences that were actually read aloud it improved, making sense of how the string of numbers generated related to words that could be understood, the results improved.
Eventually, the AI was tested for generating text directly from brain activity during speech. The system is far from perfect by making some whimsical errors like “Those musicians harmonize marvelously” was decoded as “The spinach was a famous singer”, and “A roll of wire lay near the wall” became “Will robin wear a yellow lily.”
Nevertheless, the overall accuracy of the AI system was much higher than the previous approaches. On average, 3% of each sentence needed correction, which is still better than the 5% error rate recorded for a professional human transcriber. For comparative purposes though, the machine learning algorithm was handling a smaller amount of sentences.
According to Dr. Christian Herff, who is an expert in the field from Maastricht University, the presented research is promising and impressive, since the accuracy of results produced came from less than 40 minutes of training data for each participant.
This is a very basic model and it might not be usable for severely disabled patients in its current state since it requires the people to be reading loudly to record their brain activity and subsequently translating it into readable text. But its certainly an important step towards helping people with speech inhibitions to communicate.
Complete Research was published in the journal Nature Neuroscience.