The Future of Mind-Controlled Artificial Limbs
How hard was it to open this article? Hopefully, not very: you saw the article, formed an intention to read it, half-consciously guided your hand to click the link, and here you are. Yet such a seemingly simple sequence of events is in fact quite complex. If you are using a computer, you needed precise control of your arm, wrist, and fingers to settle the cursor above the link. If you are reading this on a mobile device, think of the exact physiological processes that let you single-handedly scroll and click while maintaining the screen’s orientation and avoiding dropping your phone or launching it across the room. Such actions require us to have fine control of each digit of our hands, a detailed mental model of our hand in its environment, and a quick and accurate sensory-feedback loop.
For people with amputations or neurological disorders, many such common actions are not possible. Ideally, artificial limbs would fill in, allowing people to maintain their autonomy as they go about their day, but historically prosthetic limbs have been minimally functional. For years, robotics technology progressed with limited application to prosthetic limbs; even a fully functional robotic hand is of little use to an amputee without the proper communication channel between user and robot. The development of brain-computer interfaces (BCIs) provides the missing link, allowing humans to control robotic artificial limbs using a sensory-feedback cycle that mimics that of the nervous system.
The premise is relatively straightforward: take whatever is biologically missing and replace it with electronics. Is your brain unable to communicate with a limb due to injuries of the central nervous system? Insert electrodes in the brain and the limb to create an electronic communication channel. Are you missing a hand? Create a robotic hand, attach it to your forearm, and wire it into your forearm’s nerves and muscles. BCI technology can also augment human abilities beyond those of our natural bodies. Within our lifetimes, we may be able to remotely operate robots and computer systems with our minds¹. While BCIs could be highly impactful, substantial technical and ethical challenges stand in the way of widespread adoption².
Non-invasive vs. Invasive BCIs
BCIs include structural pieces, electrodes (things that measure or produce electricity), wires, and the computer hardware and software elements that interpret and relay neural signals. The specific technologies used in a BCI depend on the needs of the user. One key consideration is between invasive approaches, in which electrodes and structural pieces are surgically inserted, and non-invasive approaches, in which the entire BCI is removable and sits outside of the skin. Figure 1 shows examples of non-invasive and invasive electrode placements.
Non-invasive BCIs
Non-invasive BCIs have some clear advantages: they are cheaper to install and generally safer, and they can be removed, repaired, or upgraded relatively easily. The most common form of non-invasive BCI is the electroencephalogram (EEG), which sits on the head and measures brain waves, as shown in figure 2. EEGs effectively record summary statistics about brain activity. These statistics provide insight into various mental activities: For example, Neurable, a BCI startup, claims they will soon be shipping EEG-containing headphones that can analyze one’s attention levels throughout the day³.
New research has pointed towards the possibility that EEG data could control artificial limbs. A March 2021 study which equipped participants with EEG headsets found that a majority of participants could learn to send consistent motor signals to a computer within 30 minutes, leveraging a coadaptive learning program in which the digital interpreter of brain signals “learned,” or updated its model’s parameters, at the same time as did the human participant⁴. While those participants only manipulated simple computer simulations, other researchers have observed patients using EEGs to control physical robotic arms for movement and grasping tasks⁵.
There is, however, a catch. Non-invasive BCIs are hampered by their low quality of acquired signals and requirement for frequent calibration¹. To train participants to grasp objects using EEG-controlled arms, researchers were required to separate the task into two movements, each of which required exerting control in only two degrees of freedom⁵. For contrast, the simplest model of the human hand has 23 degrees of freedom, as seen in figure 3. In practice, state-of-the-art non-invasive BCIs cannot produce the highly dimensional and sensitive movement data required to articulate a fully functional robotic limb.
Invasive BCIs
Despite the complications that arise from surgery, invasive BCIs are generally more effective for motor control. Invasive BCIs can provide higher-fidelity data transfer by interfacing directly with muscles, nerves or brain tissue. Furthermore, invasive BCIs can send sensory data from the computer into the nervous system, while non-invasive systems must rely on other sensory pathways (like eyesight) for feedback from artificial limbs.
Electrodes used by invasive BCIs differ based on the purpose and location of the interface. Muscle electrodes can detect intentional muscle movements, allowing for control of artificial limbs⁷. Muscle electrodes are also often used as a means to control biological limbs through electrical stimulation, for example, by paralyzed people with otherwise functional limbs¹. Other endpoints for invasive BCIs include cuff electrodes which surround a nerve, intraneural electrodes which are inserted into a nerve, and electrode arrays implanted into the surface of the brain¹. The following section of this report provides some real-world use cases of various invasive electrode interfaces, organized by location.
Interface Locations
Location is an important variable for the BCIs that control artificial limbs. Naturally, electrochemical motor signals travel across a long network: signals start in the brain, make their way through the spinal cord, and then enter the peripheral nervous system which extends throughout the body. Peripheral nerves stimulate muscles to move and relay sensations back to the spinal cord. Many locations along this chain of control and feedback are viable sites for BCIs.
The location of a BCI will often depend on the integrity of a patient’s nervous system. If a patient can relay signals all the way to their muscles, as can many amputees, then a BCI may be integrated with the peripheral nervous system at the site of amputation. Interfaces with the peripheral nervous system can offer higher certainty that detected signals were intentionally directed towards the interface. If a patient’s nervous system cannot access the relevant peripheral nerves, as with quadriplegic people, then a BCI will need to intervene higher in the chain of command, either in the spinal cord or the brain.
Interfaces with the peripheral nervous system
A study in the New England Journal of Medicine followed four transhumeral amputees who were equipped with artificial arms for three to seven years⁷. Each arm was attached to the users humerus for structural support and connected to the user’s peripheral nervous system through a combination of muscle electrodes and cuff electrodes, as shown in figure 4. The prosthetic arms each included an elbow joint and a hand, and each hand had individually controllable fingers with sense pads to provide sensory feedback. By interfacing directly with muscles and nerves that would otherwise control arms movement, the prosthetics required no formal training. Rather, users learned to control the arm gradually over the course of weeks.
Results were very promising. Each of the four participants learned to use their new limb and integrated it into their lives in various ways: One participant regained full-time employment with the help of the limb, one started skiing, one took up rally-car racing. Every patient reported that they identified with the hand, and could feel sensation coming from the hand, which they described as resembling the touch of the tip of a pen. Patients could perceive increases and decreases in intensity of tactile sensation. All of these results point to the promise of interfaces with the peripheral nervous system for control of artificial limbs. However, the amount of control and sensory feedback was still limited by the BCI, meaning that the patients could not recover complex tasks like handwriting.
Researchers from the University of Michigan are working to create BCIs similar to those used in the study above but with higher complexity of incoming and outgoing signals⁸. Their approach involves grafting small amounts of muscle tissue onto the ends of nerves, creating a more manageable site for electrode placement. This approach allows for significantly higher signal-to-noise ratios, which is an important step towards high-bandwidth interfaces. The researchers report that they can pick up on thumb movements with many degrees of freedom, as well as individual finger movements with high accuracy. Figure 5 shows a study participant demonstrating his prosthetic’s dexterity by pulling a zipper. These promising findings, along with the success of long-term trials for other peripheral-nervous-system interfaces point towards a future where amputees can regain full function of their bodies.
Interfaces with the brain surface
While all BCIs ultimately interface with the brain, some interface directly with the brain’s outer surface, or the cortex. Most cortical BCIs fall into one of two categories: intracortical microelectrodes, which penetrate the brain tissue, and electrocorticography (ECoG) systems, which rest on the outside of the cortex. ECoG and intracortical microelectrode systems have many advantages over non-invasive EEGs, including their ability to potentially pick up on many control signals at once, and with higher accuracy⁹.
Somewhat predictably, implanting electrodes into the brain comes with many challenges. One problem is durability: many cortical BCIs lose signal quality over the course of six months to a year². Implanting cortical BCIs may also pose a significant health risk to the patient.
We don’t have good data on the safety of cortical BCIs, though we can make assumptions based on research of related implants, like deep-brain stimulation (DBS) devices, used to treat Parkinson’s disease. A review from The Journal of Neuromodulation found that 19.04% of DBS patients experienced complications, with infection being the most likely at 4.57%¹⁶. DBS devices penetrate much deeper into the brain than cortical BCIs, stimulating areas involved with basic functions like metabolism and breathing, whereas cortical BCIs interface with the outer layer of the brain responsible for higher-order cognition. So, cortical BCIs are probably less likely to cause severe complications, but could pose a risk to the patient’s cognitive capacities.
The Journal of Neuromodulation review also investigated the Utah Array, a cortical BCI commonly used in research, finding few reported complications among patients, but also few long-term trials. New implantable flexible nerve electrodes (IFNEs) may be even safer than existing rigid options like the Utah Array¹⁷. When under mechanical stress, IFNEs may be less likely to break and less likely to harm the tissue surrounding them. Despite this promising advance in electrode tech, the safety of cortical BCIs remains largely unknown, and is a key concern based on the known dangers of related DBS implants.
In general, long-term human trials of cortically controlled artificial limbs have not been performed because of the limitations of the technology and the dangers involved. Many cortical BCIs are being tested on animals instead. Studies have shown that some ECoGs can last for at least six months in a monkey’s brain without losing functionality or damaging the animal’s health¹². Neuralink, a startup creating a high-bandwidth cortical BCI, is testing their BCI on pigs and monkeys¹¹. Figure 6 shows a monkey controlling a virtual paddle in the computer game Pong using a cortical BCI. Neuralink hopes to soon begin human trials, where their interface could control artificial limbs, mobile phones, or computers.
Some patients have used ECoG-controlled artificial limbs in short-term trials. In a study published in the Journal of Neurosurgery, researchers inserted ECoG interfaces into patients who had previously suffered strokes, then asked those patients to try producing various hand movements in a set training schedule¹⁰. Using the data collected through open-loop user training, computer models predicted hand movements with relatively high accuracy, allowing for real-time control of a prosthetic hand. The study was short-term and did not involve any feedback from sensors in the hand, though cortical BCIs can provide such feedback. Related cortical BCIs may eventually become a viable option for limb control, but for now they lag behind peripheral BCIs in durability and safety.
Ethical Issues
For all of the reasons BCIs are exciting, they are also ethically tricky. A BCI is, in a sense, a purposeful blurring of the boundaries of a user’s body and brain. Any moral or legal considerations that apply to the actions of a person must be reinterpreted to include or exclude the actions performed by their BCI-controlled systems, much in the same way that traffic laws must be redefined to account for the actions of autonomous vehicles. Other ethical issues arise from the ways a BCI could impact a user’s identity.
Guglielmo Tamburrini, a Professor of Philosophy and Science at Universita’ di Napoli “Federico II,” published an article in the journal Neuroethics outlining some pressing ethical quandaries for a future with commercial BCI use¹³. Many of his questions are relevant to artificial limbs, including:
Who is responsible for damages caused by a brain-actuated mobile robot?
Is human dignity jeopardized by the use of unconscious or pre-conscious brain information processing in BCI-enabled, human-machine co- operative problem solving?
Does motor and mental enhancement by brain- robot networking affect user personality and threaten personal identity persistence?
Existing laws related to damages caused by technology may apply to damages caused by brain-controlled limbs, though the fact that most BCIs learn from their user over time complicates question of responsibility. The programmer and manufacturer of a BCI may have little ability to predict the performance of the artificial limb. Is a user responsible for the way in which they “taught” their new arm? Tamburrini points to existing laws that place liability on individuals even when another agent perpetrates the damage. For example, an employer may be liable for the actions of an employee who works on their behalf¹³. In the same way, a user of a BCI-controlled artificial limb could be responsible for any damages caused by their limb, since the limb, which is guided by the user’s intentions, can be thought of as acting on the user’s behalf. This responsibility may give pause to prospective users of BCI-controlled limbs, especially given the state of technology, which still only allows for a level of control which is far from that of biological limbs.
The ethical issues with brain-controlled artificial limbs are small when compared to other ethical issues that could arise from advances in BCIs. For example, could someone with locked-in syndrome modify their last will using a BCI¹³? When we build high-bandwidth connections between computers and our brains, what entities will be allowed to access our brains? Will search engines and social networks send data straight to our minds, and to what extent will we perceive these data as external, as opposed to part of our own thoughts? Despite these issues, some people argue that enhanced human-computer networking is vital to the wellbeing of our species. Elon Musk, CEO of Neuralink, believes that we must increase our ability to communicate directly with machines in order exist symbiotically with generally intelligent computers¹¹. In his view, with widespread use of BCIs, artificially intelligent machines could grow alongside humanity with shared communication channels ensuring greater empathy and alignment of interests.
Conclusion
Fine motor control remains elusive for human-controlled artificial limbs. However, interfaces with the peripheral nervous system can allow users to regain some normal functionality, as well as greater satisfaction and self-esteem⁷. Interfaces with the central nervous system show promise for enabling mind-controlled artificial limbs as well, and though interfacing directly with the brain comes with significant challenges, advances in cortical BCI technology could enable a wider variety of therapies, including more accurate classification of mental states, treatments of mental disorders, and enhanced cognition. With the advanced state of current robotic limbs, the bottleneck of processing neural signals is the only obstacle preventing us from implementing fully functional artificial limbs. Hopefully, continued research efforts across invasive, non-invasive, cortical and peripheral BCIs will soon remove that bottleneck.
Sources
(1) Navarro, X., Krueger, T. B., Lago, N., Micera, S., Stieglitz, T., & Dario, P. (2005). A critical review of interfaces with the peripheral nervous system for the control of neuroprostheses and hybrid bionic systems. Journal of the Peripheral Nervous System, 10(3), 229–258. doi:10.1111/j.1085–9489.2005.10303.x
(2) Ryu, S. I., & Shenoy, K. V. (2009). Human cortical prostheses: Lost in translation? Neurosurgical Focus, 27(1) doi:10.3171/2009.4.FOCUS0987
(3) Neurable (2021, April 23). Neurable: Focus on what matters most [Video file]. Retrieved April 29, 2021, from https://www.youtube.com/watch?v=Rae6NzMWTRo
(4) Cunha, J. D., Perdikis, S., Halder, S., & Scherer, R. (2021). Post-adaptation effects in a motor imagery brain-computer interface online coadaptive paradigm. IEEE Access: Practical Innovations, Open Solutions, 9, 41688–41703.
(5) Meng, J., Zhang, S., Bekyo, A. et al. Noninvasive Electroencephalogram Based Control of a Robotic Arm for Reach and Grasp Tasks. Sci Rep 6, 38565 (2016). https://doi.org/10.1038/srep38565
(6) GESTO: A Glove for Enhanced Sensing and Touching Based on Inertial and Magnetic Sensors for Hand Tracking and Cutaneous Feedback — Scientific Figure on ResearchGate. Available from: https://www.researchgate.net/figure/A-simplified-kinematic-structure-of-the-human-hand-having-23-Degrees-of-Freedom-DoFs-4_fig2_318429361 [accessed 29 Apr, 2021]
(7) OrtizCatalan, M., Mastinu, E., Sassu, P., Aszmann, O., & Brånemark, R. (2020). Self-contained neuromusculoskeletal arm prostheses. New England Journal of Medicine, 382(18), 1732–1738. doi:10.1056/NEJMoa1917537
(8) ‘It’s like you have a hand again’: An ultra-precise mind-controlled prosthetic. (2020, March 10). Retrieved May 06, 2021, from https://news.umich.edu/its-like-you-have-a-hand-again-an-ultra-precise-mind-controlled-prosthetic/
(9) Wilson, J. A., Felton, E. A., Garell, P. C., Schalk, G., & Williams, J. C. (2006). ECoG factors underlying multimodal control of a brain-computer interface. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 14(2), 246–250. doi:10.1109/TNSRE.2006.875570
(10) Yanagisawa, T., Hirata, M., Saitoh, Y., Goto, T., Kishima, H., Fukuma, R., Yokoi, H., Kamitani, Y., & Yoshimine, T. (2011). Real-time control of a prosthetic hand using human electrocorticography signals. Journal of neurosurgery, 114(6), 1715–1722. https://doi.org/10.3171/2011.1.JNS101421
(11) Neuralink (Producer). (2020, August 28). Neuralink progress Update, summer 2020 [Video file]. Retrieved April 07, 2021, from https://www.youtube.com/watch?v=DVvmgjBL74w
(12) Romanelli, P., Piangerelli, M., Ratel, D., Gaude, C., Costecalde, T., Puttilli, C., Picciafuoco, M., Benabid, A., & Torres, N. (2019). A novel neural prosthesis providing long-term electrocorticography recording and cortical stimulation for epilepsy and brain-computer interface, Journal of Neurosurgery JNS, 130(4), 1166–1179. Retrieved May 7, 2021, from https://thejns.org/view/journals/j-neurosurg/130/4/article-p1166.xml
(13) Tamburrini, G. (2009). Brain to computer communication: Ethical perspectives on interaction models. Neuroethics, 2(3), 137–149. doi:10.1007/s12152–009–9040–1
(14) Miller, K. J., Hermes, D., & Staff, N. P. (2020). The current state of electrocorticography-based brain–computer interfaces, Neurosurgical Focus FOC, 49(1), E2. Retrieved May 7, 2021, from https://thejns.org/focus/view/journals/neurosurg-focus/49/1/article-pE2.xml
(15) Neuralink (Producer). (2021, April 08). Monkey MindPong [Video file]. Retrieved May 07, 2021, from https://www.youtube.com/watch?v=rsCul1sp4hQ
(16) Bullard, A. J., Hutchison, B. C., Lee, J., Chestek, C. A., & Patil, P. G. (2019). Estimating Risk for Future Intracranial, Fully Implanted, Modular Neuroprosthetic Systems: A Systematic Review of Hardware Complications in Clinical Deep Brain Stimulation and Experimental Human Intracortical Arrays. Neuromodulation: Technology at the Neural Interface. doi:10.1111/ner.13069
(17) Shi, Y., Liu, R., He, L., Feng, H., Li, Y., & Li, Z. (2020). Recent development of implantable and flexible nerve electrodes. Smart Materials in Medicine, 1, 131–147. doi:10.1016/j.smaim.2020.08.002
