Improving prosthetic hands with biomechanics
Last summer we were fortunate enough to be able to spend a week at the Royal Society Summer Science Exhibition talking about our research into ways to improve prosthetic hands. During that week, more than 11000 people visited the Exhibition. Obviously, we weren’t able to talk to all of them, but we gave it a good go! We had some really great conversations with people of all ages and backgrounds, but one of the most common questions was: “How do users of prosthetic devices get the hand to do what they want it to?”
Since then, we’ve been busy collecting and analysing more data, and I’m happy to share with you a paper we’ve written describing the control of prosthetic hands using computer models. In this work, we demonstrate the use of a biomechanical model to help the user communicate with the prosthetic device. A biomechanical model is a mathematical computer model that helps us understand the relationship between signals from the brain, electrical activity in the muscles and the resulting movement of body parts. For a far more in depth discussion of this, take a look at Dimitra Blana’s series on modelling in our dedicated Exhibition blog.
In our long-term vision, we will determine the user’s movement intent by recording the electrical activity in muscles and nerves that once went to the missing hand in an amputee, and use these as inputs to a computer model to predict what the missing hand would have done if it were still there. This information, or model output, can then be used to control a prosthetic hand in the same way. This approach uses our knowledge of biomechanics to improve the prediction, instead of just learning the pattern between inputs and outputs without any structural knowledge.
If you look at the video below, for example, you will see a robotic hand moving in roughly the same way as the real hand being used to demonstrate it. Electrical activity in the muscles is being sensed by electrodes on the skin and then used to predict the movement of the hand. The robotic hand then performs the movements predicted by the model. And rest assured, we are not using the movement of the real hand, just the electrical activity of the muscles in the arm, to control the robotic hand.
In our new paper, we set out to answer three questions in particular that would help us understand how well this model-based approach to control is likely to work, and which areas need further research and improvement:
- How well does the model-predicted movement match recorded movement when the model is driven my muscle activity recordings?
- Can we perform all the calculations necessary to predict the hand movement fast enough so it can be used for control without big delays?
- Can we use this approach for a range of typical hand activities that involve picking up objects as well as just moving the fingers?
We have attempted to answer these questions using a combination of methods: we have recorded some muscle activity with EMG recording systems, we have monitored the movements of people’s hands with a motion capture system, and we have carried out computer simulations of typical activities.
So how does it look? Well, we found that the model could predict the movements of users’ hands quite well just by using muscle activity, but there is certainly room for improvement in this area. One of the hardest parts is recording the right signals and recording from enough muscles. The model certainly runs fast enough, but of course that was on a normal desktop or laptop computer. With an embedded system, there may be additional challenges that we will need to consider. Finally, we showed how the biomechanical model could be used to control fingertip force to achieve object gripping without slip. This one remains hard to test in users while we don’t have access to all the signals we would like, but our simulation results are definitely promising.
Finally, I should point out that this paper is not actually published yet. But in a spirit of openness, we have released it as a pre-print in advance of sending it out for review and publication by a traditional scientific journal. Let us know what you think in the comments, and if you came to the exhibition and spoke to us, we hope you enjoyed it — let us know below.
P.S. If you like this, be sure to check out my article about prosthetic limbs in the Marvel Universe!
