Symbionic project week 4: First contact for our bionic hand controller
Moving forward with our plan to improve the experience for bionic hand users.
In the fourth week since we officially started, our first open source myoelectric sensors arrived and as engineers we just had to get them working immediately. A little soldering and tinkering and we’re reading our first muscle signals. We have made first contact with our own bodily data! And with a single sensor you can already control a bionic hand if you want, though perhaps not reliable.
The prospect of applying machine learning techniques to biosensor data for a good cause is very appealing to the data scientist inside me. But I should restrain myself a bit and make sure we’re going in the right direction. So to check that, we also made first contact with a lot of people involved in the field of bionics, and we are learning from these experts in a rapid pace. It is important to spot the bears on the road, but also the opportunities. To find out where we can really add value. And to find out who the customer of our project could be. Will we deliver our product to open source projects, or could we help people more by collaborating with commercial partners? Fortunately, everyone we speak to is very enthousiastic and willing to share their knowledge.
Knowledge building
We’ve discussed our ideas with Dr. Smit from the University of Delft. He is an expert on bionic hands and kindly willing to be a source of information. An important message we got from him is to focus on the reproducibility, or the classification rate, of a bionic hand controller. Advanced commercial myoelectric sensors may reach a 99% or more accuracy. That means that if you make a fist 100 times, it is also detected as such 99 times. This may sounds impressive, and it certainly is, but is it acceptable from a user perspective? If you are loading and unloading a dishwasher, and you drop 1 in a 100 items, then you still break a few things per week. This is very annoying, not to mention expensive in the long run. Because of this the user may prefer to just use his/her remaining hand and neglect the bionic hand.
Next to that false positives are annoying, where the bionic hand starts moving at a time when the user is not doing anything. This can happen if the sensor gets detached from the skin, or some outside electrical signal that interferes with the sensor. And the most difficult scenario, is when you started sweating. Then the control can quickly be lost.
Current affordable myoelectric controllers, such as the Myo armband, that are not focussed on Bionic Hands may operate well below the 99% accuracy. So there is still a market potential for an (open source) affordable alternative with a high reproducibility.
Another thing that keeps popping up, is a force feedback method to enable bionic hand users to grab objects without looking at them. Dr. Smit mentions there is a lot of research ongoing on this subject as well. For example, by pinching or stimulating the user skin. However, this comes with several drawbacks. For one, there is generally a large delay time. And secondly, this can be uncomfortable, a main reason to stop using prosthetics. Next to that the skin can become desensitized to continuous vibrations or pinching.
New leads:
- Wolf Schweitzer, a bionic hand user who is focussed on (control) quality. He has a blog. He has also written a case study about the use of prosthetic arms in demanding work environments. Excessive sweating seems to be killing for myoelectric sensing devices.
- And there are researchers that focus more on biometric control algorithms at the Twente University.
- Biddiss and Chau have nice overview articles, such as this one.
Searching for a customer
Commercial companies. There are two major manufacturers of bionic hands that operate world wide. These are Ottobock, a German company, and Ossur, an Icelandic company. Both sell bionic hands and arms to local suppliers. Both also recently acquired companies that specialize in upper arm bionics: Bebionic was acquired by Ottobock, and Touchbionics by Ossur. These former companies do the R&D and development of the bionic hands.
I’ve spoken with a Product Marketing Manager at Ossur. We had a nice conversation and in the end I got the question how Ossur could help us out. Ossur has wide knowledge about the global bionic market and developments, so I proposed to have a representative in our workgroup to provide feedback on our developments. My contact would discuss this internally and we agreed on future contact to discuss the status.
Open source projects. So far the most ambitious open source bionic hand project that I’ve seen in our region of the world is My Human Kit. A single contact indicated an open source myoelectric controller might be nice, but force feedback even more. Their design does not make independent finger control mechanically possible though. But if we’re serious about going open source, then these kind of projects might be our 'customers’. To be followed up.
Also reached out this weekend to Ambionics, where Ben Ryan is developing certified 3D printed hydraulic bionic arms for small children! And he’s considering to move to myoelectric controlled bionics for older children and adults. We’re going to stay in touch! Here is an inspirational video about his work:
Next to that we are keeping in contact with Marco from DIYstuff. This week he was kind enough to visit us during our evening work. He has already acquired a lot of knowledge about myoelectric sensors, their electronics and their usage. He himself is now focussing mostly on the mechanical design and manufacturability of a bionic hand. He is really brimming with ideas. Marco also mentioned that force feedback might be more beneficial than independent finger control.
OYMotion. I was contacted by Hualiang Ni, founder of startup that is developing a new EMG bracelet with better control algorithms. And he seems intent on helping bionic hand users with it. This sounds similar to out own ideas, so I’ll be following their progress with great interest!
After the discussion with Dr. Smit, I am curious about the gesture classification accuracy of these devices.
Yama’s ideas. Yama Saraj is one big flurry of words. He’s an economist who talks very enthousiastically about business development, sustainable world goals, developing nations, NGO’s, high tech startups, social entrepreneurship and bionic hands. And just connecting all of that together. We spoke at length about it at Vention, a startup for rapid prototyping, where he consults about business development while learning more about technology himself.
While I sometimes had a hard time following, we summarized his concept as follows:
- There is a great deal of knowledge in the world regarding bionics. But there is a need to consolidate this knowledge and help people apply it in ways that are profitable and sustainable.
- Where Yama likes to focus on developing nations. The 'spillover effect' from innovation in developing nations can help developed nations. This is called reverse innovation.
- In our own region we could help by setting up a workgroup to consolidate the knowledge, involving our own project and employer, as well as others, such as universities and high tech companies.
- Yama already has a non-profit organisation that can facilitate in raising funding from NGO’s and big foundations such as those at ASML, Philips, and others. See YSEA.
- Existing technologies such as 3D printed hands from e-NABLE or Ambionics an already be leveraged, while funding bionic innovation for the future.
Quite ambitious as you can see, and a long term vision. Essentially, he wants to kickstart a global ecosystem for affordable bionics. We’re happy to try and help out. If anyone else feels the same, you can contact Yama, he’s quite open to support.
Technological direction
We’re current focussed on the myoelectric control part, which is inline with our skills and interest. But based on the wishes of users and other projects, we are still open to stear our technical roadmap towards the following items:
- Sophisticated EMG control that feels natural to the user, ideally independent finger control (our initial priority)
- Robust, affordable EMG control, but perhaps only a few gestures
- Force feedback method to the user (some projects really, really believe in this value). This can be done for example using vibrations or electrotactile devices.
- Last, adaptive graps using some kind of feedforward model are quite interesting. For example, using a small vision system to do automated object recognition. Or proximity controlled grasps; near a coffee machine you’d need different grasps than when you are washing your car.
In any case, we’re currently still not considering to develop the mechanical hand ourselves. There are very ambitious projects out there that we could never compete with, and we’d much rather collaborate on a topic that would help them go the extra mile.
