Symbionic project after 2 months: many signals in the noise.
Growing our network and finding focus
Two months into our Bionic Hand project, we’re focused on the design of algorithms for myoelectric sensors, we have been networking intensively and challenged ourselves to experience the problems we’re trying to solve.
Ductape Challenge
We were challenged by amputees and partners to ductape our own hand for a day. In order to experience, just a little bit, the daily frustrations of living with one hand. I added a few photos to show our impression of the day.
If you want to try it yourself, we really advise to put a sock under the ductape, and do not tighten it too much.
We experienced quite some scenarios, like:
- Opening tea bags with one hand. Using your teeth helps there.
- Writing on a laptop is terribly slow. A single extra finger on the other hand would already help.
- Zippers and buttons seemed near impossible at first, but we managed after a while.
- We thought about how to peel an orange. Perhaps with our teeth as well?
- Tying your shoe laces? That requires a lot more practice than one day.
- Writing on a whiteboard was surprisingly doable with my left hand.
- Riding a bike is possible if you still have the stump.
- Normally you are often multitasking with two hands. Like hold a cup of coffee while browsing your e-mail. With one hand this means more deliberate moves are needed.
In the end, I realized that most things that are easy with two hands can still be managed with one hand. You get quite creative after a while. But something to hold an object firmly while the remaining hand does the finegrained tasks would’ve helped. Nevertheless, I also understand why so many amputees reject uncomfortable or expensive prosthetics.
The main difficulties arise for tasks that already feel like you need 3 or 4 hands. I realized this while bathing my two children in the evening (without ductape). I can only imagine what it’s like to change a diaper for an toddler trying to wrestle out of your control … with only one hand.
Collaborative Ecosystem
Our network in bionics is taking shape.
Talk with Hualiang Ni from OYMotion. He gave a wonderful demonstration of his myoelectric armband, and all the projects he is working on. Hualiang seems heavily focussed on helping out the prosthetic community and developing a learning platform for his armbands.
Talk with Ben Ryan from Ambionics. I’ve talked about Ben’s vision in the previous blog, but we didn’t have the pleasure of talking to each other yet. Since we are eager to find a good customer/partner to work for, we had a great conference call. Ben got an OYMotion armband for free to experiment with in his prosthetics.
Talk with Prof. Rietman from Roessing R&D. He has a huge network of amputees and researchers in prothetics. If we decide on our goals, we can contact him again for further discussion.
Talk with de Hoogstraat, a rehabilitation clinic. They confessed that they hardly look at technical innovation in bionic hands, because they are focussed on helping their clients with existing technology. If anything, they experience mostly mechanical instabilities as issues with the current products on the market. The control aspect seemed less of an issue for them.
Talk with Bodo Hoenen. Bodo has found Dev4x.com, a community dedicated to Open Social Innovation, working on developing moonshot projects for the sustainable world goals. He also started a Myoelectric Robotic Assistive Arm project for his daughter who became paralyzed to her arm. Therefore, he became interested in open source Myoelectric control methods, and he is very eager to help out the bionic community. He is currently trying to retrofit the Myo armband to his daughter’s arm for further data collection. Some of the data and code can be found on github.
Talk with Betsy Lindsey from HighTechXL. She pointed out some possible subsidies and investors should be decide to go to the next level with our social entrepreneurship project and business model. She also pointed out LiquidWeb, a fascinating startup focussed on brain-computer-interfaces for the medical world. Their lead project is focussed on helping paralyzed people control objects and communicators. Very cool!
Talk with Nicolas Huchet and David Guaillier of My Human Kit. I’ve written about their project before, and it’s quite impressive. They mentioned the following priorities:
- Mechanical, light-weight, robust design of the hand
- High-fidelity (myoelectric) control of the hand
- Comfortable sockets
- (optional) Force feedback for the user
We’ve discussed the option to collaborate, where we focus on the myoelectric control so they can stay focussed on the mechanical aspects. If we could work towards a modular platform, where people could just take open source parts, and build their own ideal bionic hand as if they were lego blocks, that would be perfect! Nicolas also knows Bodo, so we proposed to try out his Open Social Innovation platform for our collaboration and shared documentation.
Talked with several amputees, who all stopped using their bionic arm or prosthetics. All because the discomfort did not outweight the benefits.
I am also connecting with Python developers that worked on myoelectric data processing and algorithm design in the past. We’re finally starting with the technology now, so any giant shoulders to stand upon are more than welcome.
We have another interested colleague and data scientist at work who is interested in joining or helping the project. I really hope we can find more experts to help us out as we converge on our problem and proposed technical solution.
Starting the data processing
We’ve decided to do our data processing, analysis and algorithm development in Python. After all our searching and knowledge building, I’ve decided to spent some time myself on the data analysis. I’ve already learned the following:
- Connecting to an arduino with pySerial.
- Creating efficient animations with matplotlib, via so called blitting.
- Using the above I can now visualize the real time streaming data from our EMG device in a Jupyter notebook.
- And I had a go at visualizing the raw data from a Myo armband (see below), collected by Bodo and Kieran.
- Oh! Also found an old Kaggle competition for EEG based arm motion detection. To be looked at.
Technology direction
We’re abandoning independent finger control for now. There is just no market pull for it. Especially if it would cost any accuracy or control, then no one really needs or wants it.
Therefore, it looks like our entire network has pushed us to focus on creating robust, affordable myoelectric control (algorithms). Keep it simple and valuable.
We’ve identified the following main challenges and ideas:
- Low classification errors; users quickly reject their bionic hand if it drops objects or misbehaves too often.
- Robust to extreme conditions, such as sweating. But also make it work well for people with weaker signals than healthy adults, such as amputees and children. This is related to the above challenge about low classification errors, but just in more realistic and extreme situations.
- Little time delays. This involves real time classifying the gesture already before it is finished. Nobody likes waiting. So this should happen within something like 200 milliseconds.
- Pleasant learning experience, such as OYMotion provides, maybe even gamified. This would certainly suit Ambionics and others focussed on applications for children.
Another backup idea remains a force feedback method for bionic hands, if we have to pivot at some point. But for now we keep our focus on myoelectric control.
In the next blog, I hope to let you know how we decide to tackle these technical challenges. Which hardware, which software and what kind of high level system architecture we have in mind.
