CIONIC Blog
Published in

CIONIC Blog

Predicting Someone’s Movement Before They Move

At Cionic, we are building a platform for human augmentation. The platform consists of three pillars: analysis, prediction, and augmentation.

Pillars of the Cionic Human Augmentation Platform

  1. ANALYZE how someone is moving
    through multiple sensor modalities
  2. PREDICT what the person intends to do next
    through predictive modeling and AI
  3. AUGMENT the person to achieve their intended movement
    through modalities ranging from FES to exoskeletons

These are the necessary ingredients to help people transcend physical limitations.

Sensorized Clothing

For the first pillar — analysis — we have built clothing that is fully sensorized with both IMU and EMG sensors. Each of these sensors provide complementary but distinct measurements of human movement.

IMUs, or inertial measurement units, are sensors that use a combination of accelerometers and gyroscopes to measure motion. An IMU is how your fitness tracker or smartphone counts how many steps you did.

EMG, or electromyography, sensors measure muscle electrical signals. These signals can be picked up at the surface of the skin whenever a muscle contracts. EMG is often used by doctors to diagnose movement disorders or conditions.

EMG sensors are harder to productize than IMUs, but we invested the effort to include it in our platform because EMG provides a critical piece of information that IMUs cannot: a user’s movement intention.

How early can we predict a person’s movement?

To showcase EMG’s ability to predict a user’s intent, we ran an experiment. Our goal was to see how early we could predict a person’s movement using IMU versus EMG signals.

We trained recurrent neural networks (RNNs) to predict different movements types, specifically side steps, forward steps, back steps, and center stomps from a neutral position. The beginning of all of these movements looks nearly identical.

For the EMG-only model, we fed the 16 EMG leg signals into the model (8 from each leg). The IMU-only model was fed 6 quaternion-formatted sensor streams (one from each foot, shin, and thigh).

Foot pressure insoles were used as the gold standard label of when the movement had started. Foot pressure data allowed us to see exactly when the foot started lifting off the ground.

The results of the experiment were stark. EMG accurately predicted which movement the person was about to do 250ms, on average, before the foot started to lift off the ground.

IMU could also accurately predict the person’s movement, but only 100ms, on average, after the foot started to lift off the ground.

Here is a video showing the EMG model movement predictions on a test case:

EMG is able to do this early prediction because there is a delay between the onset of a muscle’s electrical activity and its subsequent force production and motion [1]. This delay is called electromechanical delay (EMD).

IMU, on the other hand, measures motion as it happens. In the case of side, forward, back, and center steps, the movements all look similar at the start: weight shifting to one leg and the other foot lifting off the ground. It’s not until a little bit later in the movement when the foot starts to go slightly forward or sideways that the IMU model can distinguish between the movements.

What this means for human augmentation?

For a human augmentation platform, predicting a user’s intention is critical. It’s critical because the intended and actual movements do not always match.

For example, let’s imagine an exoskeleton that is supposed to help people lift heavy things. A person tries to lift a heavy box to a higher shelf, but they can’t even begin to move the box because it is too heavy. A model based solely on IMU motion signals will not be able to infer that the person wants to lift up their arms. EMG signals, on the other hand, can easily see the biceps firing and tell the exoskeleton to help the person lift. EMG signals can also tell when muscle fatigue is setting in and increase the amount of assistance the exoskeleton is providing to the person.

For people with movement disorders, there are often examples like this one where the intended and realized movement do not match. That is why intention prediction is critical.

If we can sense how a person is moving, predict what they intend to do next, and then augment their body to achieve that movement, then we can help folks transcend their physical limitations.

References

[1] https://pubmed.ncbi.nlm.nih.gov/527577/

--

--

--

We’re a technology start-up on a mission to build a platform for human augmentation that powers people to transcend physical limitations. https://cionic.com

Recommended from Medium

9 Best Legal Assistant Software to Save Hours of Research and Win More Clients

How to Leverage Chatbots & AI to Save Money

5 AI Trends We Should All Be Watching in 2022

Introducing The New SHIRE Website, And Other Updates

AI and Natural Language Processing: State of Affairs in the Industry

Ontology Based Multi Agent Systems

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Lina Avancini Colucci

Lina Avancini Colucci

More from Medium

What are Probabilistic Programming Languages and why they might be useful for you

Our model consists of a categorical logit distribution for the discrete valued observations. The linear predictors for the probabilities of each of the categories depend on parameters beta. We assign a normal prior, of zero mean and standard deviation 5 to each element of the parameter vector beta.

CADChain’s New Algorithm Cuts the Bias out of Textual Data Sources to Address Gender Inequality

Progress Towards Reducing Global Inequality

A metal sculpture of a man saving another man, in a boat.

Navigating uncharted territories