Data science, Heart of IoT
If you haven’t followed our precedent posts, InMoodForLife is a project to build a sleep tracking and analysing device for young people with mood disorders. We began by describing how we reverse engineered the Beddit sensor, then our Raspberry Pi based setup to collect and analyse the data and the Eclipse Vert.x and Warp10 stack that we put into the Raspberry Pi to do it. Today we talk about a key point of the project, the data analysis.
Since our last blog post, the InMoodForLife team hasn’t enjoyed many moments of farniente. From a purely technological point of view, everything seemed under control… besides one small detail: make a rigorous analysis of all the data collected by Eclipse Vert.x and stored into Warp10. This mass of data is of a specific kind, omnipresent in IoT devices: time series.
The challenge is double: first to be able to split the raw data from the sensor into respiration, heart rate and agitation and, in a second time, detect REM sleep. The evolution night by night of the time spent in REM sleep is a reliable criteria for anticipate manic episodes. Researchers have found that REM sleep gradually disappears in the fortnight before a manic episode. And that’s just the key of our project, to be able to do those analysis on the data recovered by the sensor.
We found several hindrances in out path. The first one was that we cannot base the algorithms on our intuition, as in order to verify or dispel the validity of an algorithm we would need a second way of REM monitoring (in our case EEG) in order to compare results. We had to base the algorithms on research papers on the relationship between REM and heart rate.
Fortunately, many studies exist, and I had them as my bedside books during the last month. Less relaxing than a good comic but really interesting.
How make time series analysis?
Data scientists mainly use tools likes R, Python or also Matlab for time series analysis. We had to make the same thing on the Raspberry Pi. For this job Warp10 integrates an open source language, WarpScript, dedicated to time series analysis.
In the bed or not? This is the (first) question.
This is the first step. It seems a pointless exercise to extract an hear rate when… nobody is in the bed.
The signal is clear, a limited amplitude and a long drift of the sensor.
When somebody is in the bed, the amplitude and the periodicity of the signal are clear too.
After looking those patterns, we found that good way to detect if there is somebody on the bed was to bucketize the signal into steps of a length twice the typical respiratory rate. For each bucket we compute the difference between the minimum and the maximum values, and use that difference as a score.
The criteria on the score is simple too, under 50, nobody in the bed. Upper ? Somebody is in the bed. Only 18 lines for this first simple Warpscript.
Second step: isolate movements
During the night we move in the bed. When trying to extract the heart and respiratory rate from the force sensor reading, this agitation is bothersome noise. Too much of this noise can bar us from extracting anything.
As we do analysis in one minute windows, we needed to know the time spent in agitation in order to know if we can make a signal extraction in good conditions or not. We have found that when analyzing 60 seconds of data, noise ranges longer than 4 seconds (one signal period) seriously crippled our analyzing capabilities.
So we needed to treat our signal to try to isolate those agitation data-points. We did it again WarpScript, and after several tries and failures, we found finally a solution: compute the delta and the absolute value between each tick. Smooth the obtained signal and keep the points with a value upper than 20 (computed criteria).
Third step: decompose heart and respiratory rate
We didn’t reinvent the wheel but apply a well know method named STL decomposition for this job. Given than STL was already fully integrated into WarpScript, our program was rather simple: a STL and some smoothing.
The result is really spectacular, we got clean heart and respiratory rates signals. And using another classical tool, the Fourier transform, we can easily get the periodicity of these signals.
The 3 extractions (heart rate, respiratory rate, agitation) are stored each minute as datapoints in three time series, that we will use to make the REM analysis at the end of the night.
In doubt….
Here is the first extraction (HR/RR/agitation) normalized. Well, to be completely sincere, my the first reaction was “What can I possibly do with that”!
The solution comes from one of my strange bedside books: a study of Masao Yaso, Atsuo Nuruki, Sei-ichi Tsujimura, and Kazutomo Yunokuchi about “Detection of REM sleep by heart rate”. They solve the signal variability with a simple amplification (HR*HR) and a smoothing. All became crystal clear then:
Extract REM
During REM the heart rate can increase or decrease… there is no simple rule, but the “paradox” of the REM is helpful. This phase is characterized by random movement of the eyes and low muscle tone. The agitation extraction is a good indicator of muscle tone. We can suppose than a very low muscle tone during a time range longer than 8 minute is a good indication of REM sleep (deep sleep is sliced with reflex movements each 6–8 minutes). If you correlate this information with always the same HR pattern variations (increasing or decreasing) during this phase you have a good approximation of the amount of REM sleep during the night.
Is data the new oil?
Indeed it is! With open source technologies, data analysis have never been so simple. Yes, you can create awesome services with them, but it’s not a magic wand. You always have to deeply understand what you’re doing when manipulating data. That’s we’ve done last month, deeply understand how sleep phases are organised in order to decompose them with an acceptable level of reliability.
