Breathing in my sleep: from ‘ok’ to ‘ugly’. An example of a (mostly) ‘ok’ case.
What is this text about? The main goal of this short post is to describe the basic operation principles of a system which provides single-contact-point sleep monitoring when a (ring-type) pulse oximeter is used or absolutely contactless monitoring which includes heartbeats, respiration, and movements, when the blood oxygen level is not tracked, and to illustrate the types of data this system collects and generates and the way it presents that data to its users.
Why I use this system? To determine the extent of problems with respiration during sleep and to monitor effects and effectiveness of different appliances and interventions meant to improve breathing during sleep.
Data channels provided by the system: respiration via ALT (see below) and audio; heartbeats waveform via ALT; heart rate (HR) via pulse oximeter and ALT; oxygen saturation (SpO2) via pulse oximeter; movements via ALT and video.
Hardware used: D435i Intel RealSense camera connected to an old 1st generation Microsoft Surface Go tablet. Sound recordings are performed using a USB microphone connected to the ‘Go’ tablet. Blood oxygen level is monitored using a Wellue O2Ring ring-type pulse oximeter which allows exporting the SpO2 and heart rate data collected by it into .csv files.
How the system works, in a nutshell: The D435i camera projects multiple IR light spots onto the objects around it. These light spots are invisible to the human eye. Video frames captured by the D435i camera are processed by the system’s software running on the ‘Go’ tablet. The software detects variations in the properties of the light spots created by the light projector of the D435i camera such as variations in their brightness, shape, size, as well as changes in their number. These variations are caused by various movements of the objects carrying the light texture created by the D435i camera. This light texture is called the additional or artificial light texture (ALT). In a sense, such a system is an extremely sensitive contactless motion detector capable of detecting even variations and changes in the mentioned properties of the light spots caused by heartbeats and respiration. These variations and changes can be quite subtle and invisible to the naked eye.
Some features of the ALT technology:
- It detects vital signs regardless of the person’s pose, even when no area of skin is exposed.
- ALT transforms any consumer depth-sensing camera with an infrared (IR) light pattern projector into a contactless monitor for vital signs and sleep.
- Light patterns are invisible to the human eye.
- No problem if the person is completely under a blanket (or two) or is wearing loose-fitting clothes.
The sleep report generated by the system: The software of the monitoring system generates an audio-visual report for a whole night of sleep or for specific time intervals selected by a user. A user can, for example, select one or multiple time intervals on a SpO2 plot where a drop in the oxygen level is apparent or a heart rate spike is observed, and the software will generate a report for these time intervals. These reports can be played back at a normal (1x) speed or an increased speed (e.g., 4x) while still allowing the user to grasp all features of his breathing including changes in the respiration amplitude, snoring and other breathing disturbances including apneas or hypopneas. A video frame from such a report with added labels is shown in Fig. 1 below.
The frame includes 4 panels labeled A, B, C, and D in Fig. 1.
Panel A shows a plot with the following data lines on it: the thin black line follows the raw ALT data which is obtained by processing the video frames captured by the D435i camera; the red line highlights individual heartbeats captured in the ALT data; the blue line follows respiration: it rises during an inhale and falls during an exhale.
Panel B shows 2 copies of a video frame from the D435i camera with color overlays designed to highlight respiration (as well as other ‘slow’ movements) on the right and heartbeats (as well as other ‘fast’ movements) on the left. The brightness of the frame on the left was increased 3x compared to the brightness of the original frame from the camera which is shown on the right. This was done to give a better understanding of the scene captured by the D435i camera. The light spots mentioned above are recognizable in both frames. As the video below shows, the color overlays on top of the video frames highlight heartbeats and respiration through color variations of the light spots which are linked to variations in their properties mentioned above.
Panel C shows a plot with the following data in it: the blue line follows the SpO2 level captured by the pulse oximeter; the orange line follows the HR values obtained by the pulse oximeter; the red line follows the HR values obtained using the ALT data. Note that the pulse oximeter used generates a single SpO2 and a single HR value once every 4 seconds. An ALT-based HR value reflects the most dominant frequency component in the designated heart rates frequency range in the ALT data covering the previous 15 seconds long time interval.
Panel D shows a plot of a frequency spectrum computed for the previous 15 seconds worth of the ALT data. In the absence of large body movements as, for example, is the case here, the spectrum shows several dominant groups of frequency peaks which are related to heartbeats and respiration. More details on this plot at some other time.
The video below illustrates all the system and data features mentioned above. It is an example of an audio/video report for a 100-second-long sleep interval. When watching it, notice how the blue line in the topmost plot and the sound of breathing (with a bit of snoring, … sometimes) are synchronized. Notice how color changes in the right frame follow inhales and exhales through increases in the number of the colored spots and changes in their color. Notice also how heartbeats traced by the red line in the top plot are synchronized with variations in the amount of the red color in the left frame.
Tip: set the video playback parameters to 1440p60 for this video for better video quality.
As the title says, this is an example of an ‘ok’ situation when my breathing is mostly unimpeded. As the title also hints, there are other instances where the respiration clearly becomes labored and ‘disordered’. More on that and how such breathing is reflected in the captured data later.
If you are curious, you can find some additional information about the ALT technology here, here, and here.
Till the next time.
My other posts related to sleep monitoring.
Tip jar: bc1qtn36adncmfc0p9767m9hr3shgcmcv94cfl8gwd (BTC)
