How Welltory’s Phone Camera Measurements Stack Up Against Professional Heart Rate Monitors
Overview
Only a few thousand people use Bluetooth heart rate monitors with Welltory. The other 98% use their smartphone cameras to take measurements. Thus, ensuring camera measurement accuracy was especially important for us at the development stage.
It took us over a year to create a universal cross-platform signal processing library for iOS and Android. Both Bluetooth heart rate monitors and the camera measure RR-intervals — the length of time lapsed between heart beats. Analyzing RR-intervals lets us estimate heart rate variability indicators, subsequently generating stress & energy numbers for Welltory users.
How Polar measurements work
The Polar chest strap can be synced with Welltory via Bluetooth and takes ECG-accurate heart rate variability measurements [1] [2] [3]. Like ECG machines, the Polar chest strap uses electrodermal activity to register electrical signals that control the expansion and contraction of heart chambers.
How camera measurements work
The camera measures RR-intervals with a method called photoplethysmography (PPG). The flash illuminates the capillaries in the user’s finger and the camera records a short video clip in order to analyze changes in the color of the frame.
Why? As the heart expands and contracts, the amount of blood in your blood vessels varies. When these changes in blood volumes occur, the color of the frame changes as well. The speed of these changes corresponds to the time lapsed between heart beats. Photoplethysmography is used by most fitness wristbands and smartwatches: Fitbit, Mi Band, and Apple Watch.
However, these devices are not yet accurate enough to measure heart rate variability.
The study
We decided to prove that Welltory camera measurements are reliable. In order to do this, we conducted a study in which we analyzed parallel measurements, or measurements taken with the phone camera and the Polar chest strap simultaneously.
We put forward two hypotheses
Hypothesis 1: RR-intervals measured with the Welltory camera will be the same as RR-intervals measured with the Polar chest strap
Hypothesis 2: Heart rate variability parameters from the camera will be the same as HRV parameters from Polar
In order to test our hypotheses, we established several criteria for conducting the study:
- We added the option to take measurements with the camera and Polar chest strap at the same time in the Welltory app
- We only used measurements taken at similar times in the morning, when the user is calm, before different stressors and factors could alter the results
- In accordance with the recommendations given in ‘A review on studies comparing photoplethysmographic technology with an electrocardiogram [4], we used the Bland-Altman plot and the Concordance Correlation Coefficient to assess the results.
In order to launch our study, we made parallel measurement mode available in the Welltory app. When enough users updated the app, we asked Polar chest strap owners to take a few parallel measurements in the name of science.
If you also have a Bluetooth heart rate monitor and want to contribute to our study, feel free to turn on Debug mode, turn on parallel measurements, then select your Bluetooth heart rate monitor in Settings, and measure away! The camera will turn on automatically.
Our preliminary results are already in, and we couldn’t wait to share them with you!
Hypothesis 1: RR-intervals measured with the Welltory camera will be the same as RR-intervals measured with the Polar chest strap
The graphs below show parallel measurements taken by individual users with iPhone cameras and the Polar H7. The graphs illustrate:
- The top is the overlay of two rhythmograms showing RR interval values. The blue line shows values from the camera, and the green line shows values from the Polar chest strap. RR intervals are measured in milliseconds, so a value of 1000 means the heart rate is 60 beats / min, 1200 is 50 bpm, 800 is 75 bpm, and so on.
- The middle is a scatter plot that shows correlations between data from the two measurements. The closer the dots are to the green line, the more in sync the results from both methods are.
- The bottom is a Bland-Altman plot that shows the data consistency of measurements taken with two different methods.
Conclusions
- A visual comparison of the rhythmograms shows impressive similarities, as the values match up almost exactly. Camera measurements and Polar measurements yield very similar results, and the Concordance Correlation Coefficient (LCC) is no lower than 0.99
- The scatter plot shows the absence of deviations between the two measurement methods.
- The Bland-Altman plots, which are very sensitive to deviations, show a deviation range that does not exceed ~5–8ms, which means that deviations do not exceed 1%
As we can see, there is practically no difference between measurements. However, we need to make sure that estimates of HRV parameters from camera measurements are comparable to parameters estimated from Polar measurements.
Hypothesis 2: Heart rate variability parameters from the camera will be the same as HRV parameters from Polar
In order to compare HRV parameters from the camera vs. Polar, we selected 20 parallel measurements from users who met the following criteria:
- height 177 +- 9.25
- weight 75 +-5.6
- age 36 +- 9.7
In order to assess HRV accuracy, we compared the most widely-used parameters: rMSSD, stdRR, and pNN50
rMSSD
Bland-Altman LoA: 0.543+-4.0229
LCC: 0.992445457669
stdRR
Bland-Altman LoA: 0.161+-3.6259
LCC: 0.98198042415
pNN50
Bland-Altman LoA: 0.967+-10.2732
LCC: 0.866375992706Conclusions
- rMSSD — the Concordance Correlation Coefficient (LCC) was 0.99, which means the estimates of this HRV parameter match up perfectly. The deviation range in the Balnd-Altman plot (LoA) shows a small spread of values, about 4ms, which means it is clinically suitable for analyzing heart rate variability.
- stdRR — the LCC was 0.98, which means the estimates match up almost perfectly. The LoA shows a small spread of values of about 3 units, which means it is clinically suitable for analyzing heart rate variability.
- pNN50 — the LCC was 0.87, which means there is a strong correlation between estimates. The fact that the correlation coefficient is not ideal has a lot to do with how it’s calculated. There is a strict cut-off at 50ms, which means the parameter may be unstable in 2–5 minute measurements. The LoA shows a spread of about 10 units, which means it is clinically suitable for analyzing heart rate variability.
Summary
After checking both hypotheses, we can confidently say that Welltory’s PPG-based phone camera measurements are equal to measurements taken with Polar chest straps, which are ECG-accurate. You can definitely use your phone to take Welltory measurements, and don’t need to rely on expensive gadgets. It’s important to take into account that camera measurements are more sensitive to artifacts (distortions) due to movements or other light sources, which is why it’s essential to stick to the instructions when taking a measurement.
If you know how to interpret HRV parameters, they are available for each Welltory measurement in Measurement details. You can also track changes in your HRV parameters via Welltory’s web dashboard.
Appendix:
[1] Comparison of Heart Rate Variability Recording With Smart Phone Photoplethysmographic, Polar H7 Chest Strap and Electrocardiogram Methods. https://www.ncbi.nlm.nih.gov/pubmed/28290720
[2] Comparison of Polar® RS800G3TM heart rate monitor with Polar® S810iTM and electrocardiogram to obtain the series of RR intervals and analysis of heart rate variability at rest https://doi.org/10.1111/cpf.12203
[3] Validity of the Polar V800 heart rate monitor to measure RR intervals at rest https://link.springer.com/article/10.1007%2Fs00421-015-3303-9
[4] How accurate is pulse rate variability as an estimate of heart rate variability?: A review on studies comparing photoplethysmographic technology with an electrocardiogram. International Journal of Cardiology, 166(1), 15–29. https://doi.org/10.1016/j.ijcard.2012.03.119
