Accurate and reliable wearable technology is the way of the future for accessible health and fitness information

Kevin Moncion, MSc

Follow

Aug 11, 2020 · 8 min read

Being healthy is hard to quantify in today’s world. With the advent of wearable technology, health data are readily available at low costs in the hands of consumers.

Have you ever ventured off into your Health app to get a summary of how healthy you are? Being an exercise and technology fanatic, I do this all the time. Is this a bad habit? Am I feeding into all the craze?

There’s a ton of data and metrics available here. It makes me wonder, can all of this information get extracted from my wrist? Can I trust all of these data? Maybe I’m not the only one questioning the validity of this device. See this wonderfully written piece by Dr. John Mandrola, MD:

Today I was browsing through the “Heart” section and stumbled upon my heart rate variability (HRV). Wow, I was shocked. My average HRV was 37 milliseconds. According to normative values (Nunan et al., 2010), I am ranked in the poor category. Woof.

For those who may unaware, HRV is an indicator of how well your autonomic nervous system is functioning (Shaffer and Ginsburg, 2017). Have you ever heard of the “fight or flight” response? This is your autonomic nervous system kicking in.

It made me wonder, is my autonomic nervous system shot? Am I burnt out? I exercise daily, get 8 hours of sleep, eat well, and manage my stress levels. So what’s the deal?

Let’s do some digging. A quick google search yields dozens of articles about the accuracy of the wearable device. Am I sold yet? Should consumers take everything at face value?

None of the articles have gone in-depth and explored what I am about to discuss with you today.

Before we get into the science, let me first outline a few disclosures:

1. This article is meant to educate you about potential measurement errors.
2. No single piece of consumer equipment will ever be 100% accurate.
3. I have vast experience analyzing real 12-lead electrocardiogram data.
4. I love Apple products. I have them all.

Let’s dive into the science, shall we?

What is HRV? Your heart shouldn’t be a metronome.

A healthy human heart should not beat at a constant steady pace (Shaffer et al., 2014) and has been best described as mathematical chaos (Goldberger, 1991). There should be some degree of variability between each heartbeat, and this can be quantified using heart rate variability (HRV).

There are several ways to quantify HRV, including root mean square of successive differences between normal heartbeats and frequency domain measures. However, the Apple Watch simply measures the variation between each heartbeat in milliseconds (ms). In general, a larger HRV reflects a better functioning autonomic nervous system. Higher HRV values have been associated with greater fitness, well-being, and cognitive function (Dong, 2016; McCraty et al., 2015).

How can Apple improve the Heart Rate Variability Data for the next-generation Apple Watch?

1. Follow recommended HRV guidelines.

How does Apple calculate your HRV? The Apple Watch measures the standard deviation between each interbeat-interval (aka SDRR) from the heart rate sensor in your wrist for a 1-minute segment. This is considered ultra-short-term HRV and is only appropriate when you breathe at normal rates (~11–20 bpm) (Shaffer and Ginsburg, 2017). Can a 1-minute segment randomly measured throughout the day provide an accurate indication of your autonomic nervous system function?

This is conflicting against short-term recording standards, which should be at least 5-minutes for measures of SDRR (Task Force Report, 1996). A longer recording period can provide more data about your nervous system, including cardiac regulation to a variety of environmental stimulation (e.g., heart’s response to changing workloads), circadian rhythms and sleep-wake cycles (Shaffer and Ginsburg, 2017).

The next-generation Apple Watch should consider measuring for at least 5-minutes. Even better, how awesome would it be if we can choose the recording period?

2. Conduct validation studies to the gold-standard.

The gold-standard method for measuring HRV is to use a 12-lead electrocardiogram (ECG) over 24 hours (Shaffer et al., 2014). This quantifies the time variation between each R-R interval over a prolonged period. For consumer-grade HRV analysis, a 24-hr recording segment is probably “overkill”.

However, could advanced wearable technology replicate this level of accuracy with a wrist-worn device? With consumer products, there is bound to be some measurement error, especially with a 1-minute sampling rate. We need published validation studies.

A study by Hernando et al. (2018) found that Apple HRV is highly comparable to the chest strap Polar H7 monitor. From a wearable technology standpoint, it’s important to have consumer-grade technology be comparable to each other.

What about a comparison to the gold standard ECG?

Best science says validation studies should always be compared to the gold-standard methodology (Gold et al. 2010). To truly test the validity and reliability of the Apple HRV, it should be validated against the 12-lead ECG. I am sure Apple has done some in-house testing, but I have yet to read a published scientific study validating the Apple Watch HRV to a 12-lead ECG.

3. Can the Apple Watch filter ectopic beats or movement artifacts?

Can your Apple Watch tell the difference between a true heartbeat and an ectopic beat? I haven’t found anything online about it. I think machine learning and artificial intelligence can!

For simplicity’s sake, a true heartbeat is generated from the sinoatrial node and is transmitted to the atrioventricular node to generate a heartbeat. In contrast, an ectopic beat can either be classified as physiological (e.g., premature ventricular contraction) or technical in origin (e.g., poorly placed watch). Ectopic beats do not originate from the sinoatrial node.

Fun fact: ectopic beats are common in everyone.

Ectopic beats are seen when analyzing the variability between R-R intervals. They confound the reliable analysis of HRV and make it almost impossible to accurately quantify HRV (Peltola, 2012). Therefore, all abnormal beats not generated by the SA node must be eliminated from the HRV analysis. Even having 3 or 4 ectopic beats in a 1-minute recording segment can skew the HRV values (Peltola 2012; Shaffer and Ginsburg, 2017).

Ectopic beat filtering is complex and necessary to have valid HRV values. It’s a multi-faceted approach and should require an R-R interval threshold detection-based method against an individualized local average, in addition to visual inspection of the ECG segment. You can’t set arbitrary ectopic beat thresholds (e.g., 800–1200ms) as you risk removing true heartbeats. The removal of true heartbeats can influence the HRV values and you will lose important information regarding the variability of the heart rate signal (Laborde et al., 2017).

With the advent of wearable technology, the use of ECG in real-life environments has skyrocketed. However, even a weak motion artifact can lead to a slight wandering in the ECG which can result in detecting a false QRS complex (Choi and Shin, 2018). Unfortunately, I haven’t been able to locate if the Apple Watch is capable of filtering out ectopic beats.

Apple should develop algorithms, use artificial intelligence and machine learning to individually detect and remove ectopic beats and artifacts.

4. HRV is sensitive to everything under the moon.

HRV is highly sensitive to changes in posture, breathing, sympathetic activation, medications, smoking status, sleep, caffeine (Laborde et al., 2017), pretty much everything. These are all factors that confound HRV analysis.

I’ve read elsewhere that using the Breathe function on the Apple Watch helps improve the accuracy of HRV measures. However, given what we know about ectopic beats and the lack of validation studies, I am not 100% sold on this feature yet.

Apple should consider developing a standardized protocol for HRV assessment.

Closing remarks

The Apple Watch is awesome. I love it. Imagine a valid and reliable wearable device that learns your heart rhythm and continuously improves its accuracy over time? The opportunities are endless.

I believe that this technology can be almost perfect if they consider these 4 things in this order:

1. Apple should develop algorithms and use machine learning to detect and remove ectopic beats for a given individual.
2. Apple should validate their measures to a 12-lead ECG.
3. Apple should consider developing a standardized protocol for HRV assessment.
4. Apple should record HRV for at least 5-minutes.

References

1. Shaffer F, McCraty R, Zerr CL. A healthy heart is not a metronome: an integrative review of the heart’s anatomy and heart rate variability. Front Psychol. 2014;5(September):1–19. https://doi:10.3389/fpsyg.2014.01040
2. Goldberger AL. Is the normal heartbeat chaotic or homeostatic? News Physiol Sci (1991) 6:87–91.
3. Shaffer F, Ginsberg JP. An Overview of Heart Rate Variability Metrics and Norms. Front Public Heal. 2017;5:1–17. https://doi:10.3389/fpubh.2017.00258
4. McCraty R, Shaffer F. Heart rate variability: new perspectives on physiological mechanisms, assessment of self-regulatory capacity, and health risk. Glob Adv Health Med (2015) 4:46–61. https://doi:10.7453/gahmj.2014.073
5. Hernando D, Roca S, Sancho J, Alesanco Á, Bailón R. Validation of the apple watch for heart rate variability measurements during relax and mental stress in healthy subjects. Sensors (Switzerland). 2018;18(8). https://doi:10.3390/s18082619
6. Choi A, Shin H. Quantitative analysis of the effect of an ectopic beat on the heart rate variability in the resting condition. Front Physiol. 2018;9(JUL). https://doi:10.3389/fphys.2018.00922
7. Gold R, Reichman M, Greenberg E, et al. Developing a new reference standard: is validation necessary?. Acad Radiol. 2010;17(9):1079–1082. https://doi:10.1016/j.acra.2010.05.021
8. Peltola MA. Role of editing of R-R intervals in the analysis of heart rate variability. Front Physiol. 2012;3 MAY(May):1–10. https://doi:10.3389/fphys.2012.00148
9. Laborde S, Mosley E, Thayer JF. Heart rate variability and cardiac vagal tone in psychophysiological research — Recommendations for experiment planning, data analysis, and data reporting. Front Psychol. 2017;8(FEB):1–18. https://doi:10.3389/fpsyg.2017.00213
10. Nunan D, Sandercock GRH, Brodie DA. A quantitative systematic review of normal values for short-term heart rate variability in healthy adults. Pacing Clin Electrophysiol (2010) 33:1407–17. https://doi:10.1111/j.1540–8159.2010.02841
11. Task Force Report. Heart rate variability: standards of measurement, physiological interpretation, and clinical use. Circulation (1996) 93:1043–65. https://doi:10.1161/01.CIR.93.5.1043
12. Dong JG. The role of heart rate variability in sports physiology (Review). Exp Ther Med. 2016;11(5):1531–1536. https://doi:10.3892/etm.2016.3104