What I learned from analyzing my Fitbit data

Fitbit Word Cloud Generated With Fitbit Tweets Via Python

With 2018 having arrived and New Year’s resolutions on the top of people’s minds, I decided to collect my Fitbit data over the period of November 18, 2017 to January 3, 2018 to examine intraday heart activities (hourly, daily, and in relation to events such as tests and work) as well as a sleep summary. Using Python and a few packages, let’s begin:

Link to entire code on Github

Starting with the data

Using unofficial Fitbit API clients (credit to orcasgit), I scraped information such as intraday heart rate (minute by minute data) as well as sleep summary details, which ultimately culminated into 46 files looking like this:

Messy, Uninformative Data

From there, I wanted to extract more information, so I extracted the day, month, year, and day of week from the title using the datetime package in Python.

Clean, Informative Data

I then looked at the overall summary statistics the 46 days of heart rates.

Figure 1: Initial Summary of Aggregated Data

Figure 2: Histogram Over Aggregated Data (Credits to Randy Olson for visualization tips)

While the summary statistics implies a right skew (mean > median) and Figure 2 confirms it, it doesn’t provide much information thereafter. We cannot conclude much from this as the data simply averages over the course of the approximate three months. Therefore …

So let’s next take a look at more specific graphs rather than an aggregated total, more specifically:

1. Activity by the hour
2. Activity by the day

Beginning with an overall view:

Figure 3: Hour of the Day vs Heart Rate (left) and Day of the Week vs. Heart Rate (right)

Beginning with the left graph, we can see that the time of day increases rapidly around 7AM, plateaus from 9AM to 7PM, and slowly falls down before repeating again. After years of rigorous education and a graduate degree, I can confidently say this trend mimics sleeping at night and being awake during the day.

As for the right graph, while it looks like the Day of the Week has no jarring patterns, we can note that Wednesday and Friday appear to have some spikes.

The intuition of leads me to examine the day of the week further.

Figure 4: Days of our Heart Beat

Looking at Figure 4, it appears the spikes are gone for Friday and Wednesday…

Perhaps we need to go even deeper, say by breaking down the fourth wall and using the power of hindsight to interpret this data. Thankfully, I personally can provide some details of my personal life:

1. I took final exams on December 8 from 2PM to 4PM, December 11 from 10AM to 12PM, and December 12 from 10AM to 1PM
2. I went to work on Tuesdays and Thursdays October to December ~9am:5pm)

Taking a Test

To check how being in the middle of a test effects my heart rate, I sliced my data to include the times I was taking the exam versus other days at the same time.

Figure 5: Testing Hours vs. Non Testing Hours

Interestingly, for me, the statistics show about the same numbers, except that my max is much higher for when I am not taking the test — perhaps because I tend to exercise on days I’m not in school or at work.

Work

Similarly with the test environment, I separated it into 9AM-5PM as well as the hours after when I was

1. At work
2. On a weekday not at work
3. On a weekend

Figure 6: Work Day (MW) vs. Non Work Day (MW) vs. Weekend 9AM–5PM and 5PM–12AM

From the bracketed points, we can see that at work, I had the lowest maximums and the highest minimums. On weekdays not at work, I had the highest maximum from 9 to 5. On weekends, it was an in-between. What we can extract from this is that at work, exercise, in the form of a high heart beat, was hard to obtain as well as a lower resting rate.

At this point, you might be wondering what about Fitbit sleep tracking, so let’s go to sleep! (but not actually unfortunately )

Sleep Data

While we could repeat the above process for sleep data, Fitbit API fortunately offers a detailed sleep summary. Similarly, with the power of hindsight used on real world events, I wanted to see if I could extract any patterns regarding finals season and sleep.

Figure 7: After the sleep data has been cleaned

While analyzing this next part, it reminded me of a very important point:

An Example of Misleading Data

One thing I learned the most from this was to understand the importance of carefully checking axes for using the same scale. For example, notice the difference between these two figures:

Figure 8: Sleep Summary Separated

Figure 9: Sleep Summary in Perspective

From Figure 8, it appears as though the graphs have huge increases and drops per day; that we spend almost the same amount of time asleep as we do awake or restless. But further inspection shows that the scale of minutes asleep ranges from 300 to 500 as opposed to 10 to 50! (as seen with Figure 9). So make sure to always check the axes.

With the aside, we can see that sleep restlessness and sleep awake are highly correlated, if not the same. It appears that being awake and being restless are very similar (once again justified by my multiple years in school), and that they both seemed to peak around December 4, coincidentally at the week before finals. So perhaps the stress of exams disrupted sleep.

Finally, I wanted to examine the correlations between everything.

Figure 10: Heat Map of Sleep Variables

As expected, minutes awake is highly correlated with restless duration and count, restless count is highly correlated with restless duration, and that efficiency is inversely correlated with restless duration, restless count, and minutes awake.

So that just about wraps it up for this first analysis of Fitbit data!

Conclusion and Future Work

In conclusion, we can see that

1. School exams don’t have an obvious effect on heart rate during the exam but does effect sleep quality(though this could differ for others)
2. Work centered around a desk stagnates exercise and other extreme pulses
3. Misleading axes make effects look more drastic than they really are
4. Data without interpretation is often difficult to understand; if you have the opportunity to relate it to real world events — do it!

So what’s next?

This was a far from non-exhaustive analysis of personal heart rate data. Future analyses with more data could include predictions of what days might throw off data such as holidays like Christmas and Black Friday, any seasonal effects such as getting more sleep in the early part of the year versus the latter, or predicting events from data. Additionally, as time goes on, I’ll be analyzing my data further to see if improvements to sleep and exercise will have a drastic effect as well as predicting life events from the data.

Thanks for taking the time to go through my analyses and let me know if you liked it / if there’s improvements / any errors.

Feel free to connect with me on my LinkedIn as I’ll be posting more articles and tutorials about other APIs and data science over the year!