Listen to this story
What I Learned from Six Months of Obsessive Sleep Hacking
How I tracked data to reveal the best habits for improving my own sleep—and how you can, too
It felt like I was drifting through my days, always tired, a bit out of it, yawning, grabbing coffees where I could for a quick burst of energy.
I wondered if this was just what getting older felt like (I really hoped not), or the side effects of hitting 40 and having three kids, or whether the quantity and quality of my sleep was the issue.
So I got myself a fitness tracker to monitor my sleep for a few days.
What began as a “let’s do this for a week” thing turned into a 6-month obsession with spreadsheets, graphs, experiments and hypotheses, and some interesting/fascinating/odd findings. I want to share those with you, because I was able to quickly improve my sleep—and perhaps you can, too.
The Sleep Monitoring Tool
I picked up a good second-hand fitness tracker from eBay, a Jawbone UP3 (sadly no longer in production). The UP3 also presented clear statistics on how much light sleep, deep sleep, and REM sleep I was getting, which which I was specifically interested in (more on that below). [Editor’s note: Jawbone’s UP service, an online service required for you to get data from the device, has had recent interruptions and may be unreliable. They say these interruptions are temporary, which means getting a used UP may be a worthwhile risk. However, for in-production options we recommend Fitbit’s Alta HR, Versa, or Charge 2 models.]
Fitness trackers gauge what type of sleep you’re having through a process called ‘actigraphy’: monitoring your wrist movements and how much you’re tossing and turning during the night, and translating that into the appropriate sleep type. There is obviously a margin of error and some estimation involved with this, but a decent estimate was all I needed.
The first night, I put on the tracker, paired it via Bluetooth with my phone, opened the Jawbone UP app and clicked Start Sleep as I got into bed. I eventually nodded off.
Next morning my alarm went off and, like a giddy child on Christmas morning, I excitedly tapped End Sleep in the app and waited for the results:
- Total sleep — 7 hours 28 minutes
- REM sleep — 56 minutes
- Deep sleep — 54 minutes
- Light sleep — 5 hours 26 minutes
- Awake for — 12 minutes (3 periods of being awake)
I immediately thought “7 and a half hours is sweet, I don’t remember waking up at all, but those deep and REM sleep numbers seem a bit shitty”.
By my calculations, 73% of my total sleep was light sleep. Just 27% Restorative Sleep (which is REM + deep sleep) seemed, well, not very restorative, based admittedly on my scant knowledge of the subject.
I clearly needed to do some research.
How much sleep ‘should’ we get and which types?
It turns out we need all three types of sleep — light, REM and deep (experts also divide these types into sub-types, but let’s keep it simple). If we don’t get all three types of sleep, then we have a shitty sleep and won’t feel well rested at all.
Deep sleep is particularly restorative for the body, and REM sleep is restorative for the mind. I set my goals based on “restorative sleep”—the total of deep sleep and REM sleep.
In general, a person needs 7–9 hours total sleep and 1.5–2 hours of restorative sleep per night on average.
I was shooting for 7–7.5 hours of sleep per night. I know 8–9 hours is always trumpeted as ‘The Ideal’, but I need to get up at 5.30am a few mornings a week to go train clients, and I was realistically never going to go to bed at 9pm the night before.
I wanted around 2 hours of my 7–7.5 hours to be restorative (REM + deep), which is around 30%. So, if my tracker told me that around 70% of my total sleep was light sleep, that was a win (because the remaining amount was restorative).
I hypothesized that if I got that amount more often than not, then theoretically I would feel better rested, more alert and hopefully be more productive.
I imagined myself strutting down the street like John Travolta in Stayin Alive on the days after I’d hit my sleep targets…
The following night, I went to bed a little earlier, and didn’t look at my phone in bed. The data the next morning was better — 7.8 hours, 66% Light, 34% restorative.
But what had influenced that figure? Was it the earlier bedtime, the lack of screens in bed, or both? Or neither?
This piqued my curiosity — could I literally just ‘decide’ each day how good a sleep I was going to have later that night by doing (or avoiding) certain things?
I figured I needed a way of visually comparing each night’s stats and factors on an ongoing basis, as the app wasn’t showing this.
This was no longer a 1-week fling with sleep tracking. I was in it for the long haul.
My wife rolled her eyes.
At this point, dear reader, things took a weird, obsessive turn.
Tracking The Data
I found an IFTTT (If This Then That) applet that would automatically send the UP3 stats to a Google Sheet every morning, so I could more easily track things over time. I could now start experimenting. Game on!
Here is a similar IFTTT recipes for Fitbit.
Thinking about my standard day, I listed a few things that could potentially affect my sleep:
- How active I was each day
- How much caffeine I had
- Whether I watched TV/looked at phone right up to bedtime
- How close to bedtime I ate
- Whether I had sex
- Whether I wore a sleep aid
- What supplementation (if any) I took
I decided to track these things for 6 months alongside my sleep quality to see what, if anything, could be done to improve my total sleep time (by falling asleep more easily and staying asleep longer) and the percentage of restorative sleep.
This is what the raw data looked, like plus the notes I would add each day depending on what variable(s) I’d messed around with, detailed in the Notes column. I eventually came up with short keyword codes because I got tired of typing ‘nasal strip’, ‘exercise’, etc. every day:
Experiments and Findings
The first three months involved a lot of experimentation and combining different behaviors each day. Once I noticed trends forming, I was able to put in place a kind of best-practice approach each day to make sure I got a great sleep.
So let’s look at the trends I noticed in each behavioral area:
As a heavy drinker of coffee, I knew caffeine was possibly a culprit of my less-than-great sleep quality. So I went cold turkey, like a fool and gave up coffee completely.
I inevitably experienced that shot-in-the-forehead headache from caffeine withdrawal, but the next day I coped fine.
Restorative sleep was up around 30% after a week of no caffeine though, and I was getting more total sleep because I was falling asleep faster rather than being wired. 30% increase—and of a bigger total amount of sleep—was an impressive bump.
However, I missed coffee, especially on those mornings I had 5.30am starts.
I started having only one strong coffee, once a day in the morning, so the caffeine would be well out of my system by bedtime.
This worked well for two reasons:
- It did not affect my sleep.
- That one strong coffee blew my head off (exactly what we want coffee to do first thing in the morning). The effect was stronger because I wasn’t full of caffeine all day long. Major win!
I work out 3–4 times per week, mostly weights and short bursts of cardio. I was interested in whether exercise was making my body tired so that I would fall into a deeper sleep, or whether it had no real effect.
I found that total sleep was increased on days I trained or was quite active (out walking with family, gardening etc.), but only by 10–15%. Mostly this seemed to be a result of me wanting to go to bed earlier and also falling asleep sooner, but restorative sleep actually wasn’t significantly affected.
Limiting Screen Time Before Bed
The light given off by phone screens is shown to suppress melatonin, which we need to fall asleep, so it is thought to have an impact on sleep.
I decided to try to turn off the TV for a while before I went to bed, and to not look at my phone (aside from turning on the Jawbone app for sleep tracking) once I was in bed.
This certainly contributed to making me feel less mentally stimulated at bedtime. In fact, I was often drowsy while climbing the stairs to the bedroom after reading a book for a while, having switched off the TV.
As with exercise, this led to an increase in total sleep. I was drowsy earlier and didn’t lie awake for as long. But there was no consistent change to the amount of restorative sleep I got.
Most of us have read what we should and shouldn’t do nutrition-wise to help us sleep, but many of us (me included) struggle to put the advice into practice regularly.
Before bedtime, we’re typically told to to avoid:
- Rich food, so that indigestion doesn’t interrupt sleep
- Sugar, so we’re not wired and unable to fall asleep
- Alcohol, which although helpful in making us fall asleep, stops us staying in a deep sleep for long
I can confirm that all of the above is true!
Restorative sleep was definitely lower the nights I’d had more than one beer, and on the odd occasion my wife and I bribed someone to look after the kids so we could get out for dinner (usually with a combination of these factors we are told to avoid), both my total sleep and restorative sleep were both 15–20% lower on average.
I am a mouth-breather, due to getting a few too many whacks on the nose playing rugby at high school. I can’t breathe well through my nose and I snore.
I have taken to wearing nasal strips (a cheaper off-brand ones off eBay — you may be noticing a theme with my purchases…) in bed, as I find they open up my airways better and stop me snoring (source for that data? My wife).
Even before I started this Sleep Tracking Extravaganza, it always felt like I hadn’t slept well when I hadn’t worn a strip, and that my sleep had been patchy.
The data from the UP3 bore this out too, showing multiple instances of me being awake during the nights when I was strip-less, and as a result getting 15–20% less total sleep and around the same percentage less of restorative sleep.
I sleep on the ‘window side’ of our bed, and we live by a main road. We don’t have full black-out blinds or curtains, and light sneaks through the blinds. Maybe this was affecting my sleep too?
So I bought an eye mask (yes it was cheap, yes it was off eBay—don’t judge me!). It basically looked like a really bad Batman mask. Here it is:
That’s right, lux-u-ry. I can confirm that my percentage of “Time Feeling Like a Weirdo” was very high the nights I wore the mask.
I did definitely fall asleep faster while wearing it, and I felt immediately calmer when I put it on. Whether that was something to do with sensory deprivation, I don’t know. But total and restorative sleep weren’t much different the nights I wore it, according to the data.
It may not surprise you to know that a 41 year old married man with 3 kids does not have a rampant sex life, despite my wife being super-hot.
But let me tell you what puts the dampeners on your sex life even more — wearing a nasal strip and a bloody Batman mask in bed! #moodkiller
Weekends are realistically the only time we have enough energy for doing anything at bedtime other than mumbling to-do lists for the next day at each other before trying to fall asleep.
But when we did have sex during my sleep tracking months, the results were interesting. I mean the data. The data was interesting! (“Hmm, that was interesting” are not words anyone wants to hear after sex!)
I already knew that I fall asleep pretty soon after sex (apparently the biochemical prolactin is released in men afterwards, which makes us drowsy), but the data showed that I was getting 20% more restorative sleep on nights I had sex, too.
20% was a nice bump. But whereas with caffeine I could make that change daily, there wasn’t a hope of me having sex every day! 20 year old me possibly yes, 41 year old me? No way.
For women, sex also boosts estrogen, which is shown to lead to more REM sleep — any women who can back this up or shoot it down please let me know in the comments!
I’d stumbled across a supplement called ZMA before—it’s just a proprietary blend of Zinc, Magnesium and Vitamin B6. Taken 30 minutes before bedtime, its proponents claim it helps boost testosterone and (anecdotally) sleep more deeply.
I liked the sound of both those outcomes, so I gave it a try.
I can’t say much about the testosterone boosting — it wasn’t like I was walking around with an erection all day or anything, nor did I sprout Arnie-like biceps. But I did notice an effect on my sleep.
I got more REM sleep every night I took ZMA (20–25%) to begin with, but the boost decreased the longer I took it. I hypothesized that this is because I’d ‘caught up’ on REM sleep, but I don’t know.
I experienced very vivid dreams while taking ZMA, too. A quick look on bodybuilding forums (believe me, you only want a quick look) showed that many guys taking ZMA experienced “dreams that were trippy as f-ck maaaan”.
Graphs and Conclusions
Let’s look at the two areas I targeted — getting more total sleep (by falling asleep faster and staying asleep longer), and increasing my restorative sleep (REM and deep).
Below is the graph of my total sleep figures, in minutes. I’m aiming for as many days over 420 minutes (7 hours) as possible. You’ll notice that the first couple of months is erratic as I was experimenting, and I was rarely hitting my goal.
Then I manage to hit at least 420 minutes more often than not, from February onwards, as I began to understand what I needed to do to sleep more. Nice!
The next chart shows the percentage of my sleep that was light — I wanted around 30% of my total sleep to be REM and deep combined, so my target for light sleep was around 70% and under: the lower, the better.
Again you can see it’s all over the place to start with, before settling down around February (as total sleep did), as I learned what adjustments paid off.
In fact, I ended up averaging in the mid-60% once I stumbled upon the right mix of pre-sleep behaviors. I was getting tons of deep and REM sleep.
So was it all worth it?
I would say yes, absolutely. It’s not as if I would wake up and notice I’d got lots of REM sleep and think “yeah, I feel awesome” and skip out of bed. I didn’t feel much different day-to-day, but over time I certainly noticed some differences.
My workouts were more rewarding, in that I dropped body fat and gained muscle without changing my diet at all — the extra rest was clearly all my muscles and metabolism had needed all along. After years of stagnation, changing workouts, and dietary approaches, this was both great and a Homer-esque “D’oh!” moment.
I became more productive. I could do much more with each passing week of getting better sleep. I started writing. I built a website. I took on more training clients, where in the past I thought I was ‘busy enough’.
To be honest, I even became a better dad. So much of what is hard about parenting is due to a lack of energy—the energy you need to deal with a life with kids. With more sleep, I actually wanted to do more with the kids, I could be more attentive to listening to them, and I actively wanted to hang out with them. This was probably the best pay off of all.
I recognize I’ve just thrown a ton of findings at you, which is interesting and all, but how do my experiences help you if you’re as sleep-deprived as I was?
Here is what I—admittedly just a scientific sample size of precisely 1—would recommend you do to improve your sleep:
To increase your total sleep
- Don’t eat rich or sugary foods—or anything at all, to be honest—a few hours before bedtime.
- Take a break from screens before bedtime, and try to do something dull and less mentally stimulating.
- Exercise a few times per week—but not too close to bedtime.
- Try sleep aids (like nasal strips and eye masks) if breathing or sensory overload is an issue for you.
To increase restorative sleep
- Don’t drink caffeine after lunchtime.
- Don’t have more than one alcoholic drink per night.
- Consider taking ZMA for a few weeks at a time.
- Have more sex!
You don’t have to geek out on all this sleep stuff like I did, but hopefully you can see that a few regular small changes can have a really positive effect on your energy levels, productivity and well-being.
I’d love to hear in the comments about your own sleep advice and troubles, too.