The Difficulty with Tracking Sleep
As part of a class (6.08: Introduction to EECS via Interconnected Embedded Systems), my team developed a way to use movements to track the quality of your sleep.
“One cannot think well, love well, feel well, if one has not slept well”
Overview
We created Sleepsense, a device that aims to help you better understand the trends in your sleep so you can modify your habits to become a healthier person.
After pushing a button when you go to sleep, our embedded system will begin recording movement, filter and store the values, then periodically send the values to a database in our server. The recording stops when you push the button upon waking. The data is then processed, analysed and nicely graphed on a website that you can access for your sleep history.
For every night, you’ll get a personalized number rating the quality of your sleep to allow you to compare nights and discover what sleeping habits are giving you the most effective nights of sleep. Equipped with a rechargeable battery, your device will be able to record data over the course of the night wirelessly to ensure that you can place it on your mattress in a place most convenient for you.
Design
Our system can be separated into the hardware, pre-processing software, post-processing software, and the website.
The hardware consists of the ESP32, LEDs, IMU (Inertial Measurement Unit) and a button.
The pre-processing software is all contained on the arduino and written in C/C++. It consists of files that record the acceleration rate every second from a running average filter and sends the data up to the server database every minute.
The post-processing software is written in python and is hosted on the 6.08 server. It pulls selectively from the database, filters and processes it, creating graphs and an overall sleep rating.
The website is written in a mixture of javascript, html, and python, and is also hosted on the web server. It allows a user to log in and input the date of their sleep and hosts the graphs.
The completed breadboard should look like this:
Let me know if you’d like a more detailed outline of the required connections.
Discussion of Design Challenges
Here were a few different technical challenges that required the most time and thinking to resolve:
Hardware
On the hardware front, it was essential to include indication to the user that a sleep session had began, as well as indicate the battery life. Incorporating the LED’s proved to be slightly complicated because of our limitations on power use and blocking code to keep the LED blinking every certain time interval. Initially, we connected the LED to the battery management board and controlled the brightness by altering the voltage. However, as we learned, the dimmer the LED is set to be using the battery board, the more inefficient (you’re using less of the total power to power the LED). We needed the ability to control the brightness, so the new approach was to use PWM in the period of the time when the LED flashed ‘on’. Using duty-cycling during the blinks proved to give us control over brightness, save power, and use power in the most efficient way.
Another issue that came up was our function for measuring battery life. Use of a bisection search could cause the system to get stuck in an infinite loop in cases where the search is unable to hit the precision in terms of finding a corresponding voltage in the model. Thus, we introduced a discharge_counter to ensure that if the intervals are halved more than 100 times, the closest value will be returned instead of continuing to reduce the interval size.
...
}else {
while(abs(voltage_enter-voltage_from_discharge(mid)) > error) {
discharge_counter+=1;
mid = (hi+lo)/2.0;
if((voltage_enter > voltage_from_discharge(mid))){
hi = mid;
}
else {
lo = mid;
}
if(discharge_counter>=100){
return mid;
}
...
Pre-Processing
On the pre-processing front, one difficult part was determining the best sample rate for our accelerations. We began to update the running average every 300ms, but this rate did not really provide enough temporal resolution to capture all movement during the night. We then decide to decrease this loop period all the way down to 10ms to provide more detail. Using 10ms period did not produce significantly different results from our tests using 100ms period, but we decided to stick with it since the power consumption is more or less the same either way. We also changed the weighting of the running averages on our measure() function (explained below) to take into account the more rapid updates. The weights were decided in a way such that every time we save the value it has kept about 20% of the old value and the rest 80% is new. This is a trade off, since if we keep less of the old data, it also means that we are going to keep less of the first measurements in the window. Since we store the value of the average once every second, there’s 100 updates to the average before we store, and 0.985¹⁰⁰=0.22.
This helped us overcome the problem of achieving high temporal resolution while using memory efficiently. Another problem we faced with preprocessing was being able to normalize the data. Having a list of raw values fluctuating very close to 1 is not very useful, and if we average things out we might lose movements, since acceleration tends to go up and down, meaning we will lose the signal if we take the average. To get the numbers close to zero instead of 1, we want to subtract the background acceleration, however this is not easy because the value of background acceleration the IMU reads is not constant, and it would result in uncomparable if we just subtracted some general average value. To combat this, we implemented a very slow moving running average of the acceleration to be used as baseline. Using that we can measure the actually interesting part of acceleration by taking the difference from this baseline. The absolute value of this difference is then used to update the running average of our signal as described above. We take the absolute value to not lose any data to averaging it out and since We don’t really care about the direction of the acceleration. The difference from baseline ends up being random noise around 0, with occasional spikes when there is movement. Taking the absolute value moves of everything above 0, and since we’re averaging, most of our signal ends up being small fluctuations around a small value that represents the usual change due to random noise.
The code used is shown below:
void measure(){
imu.readAccelData(imu.accelCount);
accel=sqrt((pow(imu.accelCount[0],2)+pow(imu.accelCount[1],2)+pow(imu.accelCount[2],2)))*imu.aRes;
mean_accel =0.9998*mean_accel+0.0002*accel;
mean_change = 0.985*mean_change+0.015*abs(accel-mean_accel);
}
...
loop(){
...
if(millis()-timer>10){
measure();
timer=millis();
}
}Post-Processing
On the post-processing front, determining the best way to transform raw acceleration data to a meaningful graph, and then determining a rating for the quality of the night itself were two significant challenges we faced.
We ended up using an hour-wide sliding window (3600 seconds) that would move by intervals of 5 minutes(300 seconds):
avg_values=[]
new_times=[]
for i in range((len(temps))//300):
start=300*i-1800
end=300*i+1800
if 300*i<1800:
start=0
if len(temps)-300*i<3600:
end=-1
length=len(temps[start:end])
avg_peaks.append(sum(binary_peaks[start:end])/length)
avg_values.append(sum(new_temps[start:end])/length)
new_times.append(times[300*i])We then used avg_values and new_times to create a plot of the density of sleep movements:
We also fooled around with different discretization windows. Here is an example of a 10 minute jump:
And a 100 second jump:
We eventually decided on a 5 minute window since it looked the cleanest:
For our rating system, we tried a few different approaches: using a Long Short Term Memory recurrent neural network to predict the next acceleration values, taking the correlation coefficients between better and worse nights of sleep, and even taking the norm of a covariance matrix between the different acceleration signals, but all these failed a rating system that could successfully distinguish between clearly better and worse nights.
Ultimately, we created a model that broke up the night into 30 minute periods of sleep, and calculated the movement densities for each period. After some fine tuning, we set ‘density thresholds’ for each period for what would be considered ‘restful’ vs ‘restless’ sleep. We decided that 4 periods of restless sleep a night was regular and also added the bias towards at least 8 hours of sleep.
for period in peak_density:
if period > 0.003: #good 30 minute period of sleep
restless_periods += 1 good_hours = (total_periods - max(0,restless_periods-4))/2 # 4 restless periods are ideal
Energy Management
Managing energy consumption played an important role for our project since it’s critically important that our system is able to last through the whole night, and because we want to continuously take measurements it is not possible to just turn off the ESP when we are not using it. First step towards reaching that was to get a 1200mAh battery instead of 350mAh. We also removed all the unnecessary devices from our board like the OLED, gps and microphone. This allowed our system to run for around 7 hours before the battery ran out without any form of energy management.
We then developed a better way of storing data in order to send to the server only once every 60 seconds instead of every 10 seconds, and turning of the wifi for the remaining interval. In addition to those we desoldered the red LED on the ESP that indicates when it’s on both to save power and make it more pleasant for usage at night. However we needed some indicator of whether it’s recording or not, so we added an LED that we run with duty cycling code. After these changes our system was able to last for 10,5 hours and still about one third of its battery remaining assuming the voltage-discharge curve is similar to the 350mAh battery, giving us an extrapolated battery life of 16 hours.
This about half of what it should theoretically reach based on the following calculations. Current usage of components and modes (from documentation on the class website):
WiFi 190 mA when sending, 100mA when receiving 25 mA ESP when running in normal mode 730 uA for IMU accelerometer Led uses 20 mA when on (TA estimate), its effect on total power consumption is negligible, because it’s only on for less than 1/4000th of the time due to duty cycling
Assuming the WiFi is sending for the half of the 5 seconds it takes to turn on and send the data to server and receiving the other half, average wifi current usage is 145mA*5/60=12.1mA. Total current=wifi+ESP+IMU=37,8mA. Theoretical battery life= 1200mAh/37.8mA=31.7h. It is clear that there are some inefficiencies in real life that are not displayed on documentation but a this difference seems rather large.
We then measured the actual current usage in different modes to try and identify what causes the difference between the above results by measuring the voltage difference between LDO 1a and 1b as well as LDO 2a and 2b while the device was operating on battery power. On measuring mode, the voltage across was 15mV and 17mV respectively. There is a 0,5 ohm resistor inside the B outputs, so we can calculate the current going through using Ohm’s law, I=U/R, therefore this translates to total current consumption of 64mA while measuring, which is significantly higher than the theoretical consumption, perhaps our ESP is running in fast mode or something. When our device sends information, the voltages rise to around 36mV and 33mV and falls back to measure usage over the course of around 3 seconds. This leads to peak current usage is around 136mA, which is somewhat in line with documentation. Based on these measurements we can make another calculation of how long the battery should last. Assuming the wifi uses 120mA for 3 seconds every 60 seconds, our average current is 64+(120–64)*3/60=66.8mA, which should give us battery life of 18 hours. This is well in line with our experience.
Website Design Challenges
The most pertinent design challenge was making the site “unbreakable.” There are many things that can go wrong when a website is self-referential, and controlling for all possible variables, such as users manually changing the URL, is almost impossible. However, the website currently works for all tested combinations of button presses and URL changes, and has multiple error pages that redirect the user in a smooth and understandable way. For example, if a user were to enter an incorrect password, he or she would encounter a “Wrong Password” page. If he or she were to request a user that is not in the database, a “No Such User” page would be thrown. If the user requests a sleep_id out of the range of the database, he or she is redirected to a “Out of Bounds” page. Furthermore, handling passwords is done for every page change, so users cannot reference other users’ data without the relevant password.
When designing the website, a major barrier encountered was the fact that the entire website had to be split up into strings within the Python code. Having the HTML, CSS, and JavaScript all be one green blob of text that did not have syntax highlighting, autocomplete functions, or the other amenities normally afforded to programmers was challenging.
Finally, the website was originally planned to be one functional site that would update its graphs, data, etc. through JavaScript functions. Once implemented, it was discovered that the Bokeh graphs would need to be referenced with Python functions, and it was necessary to call Python files through JavaScript. Instead of taking the dive into yet another new skill set, AJAX, it was decided that the handling of passwords, etc. would be done through Python, and the display would change based on the request variables given to the script.
Results
For external reference, here is a comparison of what the Sleep Cycle app on the App Store displays compared to ours
(Our rating system gave this 43.75%)
A pretty good visual correspondence! Our scaling is slightly different — the final dip in the Sleep Cycle graph isn’t in ours — but our peaks overall correspond pretty well.
As for the sleep rating, our model is broadly correct in the sense that it will give a good rating(>60%) to nights that we feel were restful, but there are some nights that are rated as especially bad when really we feel that they were reasonable nights of sleep (tuomas’ 100th sleep session).
More work could be done in the fine tuning of our actual sleep rating. If we wanted to be scientific about it, we could use other body measurements (using an Electrocardiogram and brain waves to adjust the measurements).
Our final graphs look like this:
We had fun implementing this project and hope this documentation inspires you to do something similar!
