My open source brain data
I want to share an experiment I’ve been doing on and off for the past year or so.
I’ve been doing exercises on Lumosity and recording the scores as well as some meta-data in Google docs to track how certain things affect my mental performance.
The idea came from a conversation I had with a friend who told me you could use Lumosity not just to improve your mental performance (although you could argue that those claims are dubious), but also as a convenient way to track mental performance.
I’ve always been interested in questions like:
- Do I work better at night or in the morning?
- Does caffeine make me smarter?
- Does caffeine make me stupider when I don’t take it?
- How much does sleep affect my mental performance?
Finally I have a way to answer questions like this.
So here’s how I’ve been doing it and I’d love to get thoughts and feedback:
Each day I do five exercises from each Lumosity category – Speed, Memory, Attention, Flexibility, and Problem Solving – and record my absolute score (not Lumosity Performance Index) as well as some meta-data. This lets me approximate some sort of aggregate mental performance metric (though I’m sure it doesn’t capture everything).
In terms of meta-data, every day I’m also tracking of additional factors that I think may influence my test results. Right now this includes what time I did the exercises, when I woke up, how long I slept the night before, whether I ate breakfast, whether I’ve drank coffee, whether I’ve taken Adderall, whether I’m wearing glasses, whether I’ve taken Creatine, and a few others.
I’m slowly adding more things to keep track of as I come up with them (let me know if you have any suggestions). For example, I recently hypothesized that I might perform better when it’s silent so I’m starting to track whether I’m in a silent location or not.
I’ll share my preliminary results and then a few concerns I have.
So what do the result show?
- For one thing, coffee didn’t have as much of an effect on me as I thought. On average, my test scores increased about +2% after drinking coffee. It had the biggest effect on attention (+4%) and the least on flexibility (-1%).
- Adderall on the other hand gave me about twice the performance boost of caffeine, increasing my test scores about +4% on average. It had the biggest effect on problem solving (+22%) and the lowest on flexibility (-2%).
- The affect of eating breakfast is actually about the same as caffeine. Without breakfast I perform 2% worse on exercises, with the biggest difference being attention-based tasks (-6% without breakfast).
- It also turns out I’m not a morning person. My morning test scores (before noon) are consistently lower than afternoon or evening, with a total average score of -3% less and individual scores ranging from -1% (memory) to -5% (flexibility and problem solving)
- Unfortunately hours slept doesn’t really indicate much, as most of my data points are at around 6, 7 and 8 hours slept, and there are very few at 5 or below.
I’m not super great at analysis so I’d love to open up the data to others to see what you can come up with and whether you have any feedback on the way I’m currently tracking things. Here’s the Google doc.
A few concerns:
A lot of people have asked whether increases in scores may indicate that I’m just getting better at the games. I tried to eliminate that as much as possible by playing them until I my scores stopped increasing with each session before starting to keep track. On top of that, if I’m alternating some factor every other day, then I’m hoping this will eliminate the effect that learning the game might have on scores when.
I’m still slightly concerned about some of the ways that Lumosity tracks scores. They’ve been criticized for unlocking advanced levels as you progress, thus giving you the illusion of progress without any real progress. I’ve tried to pick exercises where this is less likely to be an issue, but for Memory Matrix and Chalkboard Challenge I think they still mess with the game mechanics a bit too much.
For example, with Memory Matrix, you’re given a certain number of tiles to remember and if you get them all right, the next time you get an additional tile. The number you start out with depends on the number you got right the last time you played. I’ve also noticed that my ability to remember the location of all the tiles depends on the character of how they’re layed out (whether they make up coherent shapes) – which I have no control over.
Another weird factor is that Lumosity gives you bonus points at the end of your exercise for getting strings of correct answers in a row. Though my thinking at this point is that it’s not too much of an issue so long as it’s consistent across exercises.
Interestingly, I can show that the weirdness in scoring of the two exercises mentioned above actually messes with the the normal distribution pattern of my scores (you can see this on the second sheet). While the Speed, Attention, and Flexibility game results are fairly normal:
Memory has a few strange deviations and Problem Solving is all over the place.
Anyway, I hope this post has inspired others to do the same (I’d love to see the aggregate results) and I’m also looking for feedback as to how I can do this better and ideas for what I should be testing for. I plan to keep tracking my results for the foreseeable future.
