How to take back control and use your Google Maps data ?

If you didn’t disable it, Google Maps Timeline stores your location and places you’ve been but fortunately you can download your “timeline”.

All my Google Maps data are stored on Timeline and Google processes it in order to create a timeline that I can find in the user interface : https://www.google.fr/maps/timeline. I can edit it, visualize it but I cannot compute statistics about it or visualize it in a different way.

Maps is using raw data (multiple points saved per minutes, depending of your network) and is projecting it on roads, places, etc.
You can easily export all of the raw data saved by Maps or you can export one day to KML but it’s impossible to automatically export all of the processed data from the user interface (you can download each day manually … Good luck if you want to do this).

Except if you are Google, or have a dataset with many places and roads, it’s really complicated to convert raw data (latitude/longitude and time) to places and way of moving. KML files are well-formatted data for every day, it’s exactly what we can see on the user interface. That’s why I wanted to automatically download each day into KML files and afterwards analyze and visualize them.

You can see the code here.

Download KML files

In order to export processed data from Google Maps website from a python script, you need to get your actual cookie (the same you have when you download manually one day).

How to get the cURL :

1. Go to https://www.google.fr/maps/timeline
2. Inspect the page and go to the Network tab
3. Enter this link https://www.google.fr/maps/timeline/kml?authuser=0&pb=!1m8!1m3!1i2017!2i3!3i16!2m3!1i2017!2i3!3i16 (or another date)
4. Save this element as a cURL
5. Open the cURL in a text editor

You should get something like this:

curl 'https://www.google.com/maps/timeline/kml?authuser=0&pb=!1m8!1m3!1i2017!2i3!3i16!2m3!1i2017!2i3!3i16'   
-H 'accept-encoding: gzip, deflate, sdch, br'   
-H 'accept-language: en-US,en;q=0.8,fr;q=0.6'   
-H 'upgrade-insecure-requests: 1'   
-H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) '   
-H 'x-chrome-uma-enabled: 1'   
-H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' 
-H 'authority: www.google.com'   
-H '**cookie**: gsScrollPos=; _ga=GA1.1.49937635TxMcGmJ-uXXXXX gsScrollPos='  
-H 'x-client-data: XXXQ==' --compressed

Finally, save the cookie content.

Then, you need to do a simple request at the good url depending of the day you want to save and save the request as a KML file. You can now loop over the days and months you want. month_url and day_url are numerical values (I have created a python script if you don’t want to do it by your own).

url = ‘https://www.google.fr/maps/timeline/kml?authuser=0&pb=!1m8!1m3!1i2017!2i{0}!3i{1}!2m3!1i2017!2i{0}!3i{1}'.format(month_url, day_url)

Visualizing data

I have then created a well-formatted Pandas data frame with my personal data.

Sample of the data frame for one day

As you can see, we get the same information than in the user interface : activity/place category (driving, walking, restaurant, grocery store, etc.). We also have place names, durations and dates.

We can do many things with this data. The main thing I am currently focused on is the time spent at doing/being something/somewhere.

During my time living in the US, I lived 1 month in one place then I moved out (to a place farer from my work). I wanted to see if there were a big difference of driving time and distance each day.

Driving time and distance

Although, the distance difference is not really important, the driving time is really different : I spent 30 more minutes each day in the car. Let’s see if there is an impact on my time at work.

I used to spent 8h19 at work before moving out, and now, I spend 8h03 there. (This graph is the time spent at the office, it’s not the difference between arrival and departure, so it doesn’t count time to grab food, restaurants, etc.).

Plotting on maps

I didn’t go really further on how to use all this data. My main goal was just to get a little bit more information about my trips and feel to have control on my data (even if I know that’s not really true). I just did one thing more : plotting points on maps.

I used multiples libraries to plot data on maps. One using Google Maps, gmplot which can be used to draw scatterplots and heat maps and save it as .html files :

And I also plot on basemaps (probably a geolocalization error because I never went swimming, moreover with my phone) next to Golden Gate bridge) :

As you can see, we can do many things with Google Maps Timeline data and the data saved by our phones. I think I can do many more things with this data but the current aim was to recover my location data and compute some stats about it. I will probably continue this project later.

Alexandre Attia
www.alexattia.fr