Mapping location data with Mapbox
It is no surprise that Google notoriously tracks user’s location information. When you are at home, when you drive to work, when you visit businesses, restaurants literally every single move you make. Of course, Google can only track this information if the user enables this setting against their account.
Google announced on 01 May 2019 that it is Introducing auto-delete controls for user’s Location History and activity data.
After Google’s announcement, I was curious to see what type of location data is stored against my account. I downloaded my data in JSON format at https://takeout.google.com. The zip archive was around 45 MB. I wasn’t surprised at this point but little did I know, I was about to be blown away. I downloaded the file and unzipped it. There was an 800 MB JSON file named
I tried opening the file in Notepad++, but couldn’t as the file was too big. Visual Studio Code was able to open the file and all I could see in the file was lines and lines of JSON code with no sight of ending. I scrolled to the end and found about 31 million or 31,079,341 lines of JSON to be exact. I was frankly shocked by the sheer amount of data Google had collected in a span of 6 years. Below is a small section of the file.
The file contains an array of locations. Each location data contains timestampMs in UNIX epoch format, latitudeE7, and longitudeE7 as long integer data type, accuracy as a number and an optional
activity that contains different types of activities the user might be performing like running, walking, driving, etc along with a confidence value.
After discovering the amount of location data I have against my account, I thought it will be interesting to visualize it on a map. Google maps timeline feature visualizes all time data points in the map, but I wanted to if I could build one myself.
I created a new React application and then installed Mapbox GL npm package (mapbox-gl) for react. The code below reads data from
data.json file in the src folder. After the react component mounts, it loops through each location item in the JSON file, extracts the latitude and longitude values, builds up a mapbox feature object, adds the item to the array and finally returns to the caller which is finally passed on as the data source for Mapbox.
Below is the screenshot of the application after it finishes loading. A few mock data has been mapped.
The application allows uploading a JSON file with Google’s Location History data. After uploading a JSON file with a lot more data, all the locations points are plotted onto the map.
I was curious to see if I could upload the entire
800 MB file to see what the mapping would look like. I was surprised that it worked and Mapbox was able to map all the 2,360,930 data points in under 10 seconds once all the data from the file was read, impressive!
I did a screen recording of the application to show how crazy it is to map all 2.3 million locations 🤪.
Want to try to try it for yourself?
Fork or clone
this-is-too-much-datarepo available on my GitHub account, run the application and upload a json file by clicking on the
Choose a file button.🙏