Using Tippecanoe, Tileserver GL , and Leaflet to serve scale-independent (and really cool looking) Raster (or vector) tiles at higher level zooms

Look at that detail ! Notice the level of detail in the streets at higher level zooms!
This tutorial is intended for beginners to the whole map technology stack, my intention is to save some time for you- if you are looking to generate a map that doesn’t look like a highway map but more like something with an “all streets” effect, then hopefully I can shed some light on the whole process, and share some of my experiences and point you in the right direction!

UPDATE: SEPT 29: found this article in my backlong , that was complete but not published, so some things might be outdated , reach out to me if your having issues with any of it , ill try my best to help out if i can .

Here we go!


The first thing we need is Data..

Tons of map data has been published on the Internet by various organizations , ranging from data you can have wrapped up in nice packages , or data that has the possibility of being corrupted that you can get for free off some ‘sketchy’ website.

you can hunt around the interwebs for some data and experiment with different types , but for the purposes of this tutorial i’m gonna use data from openmaptiles.org. (for basic map data of streets, water, land and buildings)

They pretty much have all the map data you could ever need in one place!
Make sure to give them a ‘shout out’ if your going to use their data in your projects!

Go ahead and download map data for New York City

The file you just downloaded is called an MBtiles file, its basically an optimized SQL lite database file capable of storing millions of vector tiles in one neat little package, created by the team at Mapbox , check out their definition if your interested in how it works!

MAPBOX: What are MBtiles?


Enter TileServer GL…

Alright now that we have an MBtiles data file of new york downloaded, we need to be able to host these tiles on a server.

There are a billion different implementations of tileservers out there, so don’t go and build one from scratch. The one that I used is a library called :

This tile server is awesome! It has a multitude of different features, also it can be hosted using docker on an EC2 instance , or on your computer. For the purposes of this tutorial lets just spin up a local host instance.

Pre Req:

Will need docker installed on whatever machine you plan on hosting the tileserver.

Theres a really great tutorial on openmaptiles.org for starting up a docker instance and viewing the mbtiles file we just downloaded. So , to save time typing I will just post a link to their documentation:

By this point you have a Tileserver Hosting an Mbtiles file probably running at localhost:8080 if you followed the documentation correctly.


Enter Tippecanoe..

Mapbox has their roots deep in the open-source community, founding their company originally by open source software and data.

They are currently the leaders in mapping technologies , and have tons of resources all over github for all of your mapping needs!

One of those tools is an open source projected called “tippecanoe

In my journeys I had the awesome experience of true open source when I ran into a few issues with using the library and one of the main contributors of the library (Eric Fischer) was kind enough to promptly respond to all my “noobish needs”. He took his time in breaking down the library, and even helped me come up with new ways to use the library to solve my probems!


What tippecanoe can do for us..

Now that we have an MBtiles file, we can actually just point leaflet towards that file and display a map without going through tippecanoe. But our map (without the power of tippecanoe) would look not that great at higher level zooms, something like this:

disappointing huh?!?

Time to leverage the power of tippecanoe.

Tippecanoe takes as input a Geojson file (or files), but the file we downloaded was MBtiles. Luckily , tippecanoe has a utility built inside of it that we can use to extract the data from the mbtiles and convert it into Geojson.

tippecanoe-decode:

We need to tell tippecanoe-decode to extract the data from the highest (closest, most detailed) zoom layer.

this command will tell it to extract at the highest zoom 14, and at the lowest zoom 14:
-z14 -Z14

..then we will extract 3 layers of data from the mbtiles file:

The following commands tells tippecanoe-decode to extract only the water/transportation/building source-layer from the mbtiles file
-l water

and

-l transportation

and

-l buildings

Where am i getting these layers from!!??!?!

our tileserver running at localhost:8080

zoom layer 10: opening original mbtiles file using Tileserver GL

As you can see, if we were to open the original MBtiles file(that we downloaded earlier) using tileserver GL or (almost) any other mbtiles viewer application, we can actually see a break down of the layer names. we feed these layer names into tippecanoe-decode and its smart enough to extract these layers out and turn it into geo json for us!

On linux, the tippecanoe-decode program will spit back geojson into the terminal , so we have to append it to a text document so that we can feed it back into tippecanoe.

The following command will output the data into a file called water-layer-z14.geojson for us:
> ./water-layer-z14.geojson 2>&1

So all together we will run three tippecanoe-decode commands on the original MBtiles file (new-york_new-york.mbtiles) that we downloaded:

Extract a geojson layer for transportation:
tippecanoe-decode -l transportation ./new-york_new-york.mbtiles > ./transportation-layer-z14.geojson 2>&1
Extract a geojson layer for water:
tippecanoe-decode -l water ./new-york_new-york.mbtiles > ./water-layer-z14.geojson 2>&1
Extract a geojson layer for buildings:
tippecanoe-decode -l building ./new-york_new-york.mbtiles > ./building-layer-z14.geojson 2>&1

okay Great! now we have 3 separate layer files extracted out from the original MBtiles :

Notice how much bigger the layer files are then the original mbtiles file! wowza!

Enter Tippecanoe …

In the same directory as those three Geojson files run this command:

sudo nohup tippecanoe -o recompiled_newyork.mbtiles -Z4 -z11 -f -F -ps -pf -pk -pc -pt -Bg -d14 ./transportation-layer-z14.geojson ./water-layer-z14.geojson ./building-layer-z14.geojson

This command is similar to the one for my project , except for my project i rendered all 14 zoom layers and a map that was not just 1 city, but the entire world, It took roughly a week on a XLarge Amazon Ec2

On my mac book pro running:
2.5 GHz Intel Core i5
8 GB 1600 MHz DDR3

started: 2:03pm

..the above command took about 20 minutes …so maybe go outside and fly a kite, or do some jumping jacks , or maybe scroll on some Tumblr feed, whatever it is.. make sure your computer is running while you are doing it.

BAM!!!!!!!!

Same zoom layer as before zoom layer 10, but notice the streets are densely drawn together

Whys it look so different?! What did Tippecanoe do to it!?!?!

Heres the command we ran from earlier
sudo nohup tippecanoe -o recompiled_newyork.mbtiles -Z4 -z11 -f -F -ps -pf -pk -pc -pt -Bg -d14 ./transportation-layer-z14.geojson ./water-layer-z14.geojson ./building-layer-z14.geojson

Notice i used nohup this command allows for us to run a linux process in the background, and has a neat feature that appends the output to whatever command you used to a file called nohup.out within the same directory

I used this command in conjunction with screen which allows you to create a new instance that doesnt break connection if you were to say , log out of your ec2 instance, it would continue running the command on the server even after your ssh connection was broken. (ends up being very useful if your file takes a couple of weeks to process.)

I digress….

Please let me know in the comments below if this article was useful to you at all! or if you have any questions , checkout my other articles on dev.to :D

happy coding!

https://dev.to/nodefiend