Creating static Choropleth map images with Mapbox and shapefiles
An alternative to Google map charts
Since the deprecation of the Google map charts API in 2012 there is a need for an alternative way to generate static images for choropleth maps. Here such alternative is discussed with using styled shapefiles in Mapbox.
- An account for mabox is needed, including secret and public tokens
- First we need to add a shapefile to mapbox
- Then we write (python) code to create a style
- And use the style to create a static map
- Create a choropleth map of geocaching founds, as an example of the usage of external data
Create a Mapbox account
Get access to the Mapbox features
A Mapbox account can be created at the signup page.
When an account is created you can go to Tokens to see your find your Default public token. This token is needed to retreive data and maps.
To create, edit and delete styles and other data you need a secrect token.
You can create one on the Access tokens page by clicking the + Create a token button. Give it a name and check at least the STYLES scopes as shown below.
Optionally you can enter URLs to restrict the use of the token to those URLs.
Click the “Create token” to, you guessed it, create the token. After you have confirmed with your password. The token is shown on the tokens page. Pleas take note of warning and do copy the token as this is the last time it will be shown to you!
You can now add shapes of countries and other territories to your Mapbox account.
Uploading shapefiles to Mapbox
How to put Natural Earth and other shapefiles on Mapbox
The first step in creating cholopleth maps is defining the shape of the countries (or other areas) we want to map. This method uses ESRI’s shapefiles. Finding an up to date shapefile can be a difficult or expensive. I’ve found the volunteers at Natural Earth are maintaining a set of public domain map data.
To get the data used in this example: Go to Downloads Select the Cultural data with the highest resolution and download the Countries data.
This will put all the data in a .zip-file. Coincidentally Mapbox requires the shapefiles to be uploaded in a zip-file. Go to Mapbox Studio and select Tilesets There the zipfile can be zipfile can be uploaded.
Attention: The shapefile needs to be converted. That will take some time, so the data will not appear immediately on the page. After some time the the following will be added to the tileset page:
As you can see an unique id will be added to the name of the tileset.
That is not the only id. Each tileset has also it’s own id. You can see that by clicking on the tileset. The tileset can be seen in the URL and in the field in the upper right corner of the screen.
This tileset can now be used in a Mabox style.
Create styles
Using JSON to create and edit styles in a (python) program
It’s possible to create and edit a style in Mabox Studio. To convert the data to a choropleth map automatic the Mapbox Styles API is used. The code examples are in python. The requests package is uses for http(s) request. Below wrapper code to send and receive JSON messages.
import requestsdef delete_request(url):
"""Send a delete request to the url, returns the json payload."""
r = requests.delete(url)
if 204 != r.status_code:
raise Exception(r.status_code, r.text, url)
return r.json()def get_request(url):
"""Send a get request to the url, returns the json payload."""
r = requests.get(url)
if 200 != r.status_code:
raise Exception(r.status_code, r.text, url)
return r.json()
def patch_request(url, data):
"""Send a patch request to the url, returns the json payload."""
r = requests.patch(url, json=data)
if 200 != r.status_code:
raise Exception(r.status_code, r.text, url)
return r.json()def post_request(url, data):
"""Send a post request to the url, returns the json payload."""
r = requests.post(url, json=data)
if 201 != r.status_code:
raise Exception(r.status_code, r.text, url)
return r.json()
Also functions to create, retreive (get), update and delete styles. The so called CRUD functions. These functions assume a global varialbe _MAPBOX_SECRET_KEY is defined. See the account page for how to get one.
_MAPBOX_SECRET_KEY = 'sk.aDiff3rentStr1ngWithRand0mUpperCaseAndLowerCaseCharactersAndNumbers.0fC0urseThese1sAreFak3'def create_style(username, style = {}):
"""Create a style."""
return post_request('https://api.mapbox.com/styles/v1/%s?access_token=%s' % (
username,
_MAPBOX_SECRET_KEY
), style)def delete_style(username, style_id):
"""Delete a style."""
return delete_request('https://api.mapbox.com/styles/v1/%s/%s?access_token=%s' % (
username,
style_id,
_MAPBOX_SECRET_KEY,
))def get_style(username, style_id):
"""Retreive a style."""
return get_request('https://api.mapbox.com/styles/v1/%s/%s?access_token=%s' % (
username,
style_id,
_MAPBOX_SECRET_KEY,
))def update_style(username, style_id, style):
"""Update an existing style."""
return patch_request('https://api.mapbox.com/styles/v1/%s/%s?access_token=%s' % (
username,
style_id,
_MAPBOX_SECRET_KEY,
), style)
The most basic style consist of a version number of the style. Currently version 8 is used. The name of the style. Attention: name is not a unique identifier. Several styles with the same name can exist, even for the same user! The unique identifier is the style_id. It will be returned (amongst other information) when the style is created. And also layers, metadata and sources. These last three can be empty.
The function below will make a basic style:
def make_style(
"""Make a style."""
name = ''
, sources = {}
, layers = []
, version = 8
):
return {
'draft': False,
'name': name,
'layers': layers,
'metadata': {},
'sources': sources,
'version': version,
}Use the code above to create a style.
# Define variables for later use
username = 'yourusername'
style_name = 'My First Style'
style = create_style(username, make_style(style_name))This will return a json object. Extra data data has been added to the style: Metadata like the id and the time the style was created/modified. The location for resources like glyphs and sptites have also been added.
{
"created": "2000-03-09T03:15:01.618Z",
"glyphs": "mapbox://fonts/mapbox/{fontstack}/{range}.pbf",
"id": "y0ur5tyle1d25alphanumchar",
"layers": [],
"metadata": {},
"modified": "2000-03-09T03:15:01.618Z",
"name": "My First Style",
"owner": "yourusername",
"sources": {},
"sprite": "mapbox://sprites/yourusername/y0ur5tyle1d25alphanumchar/an0ther25chars7o1dsprit3s",
"version": 8,
"visibility": "private"
}When looking at the style in Studio only a transparant image will be shown:
This is because there has no data been supplied. Layers, created from sources that have to be painted. The following functions are used to create those.
def add_sources(source_ids, sources = {}):
"""Add an array of source ids to the dict sources and return it."""
url = sources.get('composite', {}).get('url', '')
url += ',' if url else 'mapbox://'
sources['composite'] = {
'type': 'vector',
'url': url + ','.join(source_ids),
}
return sourcesdef make_layer(source_layer, paint, filter = None):
"""Make a layer."""
hash = hashlib.sha224(json.dumps(locals()).encode()).hexdigest()
layer = {
'id': 'layer%s' % (hash, ),
'paint': paint,
'source': 'composite',
'source-layer': source_layer,
'type': 'fill',
}
if filter:
layer['filter'] = filter
return layerdef make_paint(
fill_color = None
, fill_outline_color = None
, fill_opacity = None
):
"""Make a paint property."""
paint = {}
if fill_color: paint['fill-color'] = fill_color
if fill_outline_color: paint['fill-outline-color'] = fill_outline_color
if fill_opacity: paint['fill-opacity'] = fill_opacity
return paint
To get the map in the style, the shapefile that was added as a tileset in a previous section. There you can also find how to get the tileset_id and source_name
The python script below will add the shapefile to both the sources and the layers and will colour the countries gray with a white border. In the make_paint method you can set the colour of the countries and borders.
tileset_id = 'yourusername.t1ls3tid'
source_name = 'ne_10m_admin_0_countries-uniqid'style['sources'] = add_sources([tileset_id])
style['layers'] = [
make_layer(
source_name,
make_paint('#CCC', '#FFF')
)
]
style = update_style(username, style['id'], style)
Now the map is visible in the style:
It’s possible the preview isn’t updated. Click on “share your style”, the box with the arrow to see the updated style.
To give seperate colour, the shape needs to be added again to the layers in a different colour, but with a filter, so only the country matching that filter will be shown.
The following code will generate the filter that can be added to the layer:
def make_filter(value = 0, key = 'id'):
if key == 'id': id = ['id']
else: id = ['get', key]
return ['match', id, value, True, False]Like this:
style['layers'].append(
make_layer(
source_name,
make_paint('#009', '#003'),
make_filter(46)
)
)The polygon with id number 46 represents Brasil. Brasil is not visible in the thumbnail. Clicking the “Share your style” button shows the style with Brasil like the image below.
You can use Studio to find the id of countries, but the more easy way is to use the extra attributes included in the shapefile. Natural Earth adds many attributes. The code below shows how to use some of them in a layer:
style['layers'] = style['layers'] + [
make_layer(
source_name,
make_paint('#C00', '#600'),
make_filter('China', 'NAME')
),
make_layer(
source_name,
make_paint('#CC0', '#660'),
make_filter('AU', 'ISO_A2')
),
make_layer(
source_name,
make_paint('#0C0', '#060'),
make_filter('ESP', 'ISO_A3')
),
]
style = update_style(username, style['id'], style)This will give the following style:
{
"created": "2000-03-09T03:15:02.718Z",
"glyphs": "mapbox://fonts/mapbox/{fontstack}/{range}.pbf",
"id": "y0ur5tyle1d25alphanumchar",
"layers": [
{
"id": "layer2456eb061307eaabd00ce25a81c82e589618043c84b021cddf805fa8",
"paint": {"fill-color": "#CCC", "fill-outline-color": "#FFF"},
"source": "composite",
"source-layer": "ne_10m_admin_0_countries-uniqid",
"type": "fill"
},
{
"filter": ["match", ["id"], 46, True, False],
"id": "layer80d6c25a8edeb8d2629dfb976801f4e1b934751cea99bc0649ee4d6c",
"paint": {"fill-color": "#00C", "fill-outline-color": "#006"},
"source": "composite",
"source-layer": "ne_10m_admin_0_countries-uniqid",
"type": "fill"
},
{
"filter": ["match", ["get", "NAME"], "China", True, False],
"id": "layer340d44cd437040b6d5443346bae2e6ae67ea42d1dc7df7f148cd7d3d",
"paint": {"fill-color": "#C00", "fill-outline-color": "#600"},
"source": "composite",
"source-layer": "ne_10m_admin_0_countries-uniqid",
"type": "fill"
},
{
"filter": ["match", ["get", "ISO_A2"], "AU", True, False],
"id": "layere5e83e91e194bcceabaa00ddba7c0fda9fd047d60a6de8d29bed60e9",
"paint": {"fill-color": "#CC0", "fill-outline-color": "#660"},
"source": "composite",
"source-layer": "ne_10m_admin_0_countries-uniqid",
"type": "fill"
},
{
"filter": ["match", ["get", "ISO_A3"], "ESP", True, False],
"id": "layer1c816f39a246371acba4c8811b372cf72018b0b50897d9c9667c3d8c",
"paint": {"fill-color": "#0C0", "fill-outline-color": "#060"},
"source": "composite",
"source-layer": "ne_10m_admin_0_countries-uniqid",
"type": "fill"
}
],
"modified": "2000-03-09T03:15:02.718Z",
"name": "My First Style",
"owner": "yourusername",
"sources": {"composite": {"type": "vector",
"url": "mapbox://yourusername.t1ls3tid"}},
"sprite": "mapbox://sprites/yourusername/y0ur5tyle1d25alphanumchar/an0ther25chars7o1dsprit3s",
"version": 8,
"visibility": "private"
}This style can be used to create a static map.
Create the Static Map
Using the Mapbox Static Images API to get map images
When using the Static Images API (Form now on API for short) to generate images, many parameters have to be combined into a URL. The following function takes those parameters and returns the URL. By default, it generates a map of the whole world.
def mapbox_url(
username = 'mapbox'
, style = 'streets-v11'
, latitude = 0
, longitude = 0
, width = 512
, height = 512
, zoom = 0
, overlays = []
, access_token = None
):
"""
Generates the url for the static mapbox image.
:param username: The username owning the style
:param style: The style name to be used for the map
:param latitude: The latitude of the center of the map in degrees
:param longitude: The longitude of the center of the map in degrees
:param width: The width of the map in pixels
:param height: The height of the map in pixels
:param zoom: The zoom level used in the map
:param overlays: An array of overlays that will be added to the map
:param access_token: The MapBox public access token
:return: url to ret
"""
if not access_token: access_token = _MAPBOX_PUBLIC_KEY
overlay = ','.join(overlays)
if overlay: overlay += '/'
return 'https://api.mapbox.com/styles/v1/%(username)s/%(style)s/static/%(overlay)s%(longitude)s,%(latitude)s,%(zoom)s/%(width)dx%(height)d?access_token=%(access_token)s' % locals()To use our style, the username and style_id have to be given and the _MAPBOX_SECRET_KEY set.
username = 'yourusername'
style_id = 'y0ur5tyle1d25alphanumchar'
_MAPBOX_PUBLIC_KEY = 'pk.aRandomString0f5ixtySevenUpperAndL0werCaseCharactersAndNumb3rsPo1nt.andThenYet1other22M0re'
url = mappbox_url(username, style_id)This will show the map of the whole world with the four coloured countries.
Usually only a part of the world needs to be shown. Mapbox uses the Web Mercator projection.
To convert latitude and longitude to Web Mercator coordinates and back the following formulas (coded in Python) can be used:
def latitude_to_webmercator(latitude):
"""Convert a latitude (in degrees) to web mercator."""
return math.pi - math.log(math.tan((math.pi/2 + math.radians(latitude))/2))def longitude_to_webmercator(longitude):
"""Convert a longitude (in degrees) to web mercator."""
return math.radians(longitude) + math.pidef webmercator_to_latitude(web_mercator):
"""Convert web mercator to latitude (in degrees)."""
return math.degrees(2 * math.atan(math.exp(math.pi - web_mercator)) - math.pi/2)def webmercator_to_longitude(web_mercator):
"""Convert web mercator to longitude (in degrees)."""
return math.degrees(web_mercator - math.pi)
The coordinates in pixels are dependent of the zoom level. The code below calculates the zoom factor and center of the map for the given borders and size.
def get_zoom(web_mercator, pixel):
"""Get the zoom level from size in web mercator and number of pixels."""
return math.log(pixel * math.pi / web_mercator / 256, 2)def mapbox_dimensions(south, north, west, east, width, height):
"""
Get the parameters for a static mapbox image. :param south: The southern border (in degrees)
:param north: The northern border (in degrees)
:param west: The western border (in degrees)
:param east: The eastern border (in degrees)
:param width: The resulting width of the image (in pixels)
:param height: The resulting width of the image (in pixels)
:return: a dict with parameters for the mapbox_url function
"""
# convert to webmercator
south_wm = latitude_to_webmercator(south)
north_wm = latitude_to_webmercator(north)
west_wm = longitude_to_webmercator(west)
east_wm = longitude_to_webmercator(east) # size in web mercator
width_wm = east_wm - west_wm
height_wm = south_wm - north_wm zoom = round(
max(
0, # Zoom levels cannot be negative
min( # Calculate zoom levels for
get_zoom(height_wm, height), # heigth and
get_zoom(width_wm, width) # width
) # and take the lowest value
)
, 2 # zoom levels will be rounded to two decimal places
)
# Zoom factor to convert web mercator to pixels
zoom_factor = 256/math.pi * 2**zoom return {
'width': round(width_wm * zoom_factor),
'height': round(height_wm * zoom_factor),
'latitude': webmercator_to_latitude( (north_wm + south_wm) / 2),
'longitude': webmercator_to_longitude((west_wm + east_wm ) / 2),
'zoom': zoom,
}
The following borders will be used:
Direction Degrees Name North 53.550000 Mohe East 153.638889 Cape Byron South -43.643611 South East Cape West -73.984444 Serra do Divisor
The mapbox_dimensions function converts the borders to the center latitude and longitude, the zoom level, and the width and height of the image.
url = mapbox_url(
**{
**mapbox_dimensions(
south = -43.643611,
north = 53.550000,
west = -73.984444,
east = 153.638889,
width = 640,
height= 480
),
**{'username': username, 'style': style_id }
}
)Just as the Google API the size of the image is reduced so the borders in the code are also the borders of the image.
To add a margin to the image, calculate the image with a width and height two times the desired margin smaller.
To get the width and height you desire, add these to the parameters.
url = mapbox_url(
**{
**mapbox_dimensions(
south = -43.643611,
north = 53.550000,
west = -73.984444,
east = 153.638889,
width = 600,
height= 360
),
**{
'username': username,
'style': style_id,
'width': 640,
'height': 400,
}
}
)
It’s possible to add overlays to the image. The statement to generate a marker can be created with the code below:
def overlay_marker(
latitude
, longitude
, color = ''
, label = ''
, size = 's'
):
"""
Generate the partial url for a marker overlay
:param latitude: The latitude of the maker in degrees
:param longitude: The longitude of the marker in degrees
:param color: The 3- or 6-digit hexadecimal color code
:param label: The label, see MapBox documentation for the options.
:param size: The size options are 'l' (large) or 's' (small) :return: the url part for an marker overlay.
"""
name = 'pin-s' if 's' == size else 'pin-l'
if color: color = '+' + color
if label: label = '-' + label
return '%(name)s%(label)s%(color)s(%(longitude)s,%(latitude)s)' % locals()
Mark the Olympic cities in the countries:
Olympic year City Latitude Longitude 2016 Rio de Janeiro -43.205916 -22.911366 2008 Beijing 116.397500 39.906667 2000 Sydney 151.209444 -33.865000 1992 Barcelona 2.183333 41.383333
By adding the overlays:
url = mapbox_url(
**{
**mapbox_dimensions(
south = -43.643611,
north = 53.550000,
west = -73.984444,
east = 153.638889,
width = 600,
height= 360
),
**{
'username': username,
'style': style_id,
'width': 640,
'height': 400,
'overlays': [
overlay_marker(-22.911366, -43.205916, '66F', 'r'), # Rio de Janeiro
overlay_marker( 39.906667, 116.397500, 'F66', 'p'), # Beijing/Peking
overlay_marker(-33.865000, 151.209444, 'FF6', 's'), # Sidney
overlay_marker( 41.383333, 2.183333, '6F6', 'b'), # Barcelona
]
}
}
)That gives the final result:
Using external data
Creating Choropleth maps of geocache finds
The reason developed this module is because I want to update my geocaching profile with a colourful map. You can get a so called pocked query of all the caches you have found. A pocket query results in a GPX-file.
Since a GPX-file is a XML-file, the data can be extracted with ElementTree:
import xml.etree.ElementTree as ET# Set the location of the pocket-query file
pocket_query = '../data/pocket-query.gpx'# Initialize a dict of countries
countries = {}tree = ET.parse(pocket_query)
# Each wpt contains one country tag, loop over all the country tags
for country in tree.findall(".//{http://www.groundspeak.com/cache/1/0/1}country"):
# get the name of the country
# and increase the value of that country in the dict
countries[country.text] = countries.get(country.text, 0) + 1max_value = countries[max(countries, key=countries.get)]
Define a function to generate a nice range of colours.
def color_from_value(val, max):
c = math.log(val) / math.log(max_value)
h = (1-c) * 120
# the paint property can also be hsl (hue, saturation, lightness)
return 'hsl(%d, %d%%, %d%%)' % (
h,
100,
37.5+12.5*math.cos(math.radians(h))
)Use the data int the countries dict to generate a set of coloured layers and use that in a style.
# define the name and id of the tileset
tileset_id = 'yourusername.t1ls3tid'
source_name = 'ne_10m_admin_0_countries-uniqid'# Initialise the layer array with a grey landmass
layers = [
make_layer(source_name, make_paint('#CCC', '#CCC')),
]# For each country append the coloured layer to the array
for (name, value) in countries.items():
layers.append(
make_layer(
source_name,
make_paint(
color_from_value(value, max_value),
'#CCC',
),
make_filter(name, 'NAME'),
)
)# Make the style
style = make_style(
stylename,
add_sources([tileset_id]),
layers
Use the style to create a map:
# Set the keys in the global module variables
set_mapbox_token(
public_key = 'pk.aRandomString0f5ixtySevenUpperAndL0werCaseCharactersAndNumb3rsPo1nt.andThenYet1other22M0re',
secret_key = 'sk.aDiff3rentStr1ngWithRand0mUpperCaseAndLowerCaseCharactersAndNumbers.0fC0urseThese1sAreFak3',
)# Check if there is already a style with the name
style_id = get_style_id_by_name(stylename, username=username)
if style_id:
# Update if the style already exists
style = update_style(username, style_id, style)
else:
# Create the style if it's not
style = create_style(username, style)
# Determine the url
url = mapbox_url(
username = username,
style = style['id'],
latitude = 20,
longitude = 10,
width = 640,
height = 560,
)# Print the url
print(url)
This gives (for me) the following image:
