Animated routes with QGIS

I have been posting these GIF animations with the theme Optimal routes from x to n locations already for a while to my Twitter account and some people have been asking me to write a blog post on how to do those. Well, here it finally is. The prerequisites for doing this are some experience in QGIS and knowledge how to run/modify basic Python sciripts.

This tutorial consists of three parts

1. Preparing the data (QGIS, Graphhopper)

2. Calculating the routes (Graphhopper with Python)

3. Visualizing the routes (QGIS Time Manager & GIMP)

Part 1: create the data used for routing

First you need to selectthe area where you are creating your animation from. Performance-wise with a normal laptop a city is really light, region is ok and a country still doable. Processing a whole continent already requires some heavy lifting. Based on my tests, my 16 GB RAM Dell laptop could process OpenStreetMap pbf files up to about 2 GB (edit: Graphhopper noted: You can process bigger files with GH (use JVM param -Xmx9g)).

When you have decided on your area, go to Geofabrik website and download the OSM pbf extract from the area of your choice. For this tutorial I am using Sweden pbf. You will need this file in the next part of the tutorial.

The process with QGIS starts by creating the endpoints for the routing engine. The amount of endpoints also affects your processing time. You can use any kind of points and you can have as many of them you like. I have tested doing these with anything from a few point visualization up to a 10 000 endpoint version. In this demo case we are creating just a set of gridded points inside the country with QGIS vector tools. Open the processing toolbox in QGIS and select the create regular points tool.

From QGIS processing tools you can find the regular points tool.

To set the input extent you can use the bounds from the canvas or from a layer. Here I am using a Natural Earth country polygon for Sweden. Point spacing is here in degrees, so it will create a point for every 0.5 degrees in Sweden and end up with modest amount of points.

Once you have created a shp grid that suits your needs, you should save the grid to a csv file. Just by right-clicking the previously created layer and Save as, then selecting the file format to csv, and setting the geometry to AS_XY (default) you have your file ready for the Python scripts.

If you have any other way to produce a csv file with lat long coordinates, you can of course to use those methods. The main point (sic) is that you end up with this kind of a file:

Here is a screen capture of the file in Notepad++.

Now you are ready to set up yourself a routing engine!

Part 2: API calls with Python to generate routes

Now it’s time to set up your local routing engine. Graphhopper has made it super easy for you. First go to the Graphhopper website and download the zip for the routing engine.

After that just follow the instructions on the Graphhopper

Quickstart

The following steps are simpler and only need the JRE, a jar file and an OSM file.

Install the latest JRE and get the zip of the GraphHopper Web Service

Unzip it and copy an OSM file into the created directory. For example berlin-latest.osm.pbf

Start GraphHopper Maps via: java -jar *.jar jetty.resourcebase=webapp config=config-example.properties datareader.file=berlin-latest.osm.pbf

After you see ‘Started server at HTTP 8989’ go to http://localhost:8989/ and you should see a map of Berlin. You should be able to click on the map and a route appears.

So for me to get this started I just opened the Windows command prompt and ran the following command:

C:\Users\tjukanov\Documents\graphhopper\graphhopper-web-0.9.0-bin>java -jar graphhopper-web-0.9.0-with-dep.jar jetty.resourcebase=webapp config=config-example.properties datareader.file=sweden-latest.osm.pbf

It takes on my laptop about a minute for Graphhopper to start for the first time with Sweden pbf. Larger country files naturally take longer. Like the Graphhopper tutorial says, after the engine has started you can go to your browser and check that it works. But now we want to use the API that Graphhopper offers and we are going to use a few Python scripts to do that.

You can check the following scripts from my GitHub. I am using a few packages that you might have to install (pandas & gpxpy). I am just running the scripts through my Anaconda Spyder.

The API_calls .py runs in just a few seconds and saves the files to disk. The GPX parsing and combining the csv files takes about five minutes for Sweden, and 219 routes.

Disclaimer: I’m by no means a Python expert and these scripts are actually first time I’m using Python for something proper geospatial stuff. So a real Python developer might do this very differently and more efficiently, but these scripts work for this purpose very well. I am more than interested in hearing any feedback regarding the scripts.

Part 3: visualizing the results

Now that you have a single complete csv file, you can open the file to QGIS. The x and y fields are automatically detected from the file, so basically you don’t have to do anything besides selecting the correct coordinate system (WGS-84 EPSG:4326) after you click OK from the screen below.

All the route points you have just created now appear to the map. Here’s how the ones from Sweden look like when overlayed on OpenStreetMap background map.

Next comes my favourite part: the styling! You can be as creative as you can. Try out different icon sizes, alpha levels and feature blending modes to get the result that you like. I first set the background color to black, make the points a bit smaller, change the coordinate reference system of the project to a better local version and change the feature blending mode to addition. After those changes my dataset looks like this:

Then time to bring out the most important tool of the tutorial: QGIS Time Manager. It’s a great plugin developed by Anita Graser and you can read more about it here. Install the plugin through QGIS plugins dialog.

Note that here it might be more effective to have the data in some other format (e.g. in PostGIS) but to keep this somewhat simple, we will use the same csv file to create the animation.

In Time Manager settings go click Add Layer and select your csv file. Then select which column has the time attribute, which is t in this case. QGIS then automatically detects the time format. QGIS loads the data for a while and then everything disappears from your screen. But don’t worry, because this is actually the first frame of your animation! You can use the slider on the toolbar to browse through the file. Adjust how long one frame is from the Time Manager toolbar. I am using here 15 minutes per frame.

One nice styling detail that I often use is to duplicate the layer you are visualizing, style it differently and show the other as cumulative. You can choose this option from the Time Manager settings. From the same settings you can also modify how you want to see the time on the canvas. I didn’t want to add time to this animation so I just removed it.

Exporting the frames can be time consuming if you have a lot of endpoints in your file. For my example layer, it took about 5 minutes to export all the frames. Also note that it might be good to clean up the data before you start the export! For example if there are some islands in your area and the routes there take twice as long as everywhere else in the animation, this might not work well in the final animation (and you also end up having a lot of empty frames) so it might be a good idea to delete those features.

When you have exported all the layers, we add them together to create one single GIF. You can do this with some online tools, Adobe Photoshop or with GIMP which I am using here. So if you don’t have GIMP installed on your computer, go ahead and download and install it.

You should open up all the frames with the GIMP “Open as Layers” option from the main file menu. Browse to the folder where you have exported all the Time Manager frames and select all of them. GIMP creates a new project with all of your images as layers.

If you have a lot of frames, say + 500, might be a good idea to optimize the file a bit. Go to filters → Animation → Optimize (for GIF) and GIMP will automatically reduce the resulting file size. This is optional and of course depends on how you are going to use the final file.

To export the image, just go to File → Export as in GIMP and select GIF as the final format. From the GIF export options you can choose whatever delay you think works best on your visual. Usually with these animations, I have used something around 100 milliseconds.

Export the GIF as animation from the GIMP dialog

Just export it annd there you have it! Your final GIF showing the optimal routes from your starting point to all your listed endpoints.

What is the point in doing this, you might ask? Well, there is none. It just looks cool. Of course developing this further might show you interesting patterns from urban areas, poorly accessible places or something else related to movement and urban fabric.

But if you make one of your own version, have any ideas onhow to improve the process or any other feedback, just contact me through Twitter!