In this post, I’ll walk you through 3 different Population Density datasets.

The study area is Kraków, and the datasets are available for Poland, the EU and the whole world.

This post is for you if…

1. You just love maps.
2. You’re interested in global grids.
3. You’re researching population density.
4. …or you ❤ Kraków like I do!

The Inspiration

A favorite geospatial facebook page of mine, bitgeo, recently published a map of building heights that captivated me.

Seeing the city laid out so neatly felt refreshing. Moreover, the building heights, reveal some of the cities secrets…

Average Number of Floors in Residential Buildings // Kraków

If you look closely at the map, you’ll see:

• The historic core’s three story buildings (story cover photo),
• Residential neighborhoods with their 6+ story apartment buildings,
• And finally the suburbs represented by single-story houses

Building heights made me curious about Population density. Thus, I set off the answer the question: where can I find population data for Kraków?

Population grids

We’ll explore three population grids.

1. Poland Geostat Population Grid (Poland)
2. Global Human Settlement Population layer (Worldwide)
3. EU Joint Research Center population grid (Europe) — Winner 🥇

Let’s go!

Dataset #1. Poland Geostat Population Grid

First up is a vector grid representation from the Polish Geoportal. On the map you see the Kraków city center, southern districts, and eastern Nowa Huta district as the most densely populated.

At 1km resolution, the grid is too large for comfortable city-level analysis. However a redeeming factor is the demographic information as well as the standardization of the grid and availability across other Polish census datasets.

1. Short description
   First source to check, directly from Poland’s geoportal.
2. URL
   https://geo.stat.gov.pl/aktualnosci/-/asset_publisher/jNfJiIujcyRp/content/nowe-dane-demograficzne-w-siatkach-kilometrowych-dostepne-w-formacie-shp
3. Coverage Area
   Poland
4. Grid resolution
   1x1km, vector data
5. Data describes
   - Population per kilometer
   - By gender
   - By age group (0–14, 15–64, 65+)
6. Years available
   2011
7. Methodology
   Aggregated census data in grid
8. Projection
   EPSG:2180 — ETRS89 / Poland CS92 — Projected
9. File info
   116 mb Shapefile, covering all of Poland
10. Usefulness rating: ⭐️⭐️⭐️ (3/5)

2. Global Human Settlement Population layer

Next up in the analysis, is data from the Global Human Settlement project. GHS uses satellite imagery in combination with population counts to produce this estimated dataset. This methodology produces some rather large inaccuracies on our map, where a steelworks in the eastern part of the city looks to be as densely population as the city center.

On the plus is the worldwide study area and standard methodology. On the negative is the relatively poor data quality in comparison with local alternatives.

1. Short description
   World-wide dataset allowing study of population change at various data points.
2. URL
   https://ghsl.jrc.ec.europa.eu/ghs_pop2019.php
3. Coverage Area
   Worldwide, broken into regional chunks (may need multiple files to cover study area)
4. Grid resolution
   250m, 1km, 9 arcsec, 30 arcsec (Raster)
5. Data describes
   Inhabitants per cell in decimals (float)
6. Years available
   1975, 1990, 2000 and 2015
7. Methodology
   Per the source: “disaggregated from census or administrative units to grid cells, informed by the distribution and density of built-up as mapped in the Global Human Settlement Layer (GHSL) global layer per corresponding epoch.”
   Translation: using satellite data to inform administrative unit
8. Projection
   World Mollweide (EPSG:54009)
9. File info
   250m grid: ~10 mb per regional file
10. Usefulness rating: ⭐️⭐️ (2/5)

Dataset #3. EU Joint Research Center population grid

The final map is the best of data source for Grid Level population data that I found. The Joint Research council publishes this dataset across all of Europe and is composed of actual census data.

Surprisingly, it’s data that is available at the smallest resolution — better than Poland’s own geoportal. Using the data, we can produce a lovely street-level population density map. A small minus is the file size (unwieldy) and need to crop study data.

1. Short description
   EU-sponsored dataset of population grid based on census data, aggregated to 100m grid.
2. URL
   https://data.europa.eu/euodp/en/data/dataset/jrc-ghsl-ghs_pop_eurostat_europe_r2016a
3. Coverage Area
   Europe
4. Grid resolution
   100x100m grid (Raster)
5. Data describes
   number of people per cell
6. Years available
   2016, based on 2011 Census data
7. Methodology
   “Resident population from censuses for year 2011 provided by Eurostat were disaggregated from source zones to grid cells, informed by land use and land cover from Corine Land Cover Refined 2006 and by the distribution and density of built-up as mapped in the European Settlement Map 2016 layer.”
8. Projection
   EPSG 3035
9. File info
   Single ~500 MB raster for Europe.
10. Usefulness rating: ⭐️⭐️⭐️⭐️⭐️ (4.5/5)

What’s next?

That’s all for population density mapping at the moment. I’m interested in transport and people, so stay tuned for more maps coming up.

Let me know what you think! Are there any other sources I should check out?

See also:

A pixel level evaluation of five multitemporal global gridded population datasets: a case study in Sweden, 1990–2015

Quicktime is a great tool to make screen recordings on a Mac, for example for a product demo.

The problem is that Quicktime creates files in the MOV format, which take up a lot of disk space (500Mb for 5 minutes of video).

Compress

We’re going to convert that big file to .mp4 which will compress it to a more manageable size (64Mb for 5 minutes).

We’ll use ffmpeg, a command line tool, to do that.

Install ffmpeg

First, to install ffmpeg, open Terminal and run this command.

brew install ffmpeg

(Requires homebrew, which by the way is a great way to install software on Mac.)

Convert .mov to .mp4

Now, to convert, run this command:

ffmpeg -i demo.mov -vcodec h264 demo.mp4

Note: you’ll probably need to change the name of the input from demo.mov to whatever you’re using (as well as the output, demo.mp4).

Cheers, that’s it! 🍻 You should now have a smaller .mp4 video file.

Check out the code used here and other helpful code snippets in my Github gists.

Convert .mov to .mp4 on a 🍎 Mac was originally published in Mac O’Clock on Medium, where people are continuing the conversation by highlighting and responding to this story.

Do not pip install gdal, do not install GDAL inside a virtual-env. Instead, use Conda.

These are my instructions on how to install GDAL using Conda on a Mac. Before we dive in, let me explain why I am writing this guide. GDAL stands for the “Geospatial Data Abstraction Library” and it is released by the Open Source Geospatial Foundation. For Python, the GDAL package is released with a package called osgeo as well. And as it happens, I need both for a project I’m doing. And I want them neatly wrapped up inside a virtual environment.

I started by trying to use pip and virtual-env. This obscure how-to (and first post on Medium!) comes to you after hours of googling and trying to debug the errors I encountered along the way. In the end, I shifted gears, tried something new and switched over to Conda. This was the solution to all my problems. OK, except a few, but I fixed those too.

Conda is great because it’s a package manager like pip, but it also manages your virtual environments like virtual-env does. Except it does both way better and it’s a joy to use. Treat yo self and switch to Conda.

I wanted to spare you the trouble I experienced so I wrote up the following instructions.

Environment:

A quick note about the environment I’m working in:

Mac OS 10.12.6 Sierra
Miniconda2 for Python 2 (Conda 4.3.30) info
Python 2.7.14

optional python packages:
Jupyter notebook, installs with $ conda install jupyter
nb_conda, makes Jupyter play nice with Conda, $ conda install nb_conda

If you are working in a different development environment, your mileage may vary.

And now, the instructions that I myself needed hours ago…

How to Install GDAL 🌐 using Conda 🐍 on a Mac 🍎

This tutorial assumes you have Conda already installed and a Conda environment already created. Instructions here and here, respectively.

Step 1: Activate your Conda environment 🚀

Open up Terminal, run this:

$ source activate [yourEnvironmentName]

For me, [yourEnvironmentName] = geoenv.

(We can deactivate with the command $ source deactivate.)

Step 2: Ok, now we get to install GDAL. 🔧

Still in Terminal, run this command:

$ conda install gdal

Here’s what I got:

Great. As it turns out, for the osgeo subpackage to work, we actually need the dependency jpeg version 8, rather than 9. You can read more about how I came to that conclusion towards the end of this post, under #Diagnosing.

For now, all you need to do is run this:

$ conda install -f jpeg=8

The “-f” flag forces the install (which is really a downgrade of the the jpeg module).

OK, we should have a working version of GDAL now! Let’s just test it to make sure.

Step 3: Test the installation 🔍

You can do this in the command line or in a Jupyter notebook. Since I want to make sure gdal will work in Jupyter later, I’m going to test there.

To open a new Jupyter Notebook 📙, go back to Terminal, run this command:

$ jupyter notebook

This command will open up a new tab in your internet browser with the Jupyter Notebook file viewer. Navigate to the directory where you wish to save your notebook. Now, we want to start a new notebook. Go to the upper righthand corner, click “New”.

Make sure to choose the Conda environment you’ve been working with as the Python Kernel.

Let’s go ahead and test! Run these commands in the notebook.

import gdal
help(gdal)

The help for gdal works, so we’re off to the races.

import osgeo
help(osgeo)

Success! 🤗

We’ve finally got GDAL installed as well as osgeo. Everything is working (for now). This somewhat lengthy post was a joy to write, as this problem caused me innumerable hours of strife. I hope to save you from the same. Thanks in advance for your claps 👏🏽 Let me know if something needs an edit or clarification. With that, I’m off to explore graphs with networkx!

With ❤︎,

Filip

p.s.

#Motivation for this post

I generally like to use virtual environments on projects to keep things organized. First, I tried to install GDAL inside a python virtual-env which was a huge fail. There are instructions out there how to do that for Windows and Ubuntu, but I couldn’t get it to work for Mac. Virtual-env was more like virtual-enemy. Some folks on StackOverflow suggested to use Conda instead. I ran into a few snags anyways, so I decided to publish these instructions how to Install GDAL using Python/Conda on Mac. Dear reader, I hope this guide saves you some time.

#What am I using GDAL for

I need GDAL for a very particular reason. It’s required for and a dependency of the read_shp() function of the networkx Python module. That function reads in an ESRI shapefile (geospatial data) an converts it into a network/graph object. Obviously, you might need GDAL for something else.

To install, I tried using pip install gdal inside a Python virtual environment (a.k.a. virtual-env) at first. That failed. I guess you could say it was only a pip dream, sigh. Or maybe it had something to do with having QGIS via Kyngchaos installed. That distribution includes GDAL not as a Python package, but as a Framework.

Anyways, the bottom line is that I still needed GDAL to work inside a Python virtual environment.

#Diagnosing the Conda install issue:

It was not possible for me to get GDAL installed inside a virtual-env using pip. That’s why I switched to Conda.

When running the install in Conda, I ran into a few issues. Simply running the read_shp function from networkx was giving me a generic error, much like it was in virtual-env.

ImportError: read_shp requires OGR:

In the screenshot you can see that the code requires from osgeo import ogr which is actually included as part of the GDAL module.

So when we try to import gdal, we can see what’s actually happening:

Library not loaded: @rpath/libjpeg.8.dylib

The jpeg8 library is not loading. To investigate, we can check what packages conda has installed:$ conda list

Moreover, when I uninstall and reinstall only gdal, it actually becomes evident that gdal itself updates jpeg to version 9, only to break later.

The fix is to simply downgrade jpeg 9 to jpeg 8 after installing gdal. You can find the recipe for that in Step 3 of the #Instructions above. Thanks!

Sources I used:

• egayer’s comment in this thread on the gdal GitHub
• “Having trouble installing GDAL for python”
• I’m not including my crazy, exhaustive searches for anything related to “pip install GDAL” or “GDAL python install mac virtual-env” in this list. Bless your heart if you try to go that path.

Install Python GDAL 🌐 using Conda 🐍 on a Mac 🍎 was originally published in HackerNoon.com on Medium, where people are continuing the conversation by highlighting and responding to this story.