Towards Location Intelligence: Open Data and the QGIS CKAN Browser plugin

Open Data - Toronto

Published in

Open Data Toronto

6 min readFeb 6, 2020

Original article by Ryan Garnett, republished with permission

1.0 Purpose

Data is king; the availability of data has never been higher, and it continues to grow. Quotes like “as much data is now being created every two days, as was created from the beginning of human civilization to the year 2003” from Google’s Eric Schmidt clearly indicates the vast amounts of data that is available. However available does not mean accessible. While the availability of data is increasing, it is in no way readily accessible, especially for all users.

Recently significant efforts have been made to improve data access for software developers and data scientists through application programming interfaces (APIs). An API provides a programmatic method for accessing data that is stored in enterprise systems, such as a database. The technical nature of data access through APIs can be limiting for non-technical users as well as specific software applications, as certain applications do not allow for API integration, leaving a need for alternative data access methods.

Geospatial, or location data, is a type of data that is widely used in many industries and sectors; providing valuable information required for decision making. Location data, like other types, is increasing in availability, and like other data types access to location data can be difficult.

Enter Open Data. Open Data is a growing movement where organizations, especially government, release data to the public for use free of charge; allowing residents, students, educators, start-ups, and businesses to access and leverage the data for their needs. Location data represents a large portion of open data, approximately 60–80% of all open datasets, making location data an extremely valuable data type. Users of location data vary greatly making accessing data an important topic.

There are many ideas on data access, with “go where people play” being a growing trend. The premise of go where people play is that people should be able to access data from within the applications that they currently use; rather than needing to go to an external source for data access. With this I would like to introduce the CKAN QGIS Browser plugin, a go where people play method for accessing location based open data from within the most widely used open source GIS application — QGIS.

CKAN and QGIS are both popular open source projects that have a very active community. The remainder of this story will focus on accessing location data within QGIS via the Open Data CKAN Browser plugin.

2.0 QGIS — Open Source GIS

QGIS is a professional GIS application that is Free and Open Source Software (FOSS). Designed as a user friendly Open Source Geographic Information System (GIS), QGIS is an official project of the Open Source Geospatial Foundation (OSGeo); running on Linux, Unix, Mac OSX, Windows and Android, supporting numerous vector, raster, and database formats and functionalities. For more information visit the QGIS website.

QGIS is available for Windows, Mac, Linux and Android users. The software can be downloaded here; while installing software is very important it will not be covered within this story.

3.0 The CKAN QGIS Plugin

The CKAN QGIS plugin was developed by BergWerkGIS, originally for QGIS version 2.x, but recently has released a version for QGIS 3.x. The plugin provides functionality to “load and display Open Data from CKAN enabled data portals”. For more information visit the CKAN extension page or BergWerkGIS GitHub page.

3.1 Installing the Open Data CKAN Browser Plugin

Plugins are installed in QGIS through the manage and install plugins tool within the plugins menu. There are two main classes of QGIS plugins, official and experimental. Official plugins have been accepted and are available on the QGIS plugin repository. Experimental plugins typically have been developed by community members and are available from their personal repository (i.e. GitHub). To access a wider range of plugins, allowing experimental plugins should be enabled.

The plugins are access within the QGIS menus from: Plugins — Manage and Install Plugins…

The plugin can be installed two different ways:

Searching for ckan-browser within the manage and install plugins tool
Download the zipped version of the plugin and install it via the install from zip option within the manage and install plugins tool

Figure 2: Search of CKAN Browser plugin within QGIS

Figure 3: Install plugin from zip file within QGIS

3.2 Configuring the CKAN Browser Plugin

Once the plugin is successfully install it is ready for use. The plugin works specifically for data that is stored in CKAN repositories. Prior to accessing data the plugin must be configured to link to a specific CKAN repository. This can be achieved by opening the CKAN Browser via Plugins — Open Data (CKAN) Browser from and clicking on the Open Data (CKAN) Browser icon from within QGIS.

Figure 4: Accessing the CKAN Browser plugin

The first time the CKAN Browser plugin is opened a cache location needs to be identified. The cache stores temporary files that are required by the plugin. For this story C:/Temp was used. With the cache directory identified click the save button.

Figure 5: Identifying the plugin cache directory

With the cache directory save the CKAN Browser plugin will be available.

There are 200+ publicly available CKAN instances, however not all have the api-url identified, which is the method used to connect the CKAN Browser plugin to an open dataset. Any CKAN api-url can be added into the CKAN API Endpoint, providing access to open data managed within CKAN.

In this story we will access data from the City of Toronto’s Open Data portal. To accessing a CKAN repository click Select CKAN Server button, a new window will open (see Figure 7). From within the new window (CKAN Servers) enter the CKAN api-url (i.e. https://ckan0.cf.opendata.inter.prod-toronto.ca/api/3/) to access a specific CKAN instance that is not listed below in the list of publicly accessible CKAN api-urls. If you would like to make a CKAN api-url publicly accessible complete the following form (https://ckan.org/census/) to add your CKAN instance.

The final step is to check the City of Toronto CKAN instance and click the save button

Figure 8: Identifying the CKAN instance to connect to

Once the configurations have been set you are ready to access data via the plugin.

3.3 Accessing Open Data

Once configured and connected to a CKAN repository all available datasets are listed in the Search Results section of the plugin. At the time of this story the City of Toronto’s Open Data CKAN repository had 309 datasets.

There are two ways to find data:

Scrolling through the dataset list and pages
Using the Search term search bar

By selecting a dataset, clicking on and highlighting a dataset in the Search Result section, different information about the dataset becomes available, such as the description, file formats, and resource URL. Selecting a dataset and checking the desired version/format (Ambulance Station Locations — SHP: 2019-ambulance-station-location-wgs84) the data is loaded into QGIS by clicking the Load Data button.

The data will be loaded into QGIS as a layer allowing for analysis, modelling, or visualization. Additional layers from the Open Data CKAN Browser, or other sources, can be added for full location intelligence capabilities.