EPCs in 3D — Help needed!
I was going through the recently-released energy performance certificate data, and wanted to join the ranks of people who’ve made awesome things with the data.
One of the things I really wanted to do was to make use of the fact that the data has been released at address level (and we’re actually allowed to do stuff with it (I think)).
I thought it would be pretty sweet to make a 3D model of Manchester City Centre, and colour the buildings according to their energy efficiency rating.
This isn’t a how-to guide (oh, how I wish it was). Rather, it’s an ‘I’ve got so far, but it’s not good enough, so please help’ post.
Here’s the map I made. I think you’ll agree that it is pretty sweet. Unfortunately, it’s wrong. Here’s my journey:
My big mistake was assuming it would be trivial to programmatically match addresses to buildings, and merrily began building Manchester City Centre in 3D. For this, I leaned very heavily on this excellent blog post by @_nadnerb.
I started by downloading Lidar data for the city centre — both DSM (Digital Surface Model) and DTM (Digital Terrain Model).
I did a raster calculation in R (QGIS borked, I think because of the size of area I’d selected), and loaded the results back into QGIS. This meant I now had a raster layer which showed the height of things. Because I was specifically interested in the heights of buildings, I needed to bring in the building polygons. I tried two approaches to this — firstly using Ordnance Survey’s Open Map — Local, and then using Open Street Map.
Ordnance Survey’s Open Map Local was really good in terms of the quality of the polygons. Each building was nice and defined, and felt right given what I know of buildings in the city centre.
I had a nagging doubt about the data table that supported this file — aside from the data describing the shape and location of each polygon, there was only an ID field — containing some sort of unique(?) hash.
I ploughed on anyway, and used the Zonal Statistics plugin in QGIS to assign a height attribute to each building polygon, and merge the height value into a new table with the other polygon details.
Using the excellent qgis2threejs plugin in QGIS, I was able to very quickly turn the image above into this, a 3d representation of the city centre:
Building heights look pretty good, the Beetham Tower is bottom right, with Deansgate running up and down to the left of centre.
Out of curiosity, I also decided to look at OpenStreetMap, and used the Quick OSM plugin to retrieve all buildings from the Nominatim database. This worked extremely well — quick to retrieve the data. Here’s that data from the city centre on the map. In places, it’s more detailed then the Ordnance Survey data. In others, less so.
Once again, assigning building heights to polygons is possible, and this is the 3D map we get from the data.
Some small differences between the OpenStreetMap and Ordnance Survey buildings map, but nothing major — a few details in buildings here and there.
Now that I’d tested this principle, I decided to put the data into Mapbox Studio for a more complete 3d Map. After some fiddling with datasets, styles, etc, I managed to get the buildings extruded nicely.
So rather than being constrained to a specific rectangle, this is the whole world (though still only buildings in Manchester City Centre — for now).
What works really well is that Mapbox supports data-driven styles. The shape and extrusion of buildings are defined by the supplied geoJSON, and we can put additional key:value pairs in here that allow us to define their colour. So we could give a specific building, for example, a property of EPC rating: 48, which would result in the building being coloured an environmentally-friendly green colour.
The challenge then, is working out how to update the geoJSON for 1,400 buildings. We can’t do this by hand (we could, but it would take a looooong time). So we need to work out how to do it programmatically. To do this, we need to match the address data in the EPC datafile to the some attribute of the 3D building layer. And this is where my difficulties start.
Going back to the Ordnance Survey data, this is a view of the attribute table:
There is nothing in here that I can use to attach EPC data to. EPCs are available at address level — and whilst this is amazing, there are no unique identifiers for the property — something I suspect an open address system would solve (looking at you Peter Wells…). The OS data has nothing useful at all, so I can colour individual buildings by looking at a building’s OS ID, then looking up its address on Google Maps, then looking up its EPC rating, and keying that back into the geoJSON. This, obviously, is highly unsatisfactory.
Looking at the OpenStreetMap data, we get much more information than with Ordnance Survey, in that it comes with other fields — things like building height, building address, building type, and various other things:
Unfortunately, because the data is crowd-sourced, this data is actually pretty sparse, so there was no point even trying to work out what the match rate was to my EPC addresses.
I started to think about other ways of geocoding my data. There are a few services out there that provide geocoding APIs — the most famous being Google. Google seems to be give really good matches — it uses the same engine that powers their address search on Google Maps:
Here’s where it says Central Library is, in Manchester — the marker is plum inside the building polygon on my 3d Map:
But, before we get too carried away with this roaring success — 1. this is a pretty famous building, and other addresses do nowhere near as well in the geocoder, and 2. The terms of Google’s API are such that you can use the API as long as you only visualise the results on a Google Map. Google maps doesn’t support WebGL, and I didn’t want to get into Google Earth, so I abandoned this.
My next avenue was to look at Mapbox’s geocoder. This would have been ideal — given that I was doing this stuff using Mapbox, but the quality of the geocoding wasn’t good enough for what I needed. Attempting to match the addresses in the geocoder typically resulted in points which were in the middle of roads. This meant that I couldn’t reliably assign them to a building polygon without either moving the point, or increasing the size of buidings. I abandoned this.
My final option was to try and geocode the postcode of the address using Ordnance Survey’s Code-Point Open dataset. This was likely to be less precise in terms of the geocoding, but I hoped that each point would sit in the middle of a building (though in my heart, I knew that this wouldn’t be right).
These are the results of that exercise:
Each pink point is a an EPC, geocoded by postcode, though each point may represent more than one EPC, where two or more EPCs share a postcode (which happens a lot).
There are blanks where there are no EPCs. Manchester Town Hall, (the trapezoid in the middle-left, above), doesn’t have an EPC. As a public building, it has a DEC (Display Energy Certificates), which are available in a different dataset, that I haven’t done anything with for this purpose.
So in QGIS, I then used the vector data management tools to join the buildings and EPC data by location, and created a geoJSON file of building polygons, containing the mean EPC value by building. The data in each pink dot (in the map above) is allocated to the green building that the dot sits within, and these are aggregated to the mean of the rating in the EPC.
This geoJSON is what I used to create the 3D map here:
Whilst I’m proper pleased with the way the map looks, I have real doubts over the data — the different coloured buildings don’t feel right -such as the red building at the top of the image above, which is an office block on New York Street. When we look at the individual EPC data for that building, there are no EPCs that suggest that that building should be rated red. This points to problems with the geocoding and geoaggregation.
This, then, is my call for help. I think that this way of displaying EPCs could be a really useful way of showing the environmental impact of buildings, and could encourage people to improve the efficiency of the buildings they are responsible for. It would also allow people to easily understand which buildings in their local area are most responsible for damaging the environment, so they can lobby appropriate people to do something about it.
This could extend to residential properties, where showing all EPCs on a particular estate could lead to people understanding how they could improve their own homes to become more energy efficient.
But to do all this, I need help to understand if there’s a way I can geocode EPCs to buildings, if I’ve overlooked something, or there are tools that I’m not aware of. There are people doing amazing stuff with geodata. People like John Murray, Alasdair Rae, Owen Boswarva, and many others who are probably way better equipped to have a go at solving this problem.
