Optimization of RGB DEM tiles for dynamic hill shading with Mapbox GL or MapLibre GL
Dynamic hill shading and terrain elevation use raster tiles in a specific encoded format. These tiles have the drawback of being quite heavy. Here we study a solution that reduces this volume by 60%.
This article is an English translation of my own French article from Makina Corpus Blog.
The French BD ALTI® is a Digital Elevation Model (DEM). At 25 m, it contains a point of terrain elevation on the territory every 25 m. It is a raster data, that is to say, it is made of pixels. Each pixel is a square of 25 m per side.
To use a terrain model on web maps we can:
- Pre-compute hill shading: i.e. make lighter the faces exposed to sun and darker the ones in shadow. This shading is an image that we can manipulate as for a raster map background in the form of PNG raster tiles.
- Compute vector hill shading tiles, which are a vectorized version of the previous tiles. These are more efficient to transfer on the web. A graphic style can be applied on the client side but the graphic rendering of the relief is not as fine.
- Provide directly the terrain model. The computation of the hill shading being performed on the client side, especially in Web GL. We can with this way apply a graphic style to the hill shading. The position of the light source can be adjusted or even be moved dynamically on the client side: for example when the map is rotated or to make the sun move. These are raster elevation tiles in PNG file format, with an encoding of the altitude called RGB (red, green, blue).
- Extract contour lines from the terrain model and distribute them as vector tiles.
In a previous article we saw how to use the 75 m BD ALTI® terrain model of the French National Geographic Institute (IGN) to produce tiles for Web GL maps (Mapbox GL, MapLibre GL, Césium…). Since then, the IGN has made available almost its entire catalog in OpenData, including the 25 m version of the BD ALTI® and the RGE Alti distributed at 1 m and 5 m but whose real accuracy varies depending on the location.
Now, we presented how to use a digital terrain model to produce dynamic hill shading raster tiles (RGB) and contour vector tiles. So we adapted the process to the new 25 m version. The 75 m version allowed to produce raster tiles up to zoom level 11, while the 25 m version allows to go up to zoom level 12.
Indeed, with a latitude of 45 ° in metropolitan France we have:
- zoom level 8: 431 m/pixel,
- zoom level 9 : 216 m/pixel,
- zoom level 10: 108 m / pixel,
- zoom level 11: 54 m/pixel,
- zoom level 12: 26 m/pixel.
For each additional zoom level, the amount of data required is approximately twice.
Furthermore, the total weight of these raster tiles is quite large. The more hilly a tile is, the heavier this PNG RGB encoded tile weighs. Tiles of 300 KB to 400 KB are common in mountains areas. These are images with 512 pixels on each side.
The zoom level is also important: at maximum zoom 12, the terrain will be relatively less rugged than at lower zooms. At lower zoom levels, the area covered by a tile will be larger, and therefore more varied.
These images are not made to be viewed directly, it is RGB encoded data.
The relationship between the altitude and the color of each point is :
altitude = -10000 + ((Rouge * 256 * 256 + Vert * 256 + Bleu) * 0.1)The vertical resolution of these images is thus encoded with a vertical resolution of 10 cm. The horizontal resolution depends of course on the zoom level, but also on the latitude of the tile coordinates.
RGB tiles, a big problem
The RGB encoded DEM offers the best visual result while offering the largest number of possibilities on the client side. In addition to shading, it also allows 3D rendering of the map (2.5D in fact).
However, these RGB tiles are still heavy to load for the client. A map on a 1300x900 pixel web page at zoom level 8 with terrain shading and partly in mountains area consumes 3.0 MB of vector map data tiles and 7.0 MB of PNG RGB tiles for hill shading.
Optimizing the size of RGB encoded tiles
This size problem occurs all the time, but especially if you want to pre-download hill shaded cartography on mobile for offline use… which can be relevant in the mountains area.
Switch to JPEG format
JPEG is a lossy image compression format. That means that it will adjust the content of the image in a way that is not very visible to improve the compression. The JPEG is adapted to the compression of photographies.
The compression is more effective when the content is more predictable in relation to the rest of the image. Basically, the more “abruptly” the colors of the pixels change, the harder is to compress it. In other words, the lower the entropy, the easier it is to describe.
Terrain hill shading is based on a relative difference in elevation. The lighting and shading depend on whether the terrain is rising or falling.
Lossy compression changes the data and introduces artificial artifacts of terrain variations. These artifacts, combined with the fact that the compression is done for color perception to the human eye while our images contain data, completely disqualify the use of JPEG for this purpose.
WebP
Instead of the PNG image format, it is also possible to use the WebP format which can be lossless or not. It is based on a more modern algorithm originally designed for video compression. It allows a better compression rate than PNG.
The processing of the BD Alti 25 m over France in PNG RGB for the zoom levels from 5 to 12 gives a total of 3.7 GB. The same tiles but in lossless WebP give a total of 2.2 GB, which represents a gain of 40%.
However, WebP is not yet a fully supported format by all web browsers.
Statistics tell us that by 2022, 5% to 7% of users will be using browsers that do not support it.
Depending on the use case, it may still be acceptable to serve these users maps without hill shading in regard to the size gain provided by WebP.
Optimization of PNG tiles nevertheless
The PNG image format, like lossless WebP, cannot produce images smaller than it already does with its compression algorithm.
However, there are upfront techniques for reducing the size of PNGs, such as quantization. This is the adjustment of pixel colors to reduce the number of different colors and use a color palette instead of a full color space RGB (red, green, blue). The palette contains a maximum of 256 colors. As an example the zoom level 8 tile above initially contains 14,616 different colors. However, the generic “quantization” is not adapted to the altimetry data encoded in the image. Given the RGB encoding formula for altimetry, changing the value of green has 256 times more impact on the encoded altitude than changing the value of blue.
The inherent problem with the compression of these images
We must think about the nature of the encoded data and not about the colors represented in the image. The three colors resulting from the encoding of the altimetry cannot be treated in a similar way. Red has 256 times more weight than green, which itself has 256 times more weight than blue. Standard image processing tools are not aware of this particularity.
If we look at the different colors separately of a level 8 tile, we see clearly that red does not vary, green vary a little and blue a lot.
We see that the blue channel is much noisier than the others, especially at low zoom levels where the altitude varies a lot from one pixel to its neighbors, almost randomly. If we remove the blue channel (we can’t really remove it, but set it to 0 for all pixels), then the same PNG-compressed image is only 91 KB, or 80% smaller. This is the very noisy data that makes the image difficult to compress. The more random the values, the greater the entropy, and therefore the more inefficient the compression.
The blue channel has an amplitude of 256 levels, one level represents 10 cm. We have therefore introduced a maximum error of 256 * 0.1 = 25.6 m in altitude.
For example: at a zoom level 8, the distance between two pixels is 431 m, an altimetric error margin of 25 m can be acceptable.
Extending the reduction of the accuracy of the altitude
We can extend this idea of reducing the precision level of the encoded altitude according to the zoom level. This is already done for maps where the accuracy and the level of detail depend on the scale of representation.
Removing the blue encoded part is in fact equivalent to rounding the altitude to the nearest 25 m and setting the last 8 bits (the bits encoded in blue) to 0. We can extend this by varying the level of precision on which we round the altitude and therefore the number of last bits that we set to 0 depending on the zoom level.
We have decided to delete the last 11 bits for zoom level 5, i.e. to round to within 200 m. For zoom level 12, we delete the last 4 bits, i.e. rounding to the nearest 80 cm. Between zoom 5 and 12 the number of bits deleted is a linear interpolation. Note that zoom level 12 is the last one, so it plays a special role. When the map is displayed with a zoom level higher than 12, it is the tiles of zoom level 12 that are stretched.
The processing of the BD Alti 25m now weighs 1.3 GB in PNG in place of 3.7 GB, a 65% reduction in size, and 0.8 GB in WebP instead of 2.2 GB. This is also a size reduction of 65%. If we combine the accuracy reduction and the switch to WebP we can even get an 80% gain from original PNG.
We could make other choices about the number of final bits deleted to further reduce the size of the tiles at the detriment of visual quality, or conversely to increase it. Note also that we could make a specific choice about the precision of the last level of tiles, here zoom 12. Knowing that these tiles will be stretched to be displayed at the additional zoom levels.
Proposed enhancement to the rio-rgbify project
The free software “rasterio” (rio) is a tool for processing raster images, typically orthophotographies. “rio-rgbify is a plugin to rasterio for generating RGB elevation tiles.
We propose to add an option to rio-rgbify to allow to adjust the accuracy of the elevation and thus reduce the volume of the tiles produced.
Data and production script
The complete procedures to produce the tiles for dynamic RGB shading as well as the vector contour tiles are available, as well as the results of the processing on the DB ALTI® 25m in PNG and WebP MBTiles.
Conclusion
The approach presented here allows a significant optimization of the image size for dynamic hill shading and terrain elevation. The level of accuracy of the elevation can be adjusted to keep the visual impact almost imperceptible. The gain is both important for direct display by reducing network transfers and therefore user waiting time. It is also very important for pre-downloading these tiles for offline use, reducing download time and storage requirements.
