Hidden Data Stories — Exploring open geospatial datasets

As part of our efforts to promote everyday use of open data, we will publish one short data story biweekly based on seemingly inconspicuous datasets to uncover the hidden data stories on Data.gov.sg.

Lamp posts in Singapore?

Besides Data.gov.sg, other agencies such as the Land Transport Authority (LTA), also publish government data for public use. For this week’s story, we’ll be taking a look at the Lamp Posts dataset provided on LTA’s Data mall.

Source: LTA’s Datamall Lamp Posts Dataset

There are over 118,588 lamp posts lining the streets all over Singapore. To make the dataset more human readable, we have converted the Shapefile data into a GeoJSON formatted file.

The snippet below is an example of two data points taken from the output:

[
  ..., {
    "type": "Feature",
    "properties": {
      "LAMPPOST_N": "58"
    },
    "geometry": {
      "type": "Point",
      "coordinates": [23730.529397564358, 28650.274720978738]
    }
  }, {
    "type": "Feature",
    "properties": {
      "LAMPPOST_N": "64"
    },
    "geometry": {
      "type": "Point",
      "coordinates": [23657.594820220933, 28675.289182497487]
    }
  }, ...
]

Each data point contains a coordinate pair marking the geographical location of the lamp post. The coordinates provided by LTA are represented using the SVY21 projection, which is why it may look different from the standard -90° to 90° latitude, -180° to 180° longitude system we are familiar with.

SVY21 (aka EPSG:3414) is a map projection system that is designed specifically for Singapore. Click here to find out more about the why there are different map projections. We will be explaining why most Singapore-related geospatial datasets use SVY21 in another blog post.

How far apart are lamp posts in Singapore?

Out of curiosity, you might wonder how far apart two sequential lamp posts might be.

To determine this, let’s start with a simple strategy:

1. For each lamp post, find its nearest neighbour among the whole set of lamp posts
2. Measure the distance between the two lamp posts

If we apply this simple strategy, this is what we get:

Histogram of distance to nearest neighbour for each lamp post — data

The histogram has a single peak at 24 metres. But the large range in values suggests that there might be other factors that we might have not considered.

Other considerations when measuring adjacent distance

First Consideration: Are lamp posts along the major roads like expressways spaced the same as lamp posts along small streets?

Second consideration: Is the nearest neighbour an adjacent lamp post on the same side of road? See below for the many cases this might not be true.

Nearest lamp post might not be the adjacent neighbour

Third consideration: Is the width of the road smaller than the spacing between adjacent lamp posts? (This will have a bigger impact on the measurements because it affects every data point along that stretch of road.)

Addressing the challenges

To address these considerations, we can start by reducing our problem statement.

Initial problem statement:
Lamp posts spacing in general
Refined problem statement:
Lamp post spacing for one particular road (eg. PIE).

By using our refined problem statement, we can address the first consideration and to a smaller extent the second consideration.

Implementing our refined strategy

To implement the refined strategy, we can make use of another dataset in LTA’s geospatial datasets — Road Section Lines (User guide: page 46).

Here is our refined strategy (new steps bolded):

1. Pick a road from the Road Section Lines dataset
2. Apply a buffer around the LineString
3. Filter out the set of lamp posts that fall within this buffer region
4. For each lamp post, find its nearest neighbour among the reduced set of lamp posts
5. Measure the distance between the two lamp posts

Lamp posts along the PIE (Demo app)

The histogram now shows a peak at around 35 metres.

Histogram of distance to nearest neighbour for each lamp post (PIE) — data

Let’s repeat this for a few other roads:

Histogram of distance to nearest neighbour for each lamp post (Other highways) — data

It can be seen that most expressway lamp posts are spaced around 35 metres apart with the exception of ECP and AYE which peak closer to 40 metres.

What about roads with smaller widths?

However, when the same strategy is applied on smaller roadsim, we observe multiple peaks in the histogram.

Histogram of distance to nearest neighbour for each lamp post (Bukit Timah Road) — data

A sanity check on the set of filtered lamp posts shows that when narrower roads are selected, the lamp posts located on the opposite side of the road are selected instead of their adjacent neighbour (third consideration above).

Lamp posts on the opposite side of the road being mistaken for adjacent lamp post (Demo app)

Further work

To further refine this algorithm, additional steps can be taken to ensure only the adjacent neighbour is selected:

1. Draw a line connecting the target lamp post to a potential neighbour
2. If the line is oriented more or less parallel to the road segment, shortlist the neighbour. Otherwise, discard it
3. Repeat for all potential neighbours
4. Among the set of shortlisted neighbours, find the one nearest to your the target lamp post

We have not implemented this strategy but you may give it a try if you are up for it. All code we used to prepare this blog post can be found at here. Alternatively, you can explore the data yourself with this demo app.