Building CampaignHawk: Making a Precinct Data Layer (Part 18)

6 min readSep 12, 2015

So now that we can effectively render data layers, we need to make a few. The first one we will do is separate by precinct and generate a concave-hull around each of them with colors based on party affiliation. Turf takes in a FeatureCollection of points, so our first task is to separate our current FeatureCollection into precincts.

We’re going to do this with the help of underscore and turf-featurecollection. At least… that was the plan. Something is wrong with precinct “DUV 45–3481”. I get the error:

Uncaught TopologyError: no outgoing dirEdge found [ (-121.9717867, 47.73397465) ]

Unfortunately, I have no idea what that means. My first thought is that there might be something wrong with my data. So, I installed geojsonhint, which is has a command line interface that tests your geojson data.

$ npm install -g geojsonhint

I then saved the feature group to a file and ran geojsonhint… which told me that my data was perfectly fine.

Two hours of debugging, reading through documentation, and sorting through data followed. Eventually I gave up and posted a message on StackOverflow. If you have the answer, I would be grateful for your reply.

Until I get the answer, I’m just going to go ahead and use a convex hull, which is not ideal, but can be easily changed out for concave once we get this sorted.

Reformatting this data is a long process, but the final result will look like the image below. Once we fix the concave hull problem, the shapes should not overlap.

The first thing we should do is move our clustering functions into a separate function within toggleDataLayer and not call it.

Then we should create another function called precinctDataLayer, in which we will add the functionality to make polygons that represent the various precincts.

These polygons are made by passing a FeatureCollection into turf-convex/concave, which returns a polygon.

If we pass the data as it is now, we will have three problems: 1) there are many points with the same coordinates, which breaks turf-convex/concave, 2) there are several points that failed geocoding and have coordinates of [0,0], 3) our FeatureCollection is not split into precincts, so it will just render one big blob.

But we can solve all of this with functions — mostly using underscore. I’m going to break this process down into the smallest steps possible to make it easy to understand what is happening at each step. Every one of these is going to go in order within our precinctDataLayer function.

toggleDataLayer(layerName) {
  if (!this.props.loading) {
    let precinctDataLayer = function() {
      <all the code goes in here>
    }
  }
}

The first thing we need to do is get our data. But we don’t need all of it — just the array of features:

let allDataFeatures = VoterDataGeoJSON.find().fetch()[0].features;

Then we need to remove duplicates. Many people live at the same address, which leads to duplicates. In this dataset, more than half of the addresses are duplicates. We’re going to use a little bit of ES2015 wizardry to make this easier by using a Set. A Set contains only unique items. In this case, we’re using it as a temporary store for unique coordinates.

First we declare the set as a new Set. Then we want to filter out the points that are duplicates. We’re going to do this with underscore’s _.filter function. If you are not familiar with this function, you should read up on it because it is commonly used and immensely helpful.

Within our filter function, we are converting the coordinate array into a String so we can compare them. Arrays are really just Objects, so you cannot compare them. For example:

[1] === [1] // returns false

We are also using the has function from our Set to check if the set has a particular set of coordinates. The old way of doing this with an array would be to use indexOf.

So in our if conditional, we are saying “if the coordinates of this feature do not exist in the set…” We then have an and operator (&&) in our conditional that checks to make sure the coordinates are not [0,0], which place it somewhere off the coast of Africa.

If it meets both of those criteria, we add the string of coordinates to the Set to make sure no future features are added with the same coordinates, and then we return the feature, which adds it to uniqueDataFeatures.

var stringOfCoords = new Set();
let uniqueDataFeatures = _.filter(allDataFeatures, 
function(feature) {
  if (!stringOfCoords.has(feature.geometry.coordinates.toString()) && feature.geometry.coordinates.toString() != "0,0") {
    stringOfCoords.add(feature.geometry.coordinates.toString())
    return feature
  }
})

So now that all of our features are unique, we need to separate them into precincts. We can do that with underscore’s _.groupBy function. In this case, we’re taking the uniqueDataFeatures and separating them by precinct and indexing them with a key based on the precinct_name.

let groupByPrecinct = _.groupBy(uniqueDataFeatures, (feature) => { return feature.properties.precinct_name; })

At this point, our data looks something like this:

DUV 45-0389: Array[388]
DUV 45-2959: Array[175]
DUV 45-3218: Array[393]
DUV 45-3219: Array[248]
DUV 45-3481: Array[348]
DUV 45-3502: Array[198]
DUV 45-3642: Array[211]

Pretty cool! Now we need to get a list of the keys from our groupByPrecinct object that we just created so we can iterate over the object later on. Once again, we can use an underscore function — in this case _.keys, which simply returns an array of the keys of an object.

let precinctKeys = _.keys(groupByPrecinct);

So now we have our keys and we have features split by precinct. Now we need to create FeatureCollections out of these, which we can do with turf-featurecollection. We’re going to iterate over the keys and for each precinct, we are going to convert the array of features into a FeatureCollection.

let precinctFeatureCollections = [];
_.each(precinctKeys, (key) => {
  precinctFeatureCollections.push(
    turf.featurecollection(groupByPrecinct[key])
  )
})

Then we need to convert those FeatureCollections into concave/convex hulls, which in this case is really just a fancy way of saying “shape”. We can accomplish this by iterating over the precinctFeatureCollections that we just made, converting each FeatureCollection into a convex/concave hull using turf-convex/concave, and pushing that result to the precinctConcaveHulls array.

let precinctConcaveHulls = [];
_.each(precinctFeatureCollections, (precinct) => {
  precinctConcaveHulls.push(turf.convex(precinct, 0.1, 'miles'))
})

And finally, we get to add these shapes to our map. We do this by creating a new featureLayer and passing the array of precinctConcaveHulls that we just made. Then, we use the addLayer function to add the layer to our map.

let precinctFeatureLayer = L.mapbox.featureLayer(
  precinctConcaveHulls
);
map.addLayer(precinctFeatureLayer);

Next Steps

I think we should make at least one more data layer before we worry about toggling between them. A good one would be a slider that filters out unlikely voters based on their voter score. Then we can figure out the best way to toggle.

Sam Corcos is the lead developer and co-founder of Sightline Maps, the most intuitive platform for 3D printing topographical maps, as well as LearnPhoenix.io, an advanced tutorial site for building scaleable production apps with Phoenix and React.

Cumulative Time

If I hadn’t gotten stuck for two hours thanks to whatever dirEdge is, this would have been pretty quick. Oh well. Debugging is a part of the process.

Building CampaignHawk: Making a Precinct Data Layer (Part 18)

Next Steps

Cumulative Time

Additional

Written by Sam Corcos