Near the end of 2017, I released the first version of the West Coast Estuaries Explorer. Working with partners with the North Pacific Landscape Conservation Cooperative (U.S. Fish and Wildlife Service) and Pacific Marine and Estuarine Fish Habitat Partnership (PMEP), and Pacific States Marine Fisheries Commission (PSMFC), and using data from PMEP (available here), I developed this application to make it easy to explore estuaries along the coasts of Washington, Oregon, and California. Estuaries are a unique and diverse ecosystem where rivers flow into the ocean. They serve a key role in the lifecycle of several important fish and invertebrate species along the coast. We wanted to make information about these estuaries accessible to a fairly broad audience and include interactive exploration of the data to let users really dig in and learn more about estuaries in this area.
One of the core functions of this application is to filter estuaries based on different characteristics and see those on the map, as well as explore the data quantitatively to see how many estuaries meet combinations of different criteria. This approach empowers users to really dig into the data and ask their own questions. For example, which species are found most often in an estuary of a particular type, within a particular geographic region?
Recently, I worked with PMEP and PSMFC to update this application for hosting by PSMFC, and I used this as an opportunity to upgrade several of the internal components and address some of the rough edges from the first version. I’m very happy with the updates and I wanted to share some of the highlights with you. This post assumes some familiarity with the React ecosystem.
The first version was built as a static app using Create React App, Redux, Leaflet, Crossfilter, and other components to create an integrated application that allows you to search for and filter estuaries on the map based on different characteristics.
Create React App is a great way to spin up a new React application, without mucking about getting all the dependencies and build environment set up manually. While you trade off some of the control you get from building things up from scratch, you get an environment that lets you start developing your app — rather than the build environment — much more quickly. Redux is a global state container often used in React apps to manage state — such as what filter is turned on — across an application of several components.
When a user selects a particular value from one of our dimensions, we apply a filter function to that dimension; this tells crossfilter how to determine if a given record meets that condition. For example, that a given estuary is of the type selected by the user. What is special about crossfilter is that it allows us to easily query out the values and counts of records still present in each of our other dimensions after applying each filter. This makes it very powerful for combining filters across different criteria, and exploring how those interact.
I created vector tiles of estuary boundaries and biotic habitat types using tippecanoe and hosted on our lightweight tile server, mbtileserver. Vector tiles are a way of cutting and encoding geometric data, such as points, lines, or polygons so that you can display an appropriate level of detail at a given zoom level in the map. One of their greatest features is that you can define your own dynamic styling based on properties associated with each geometry; you aren’t limited to the style used when the map tiles are created. These made it possible to see the very detailed information about estuaries as you zoom in on the map, without showing too much detail — or consuming very large tiles — when you were looking at the entire coast. I also used clusters of representative points for estuaries when you are viewing the entire region so that you could more easily visualize the location of estuaries, especially as you start applying filters.
I created compact tabular data in advance using geopandas from the geospatial data and then used webpack to compile that into my application at build time instead of requesting it after the page loads in your browser. While this made the JS bundle loaded with page load bigger, it reduced the complexity of fetching additional data immediately after load — especially for data that were needed in nearly every view. The data ended up being small enough that this didn’t appear to cause performance issues.
While I am still proud of the original version, there were a few rough edges. These included:
- using an ejected version of Create React App meant that upgrading dependencies needed to be done carefully to avoid breaking cross-dependencies between 3rd party libraries.
- using vector tiles in Leaflet presented performance issues as well as a bug that still hasn’t been resolved for multi-part polygons (fix here, not yet merged).
- the way I originally coupled crossfilter and redux meant that knowledge about data flow and state was spread around multiple files, making it a bit harder to gain a cohesive picture of how that state interacts with components that are supposed to interact with it, such as the filter bars.
- I originally used Bulma as the front-end CSS framework. Bulma is great and allowed me to move quickly, but I ended up overriding a bunch of styles, and struggled to get some of the responsive aspects (especially for the navigation menu) to work in just the way I wanted.
You might wonder why I used a custom integration of Leaflet and React instead of using leaflet-react. Ultimately, I wanted more programmatic control over Leaflet and I needed to be able to use several of our existing Leaflet plugins, which haven’t yet been ported to leaflet-react compatible plugins. I’ve always preferred having more direct control over native JS libraries by wrapping them myself for use in React, rather than relying on 3rd party wrappings.
Enter GatsbyJS and Mapbox GL
Lately, I’ve been using GatsbyJS to build static apps, and I’ve been very pleased with the developer experience and performance. I get more of what I need right at the start with a good Gatsby starter, whereas with Create React App I always found myself having to modify a bunch of files at the very outset of starting up a project. (Note: this isn’t a slam on Create React App, I’m still very thankful that it made firing up a new app so easy compared to building one out from scratch!)
I’ve also been working on refining how I couple React with JS libraries like Leaflet and Mapbox GL, especially using React hooks now that they are fully supported.
I wanted to see how hard it would be to migrate this existing application to my latest approach to building static apps. In particular, I wanted to swap out Leaflet for Mapbox GL, so that I could get high-performance native support for vector tiles. I wanted to migrate from a mix of React class-based components and functional components to functional components everywhere, and leverage React hooks heavily. If possible, I wanted to replace the global Redux data store with more local state management. Lastly, I wanted to swap out Bulma for styled-components, which I’ve been using in other apps. All told, it took me a few days of migrating logic from the original components to the new version, much of which was getting a much better handle on how to use React hooks alongside crossfilter and Mapbox GL.
Some of the major wins:
Mapbox GL gave me much better performance with my vector tiles and also gave me better control over dynamic styling. Instead of only being able to show the highly detailed areas of biotic habitat for a single selected estuary due to performance limits of complex vector tiles in Leaflet, I am now able to show these everywhere. Since I was able to leverage Mapbox GL’s API for querying features visible within the current view, I was able to make the legend dynamic based on what is actually visible, rather than the full range of possible values for each layer in the map.
I was able to use Mapbox GL’s built-in clustering support, instead of having to wrap supercluster myself, as I did in the Leaflet version of the app. I was also able to set up zoom-level dependent rendering, so that large estuaries are more visible when you are looking at the entire coast without having visually obtrusive boundaries, but more clearly showing those boundaries as you zoom in. The PMEP data team worked really hard to create highly detailed boundaries, and I wanted those to look good!
With the greater control over styling layers that Mapbox GL provides, it also requires a bit more effort at the outset to design and configure appropriate styles.
I really enjoy using styled-components — it gives me exactly the level of control over styling that I’ve always wanted. I was a die-hard proponent of only using CSS files loaded at the root of the project until styled-components won me over. Now I can provide styles directly alongside my components, reuse them across components, and use them to create more semantic component HTML (e.g.,
<HelpText>Instructions go here...</HelpText>) which is a big readability win versus a more CSS class-based approach. And I still get to use CSS.
I was able to leverage React hooks — specifically
useContext — to help wrap and consume crossfilter in my components. This meant that I was able to derive some of the same benefits I was originally getting from the reducer pattern used by Redux, without as much boilerplate and multiple files associated with those conventions. Using
useContext within specific components made it really easy to hook those components into crossfilter — either to update the state of the filters — or to re-render themselves based on the updated filters.
Net balance, I can’t say this approach is clearly superior to using Redux, since overall the data flow is still pretty similar: filter state is passed down into components that need to render themselves based on that, and they dispatch events to modify that state up into the reducer defined by the hook. However, one of the things that I like about it is that it provided a better way of compartmentalizing crossfilter, making it easier to port this implementation to other projects — which was much harder when it was deeply entangled in the global application state of this application.
Instead of loading data using
webpack during compile time, I migrated to a GraphQL based approach in Gatsby and achieve the same effect — but with more obvious data querying and flow. Gatsby still bundles the data into the built assets for the static site.
I used a custom React hook to help load the data using a static query in Gatsby. This allowed me to isolate the logic of loading the data from other components, which would enable me to load the data via a separate request after the page is loaded instead of built with the application, if I decided to refactor in that direction.
Not without a few issues…
ArcGIS Vector Tiles
Unfortunately, the new vector tiles hosted by PSMFC on ArcGIS Server / ArcGIS Online were not fully compatible with Mapbox GL by default. This is because ArcGIS Vector Tiles use a hierarchical tile index by default, which is both not obvious to the user creating them, nor does it meet the vector tile specification. It doesn’t even look like this support will make it into Mapbox GL anytime soon, either. It is a clever idea, I just wish that it would have made it into the vector tiles specification instead of arbitrarily provided by a single vendor.
It was also harder to figure out the contents (layers and properties) of ArcGIS vector tiles, compared to those generated using
tippecanoe. In contrast,
tippecanoe coupled with
mbtileserver provides a complete listing of all layers and properties for each layer. While I think it is great that ArcGIS includes support for vector tiles, they still don’t feel like first-class citizens in that environment.
Gatsby build with Mapbox GL
build process runs in an environment where the browser
window is not available, yet Mapbox GL assumes the window to be present at the time it loads. This causes your builds to break with somewhat confusing error messages. In order to get around this issue, you need to add a bit of info to your
gatsby-node.js file to let Gatsby know that it should not attempt to load the Mapbox GL dependencies when in build mode, as described here.
Depending on how you are initializing Mapbox GL in your components, you may also need to add a check to make sure that
window is defined before starting up Mapbox GL. In my case, it was as simple as adding a condition to the top of my functional component that wraps Mapbox GL, which immediately returns
null instead of rendering the component if
Not understanding how often functional components rerender
I migrated to functional components for everything in this application. These allow you to define your component as a function that renders based on properties passed in. However, I didn’t fully appreciate just how often these components re-render — even when the props passed in are the same! For some components, such as list items in a list of a few hundred items, I found that I needed to wrap these in
React.memo to memoize the functional components. In short, memoization basically instructs React to remember the last time it called that function based on certain incoming props, and use the result of that last call unless specific props (identified by us) change. Prior to memoizing some of these components, I was experiencing major performance issues due to re-rendering many of these components unnecessarily; afterward, the performance was significantly better.
This project was funded by support from the U.S. Fish and Wildlife Service — North Pacific Landscape Conservation Cooperative and National Oceanic and Atmospheric Administration — Habitat Division.