Integrating with Kepler.gl for visual based spatial analytics
by Ali Kim, software engineer at Zoba. Ali holds a degree in Computer Science and Geochemistry from Brown University, loves topographical maps, prefers skateboards to other four wheeled vehicles, and looks forward to the future where all transportation is electric.
Zoba provides demand forecasting and optimization tools to shared mobility companies, from micromobility to car shares and beyond.
At Zoba, we leverage geospatial and time series data to help shared mobility companies across the world improve the performance of their fleets. Our platform provides deep insights and real-time operational strategies to clients. As many of our clients work in teams that span the globe, saving and sharing these insights is essential for surfacing street-level understanding that drives overall business strategy. To do this, we make use of kepler.gl; an open source, data agnostic, web application for geospatial analysis and visualizations.
We have found kepler.gl an invaluable tool, both internally and as something to build into the platform our clients use on a daily basis, and are extremely grateful to the kepler.gl team for all of their work. Out of the box, kepler.gl is an application that allows teams to create high-performance maps based on large-scale datasets. It is uniquely powerful for Zoba’s purposes since it supports custom layer creation and data manipulation which translate to configurable and intuitive visuals. All layer geometry calculations are GPU-accelerated, enabling smooth rendering of hundreds of thousands of points and, therefore, a seamless user experience. The result is an easy to use end-to-end geospatial data exploration environment that allows our clients to operationalize outputs from geospatial and temporal optimization models.
Zoba’s use case
Zoba is an automated operational decision engine. That is, we can integrate with a company’s internal flow to inform decision-making under the hood. While most of these decisions are API-driven, getting a holistic view of how a market operates is still extremely beneficial and can be furthered by visualizing data (both historical and forecasted) on a map. That is where our web application comes in and where we saw opportunity with kepler.gl.
For the most part, kepler.gl is an entirely client side application; any data you upload and visualize is contained entirely within your browser. This makes it accessible and scalable, but does not facilitate sharing or revisiting visualizations. Because these are two key features for organizations that serve markets globally, we wanted to both integrate kepler.gl into Zoba’s web portal and modify it to work with our backend, enabling our clients to perform CRUD (create, read, update, and delete) operations on all maps.¹
Below you’ll see what kepler.gl looks like built into the Zoba platform. On the frontend we made several modifications:
- We enabled users to save and load maps to and from our backend. This took several forms in our web app including the map dropdown and save button shown below in the top left of the kepler.gl sidebar. We also changed the flow of adding data to the map so that it’s possible to load in an individual dataset from our backend.
- We modified the header to include a search bar. This enables one to both easily locate addresses within a city as well as quickly navigate the globe. Since we added this feature a year ago, kepler.gl has also added the ability to geolocate addresses and drop pins.
- We added a scale in the bottom corner to better assess distances (seen in the bottom right of the image below). This is especially important when trying to make operational decisions based on visualizations. Although kepler.gl does not explicitly have a scale, they do expose the underlying Mapbox map with which it can be configured (details below).
This was all rather straightforward thanks to the work done by the kepler.gl development team; in the rest of this post we’ll walk through the basics of integration and customization that made these changes possible. We’ll also look at some issues that we ran into during the process and how we solved them.
Integration
Kepler.gl has a pretty comprehensive guide for those looking to integrate it into their own web application. You can find that here. As such, I’ll cover the basics but focus on insights that we found helpful while going through the process.
Kepler.gl is a React application that uses Redux for state management. This post will assume a basic working knowledge of both, but if you’re not familiar with either technology there are a lot of good resources online to get you started (e.g. here).
Another important point worth mentioning is that kepler.gl is built on top of MapboxGL and therefore requires a Mapbox token to display the map. You can get an access token by creating a free account on mapbox.com. The token will then be used when mounting the KeplerGl component.
Setting up the Redux store
If you don’t use Redux, you’ll need to introduce it into your application for kepler.gl to work. This consists of creating a store.js file where you mount the kepler.gl reducer as follows.
store.js1 import keplerGlReducer from "kepler.gl/reducers";
2 import {createStore, applyMiddleware} from "redux";
3
4 export default createStore(keplerGlReducer, {}, applyMiddleware(taskMiddleware));main.js1 import React from 'react';
2 import document from 'global/document';
3 import {render} from 'react-dom';
4 import {Provider} from 'react-redux';
5 import App from './app';
6 import store from './store';
7
8 const Root = () => (
9 <Provider store={store}>
10 <App />
11 </Provider>
12 );
13
14 render(<Root />, document.body.appendChild(document.createElement('div')));
We already managed our web application state using Redux, and it was just as easy to combine the kepler.gl store with our existing store by using Redux’s `combineReducers` feature.
store.js1 import keplerGlReducer from "kepler.gl/reducers";
2 import {createStore, combineReducers, applyMiddleware} from "redux";
3 import {taskMiddleware} from "react-palm/tasks";
4 import appReducer from "./reducers/index.js";
5
6 const reducer = combineReducers({
7 // kepler.gl reducer
8 keplerGl: keplerGlReducer,
9 // And any other existing reducers
10 app: appReducer
11 });
12
13 const store = createStore(reducer, {}, applyMiddleware(taskMiddleware));
In either case, it is often helpful to add Redux DevTools to your application for debugging purposes.
store.js1 import keplerGlReducer from "kepler.gl/reducers";
2 import window from "global/window";
3 import {createStore, applyMiddleware, compose} from "redux";
4 import {taskMiddleware} from "react-palm/tasks";
5
6 export const enhancers = [applyMiddleware(taskMiddleware)];
7
8 // This adds redux devtools to help in debugging
9 const composeEnhancers = window.__REDUX_DEVTOOLS_EXTENSION_COMPOSE__ || compose;
10
11 export default createStore(keplerGlReducer, {}, composeEnhancers(...enhancers));
If you’re wondering what all the middleware stuff is doing in there: Redux middleware is code run after an action has been dispatched but before it reaches the reducer. Don’t worry so much about this now, kepler.gl just uses react-palm to handle side effects which helps with testing.
Mounting the KeplerGl component
Once the store has been set up, we’re ready to mount the KeplerGl component. There are several props that allow for configuration, you can find a list of those here.
1 import KeplerGl from "kepler.gl";
2
3 const Map = props => (
4 <KeplerGl
5 mapboxApiAccessToken={MAPBOX_TOKEN}
6 id="map"
7 width={width}
8 height={height}
9 getState={state => state}
10 />
11 )The KeplerGl component takes in a `height` and a `width`, both of which expect a number corresponding to the value in pixels. This is straightforward if you have set dimensions for the map, but we wanted the map to take up the entirety of a div with variable dimensions. For this case (or a similar one where the desired map is full screen), a good solution is to use AutoSizer (also see the implementation in the demo-app).
1 import AutoSizer from "react-virtualized/dist/commonjs/AutoSizer";...10 <div style={{position: "absolute", width: "100%", height: "100%"}}>
11 <AutoSizer>
12 {({height, width}) => (
13 <KeplerGl
14 mapboxApiAccessToken={MAPBOX_TOKEN}
15 id="map"
16 width={width}
17 height={height}
18 />
19 )}
20 </AutoSizer>
21 </div>
Debugging
At this point, the map should render in your application. If it doesn’t, checking the console for errors should help identify the issue. Often the culprit is the redux state; if kepler.gl can’t find its state, the following message will appear in the console:
`kepler.gl state does not exist. You might forget to mount keplerGlReducer in your root reducer.If it is not mounted as state.keplerGl by default, you need to provide getState as a prop`
As suggested, the solution is to pass the `getState` prop to the KeplerGl component. The default value of `getState` is `state => state.keplerGl`, so if the KeplerGl state is mounted anywhere else, you’ll need to overwrite this.
If you’ve enabled Redux DevTools earlier, you can use these to look at the Redux state. In the first example of setting up the store above, we used the keplerGlReducer as our app reducer, which resulted in a structure like this:
Here, the kepler.gl state is the root state of our application and not located under `keplerGl` as assumed by default, so we would need to pass `getState` in like so: `getState={state => state}`. (note that `map` is the root because it’s the `id` we passed in to the KeplerGl component).
Customization
Great! We now have kepler.gl embedded into our application. At Zoba, we then started to consider how we could customize it to match some additional requirements: our custom color palette and a few supplementary feature requests. Thankfully, kepler.gl offers two solutions: pass customizable parameters into the KeplerGl component or inject our custom components into kepler.gl. The former is best when making smaller, more cosmetic changes (such as the aforementioned color palette), while the latter accommodates larger structural changes.
Customizable parameters
Let’s first look at customizable parameters. Earlier, I linked to the list of props accepted by the KeplerGl component. We began by changing the name and version that appear at the top of the sidebar. Both of these can be achieved by just passing in the `appName` and `version` as parameters.
1 const Map = props => (
2 <KeplerGl
3 mapboxApiAccessToken={MAPBOX_TOKEN}
4 id="map"
5 appName={“Professor Oberon’s Map”}
6 version={“v3.0”}
7 />
8 )This is also the place to customize the color palette by overriding the default theme. The `theme` parameter can be given a theme name (as a string) or an object. Kepler.gl has three built-in themes, the names of which are “dark”, “light”, and “base”. If an object is passed in, it will be used to overwrite specific values and everything else will default to the values from the dark theme. See the source code for all of the possible values as well as their defaults.
1 const theme = {
2 sidePanelBg: '#ffffff',
3 titleTextColor: '#000000',
4 sidePanelHeaderBg: '#f7f7F7',
5 }
6
7 const Map = props => (
8 <KeplerGl
9 mapboxApiAccessToken={MAPBOX_TOKEN}
10 id="map"
11 theme={theme}
12 />
13 )Another perhaps less intuitive possibility for customization using the parameters is with the `getMapboxRef` prop. This allows you to specify a callback function which is called whenever kepler.gl adds or removes a MapContainer, the general component that holds the lower level Mapbox map. You can use it to perform certain actions on the Mapbox reference which kepler.gl may not expose. For example, we did the following to display a scale on the map.
1 getMapboxRef = (mapbox, index) => {
2 if (mapbox) {
3 const map = mapbox.getMap();
4 const scale = new mapboxgl.ScaleControl({
5 maxWidth: 120,
6 unit: 'metric'
7 });
8 map.addControl(scale, 'bottom-right');
9 }
10 };
11
12 const Map = props => (
13 <KeplerGl
14 mapboxApiAccessToken={MAPBOX_TOKEN}
15 id="map"
16 getMapboxRef={getMapboxRef}
17 />
18 )Injecting custom components
For flexibility when customizing, the engineers working on kepler.gl have provided a dependency injection system which makes replacing kepler.gl components with customized alternatives easy. After creating a component factory (simply put, a factory is just a function that generates a component) for the component you wish to replace, you just have to import kepler.gl’s version and use `injectComponents` wherever KeplerGl is mounted in your application.
Any component whose factory is exposed by kepler.gl (so any exported in this file) can be replaced with a custom version. As development on kepler.gl has progressed, more and more components have been added to this list. Note that the larger the component, the more likely a future update will touch something within it. Early on, we ran into some complications in cases where we only wanted to adjust one small thing but had to replace the larger component it was contained in to do so. This resulted in having to make manual updates to the injected components when we updated the package. The frequency of this has decreased since lighter weight components have been added to the exports, but it’s worth being cognizant of.
For example, we soon decided that the custom name and version we added above were not quite satisfactory and wanted to instead replace the entire sidebar header with our own. After building our custom header component, we just created a factory and injected it into kepler as follows (we did this in separate files for organizational purposes, but it can also be contained to whichever file you mount the KeplerGl component):
factories/header.js1 import {PanelHeaderFactory} from "kepler.gl/components";
2 import ZobaHeader from "../components/header";
3
4 export const CustomHeaderFactory = () => ZobaHeader;
5
6 export function replaceHeader() {
7 return [PanelHeaderFactory, CustomHeaderFactory];
8 } app.js1 import {injectComponents} from 'kepler.gl/components';
2 import {replaceHeader} from './factories/header';
3
4 const KeplerGl = injectComponents([
5 replaceHeader()
6 ]);
7
8 // render KeplerGl, it will render your custom header instead of the default
9 const MapContainer = () => (
10 <div>
11 <KeplerGl id="foo" />
12 </div>
13 );
As we kept the “Save Map” functionality in our custom header, we needed it to have access to some of the kepler.gl state. Kepler.gl provides a `withState` helper for this reason, which allows you to pass state and actions to your custom component. In the following snippet, customMapStateToProps is a function that maps any part of the application state to the props of the component and customAction could be any action creator you wanted to use in the component. See more documentation on this here.
factories/header.js1 import {withState, PanelHeaderFactory} from 'kepler.gl/components';
2 import ZobaHeader from "../components/header";
3
4 export const CustomHeaderFactory = () => withState(
5 [visStateLens],
6 customMapStateToProps,
7 {customAction}
8 )(CustomHeader);
9
10 export function replaceHeader() {
11 return [PanelHeaderFactory, CustomHeaderFactory];
12 }
There are a lot more ways to customize kepler.gl including dispatching custom actions, introducing additional middleware, and replacing the base map styles with custom ones. I’d recommend taking a look at their Advanced Usage documentation for more information.
Hopefully this gave you a good sense of how we went about embedding kepler.gl into our application and helps you avoid some of the issues we ran into when doing so. At Zoba, most of the frontend modifications to kepler.gl we made were to facilitate the interactions with our backend that enable CRUD operations on all created visualizations. Kepler.gl has been a critical part of enabling our clients to explore complex geospatial data in an intuitive way that lends itself to operational decision-making. While this post focused on the frontend changes, we also built backend functionalities around kepler.gl’s map data. In a future post, we will dive deeper into that backend integration: how we store maps, save styles, and programmatically generate maps with saved styles and new data.
¹ In the past year, kepler.gl has added integrations for Dropbox and Carto that serve this purpose as well.
Zoba is developing the next generation of spatial analytics in Boston. If you are interested in spatial data, urban tech, or mobility, reach out at zoba.com/careers.
