Creating Interactive Historical Maps With React, Mapbox, and Wikipedia

One of the biggest flaws with how schools teach history is that they’re rarely able to tie things together into a cohesive global picture. Students learn about individual events and figures in great detail, but few are able to simultaneously describe the state of the world across geographies in a given era.

I’m a visual learner, so I’ve been thinking about the sorts of web apps that could help engage with history in a more interactive and scalable way. At a high level, I wanted to make something that enables a user to:

1. Control the geographical search space
2. Control the returned information chronologically
3. Easily navigate to finer-grain detail if desired

Below I’ll describe the app I came up with — it’s very much a work in progress, but hopefully you’ll find the code, the APIs used, or any of the design decisions made so far useful. The full code is available in this github repo.

The Back End

For an application like this to be successful, the first requirement is data — and a lot of it. While exploring any given time period, a user needs the app to return data about that period, with high-level information, links, and other attributes immediately available. The easiest way to structure this requirement is to create a model around historical events, which fit nicely into a database model and are easy to enumerate programmatically .

So how does one acquire a large amount of highly structured data about historical events and populate a database with those entities? Fortunately, a project called DBpedia provides a public database built on top of Wikipedia entries that can be queried with a SQL-like language called SPARQL. This means that not only can the data be scraped once to build out an initial database, but our app’s back end can also refresh the database as new events are added to the public records.

The code for this step, implemented in Python and loaded into a MySQL database, can be found here. The key features in the SPARQL query itself are that we’re able to query event names, dates, location data, and links to additional resources; the results of this query can then easily be inserted into a database table with the same fields:

PREFIX dbo: <http://dbpedia.org/ontology/> 
PREFIX n1: <http://www.w3.org/2003/01/geo/wgs84_pos#> 
SELECT 
    (?Event_69 AS ?event) 
    (?date_89 AS ?date) 
    (AVG(?place_lat_158) AS ?latitude) 
    (AVG(?place_long_159) AS ?longitude) 
WHERE { 
    ?Event_69 a dbo:Event . 
    ?Event_69 dbo:date ?date_89 . 
    ?Event_69 dbo:place ?place_92 . 
    ?place_92 n1:lat ?place_lat_158 . 
    ?place_92 n1:long ?place_long_159 . } 
GROUP BY 
    ?Event_69 
    ?date_89 
ORDER BY 
    ?date_89

Now that we have a database full of clean historical event data, we need a way to expose that data to the front end of the app. A natural solution for this is an API, which we’ll build out with Express, a web framework for Node.js.

The most important piece of the API code is a simple handler for a GET request to the events endpoint:

// List events
app.get('/events/:year', (req, res) => {

  const year = req.params.year;

  pool.query(`
    SELECT * FROM timeline_event WHERE date LIKE ?`,  [year + "%"], 
    (err, results) => {
      if (err) {
        console.log(err);
        return res.send(err);
      } else {
        return res.send(results);
      }
    });
});

The endpoint accepts a single parameter, year, which is used as a filter while querying the timeline_event table. An example request and response might look like:

// http://<base_url>/events/1950
[
 {
 “id”: 24248,
 “name”: “Battle of Bamianshan”,
 “url”: “http://dbpedia.org/resource/Battle_of_Bamianshan",
 “date”: “1950–01–19”,
 “latitude”: 37.4583,
 “longitude”: 109.692
 },
...
]

With this API available, we can make queries like the above from our front-end code and use these results to populate a dynamic map.

The Front End

The primary feature of the front-end code is a WorldMap React component that serves as a wrapper for the open-source ReactMapGL component. The latter integrates with Mapbox’s public API, so it requires setting up an account with them and generating an access token, but once that’s done it makes rendering an interactive map as easy as inserting a snippet like this into our React app:

<ReactMapGL
    {...viewport}
    width="90w"
    height="80vh"
    mapStyle="mapbox://styles/mapbox/outdoors-v11"
    mapboxApiAccessToken={process.env.REACT_APP_MAPBOX_TOKEN}
></ReactMapGL>

A map alone isn’t of much use without a way to render our events and navigate through the historical timeline. For this, we’ll use a “range-slider” input that lets users drag a button in the web UI that updates the current year displayed on page.

The input year can then be used to make a GET request to the back end API we set up above, which will return the events for that year in our database. Each event has a latitude and longitude associated with it, making it possible to render the event as a Marker component on our map.

Sample navigation within the Timeline app

The full code for the WorldMap component can be found here; I’ve added some extra functionality such as popups with links to more info for each event, click event handling, and component state management.

Orchestration and Deployment

So how can we actually run any of this code and make the services we’ve created accessible to each other? I’ve been a fan of Docker Compose lately for managing apps with multiple services, and it’s a pretty good fit for this case as well given that there’s a database, an API back end, and a front-end client involved.

Docker Compose lets you organize each service in a container and then manage dependencies, networking, environment variables, entrypoints, and more from a single YAML configuration file. For this example, there are three services, the database, the front-end client, and the back end API, named “mysql”, “client”, and “server” respectively:

version: "3.4"

x-common-variables: &common-variables
  MYSQL_USER: ${MYSQL_USER}
  MYSQL_PASSWORD: ${MYSQL_PASSWORD}
  MYSQL_DATABASE: ${MYSQL_DATABASE}
  REACT_APP_MAPBOX_TOKEN: ${REACT_APP_MAPBOX_TOKEN}
  REACT_APP_SERVER_PORT: ${REACT_APP_SERVER_PORT}

services:
  mysql:
    image: mysql:5.7
    environment:
      <<: *common-variables
      MYSQL_HOST: ${MYSQL_HOST}
      MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD}
    ports:
      - ${MYSQL_PORT}:3306
    restart: unless-stopped
    volumes:
      - ./db/ddl.sql:/docker-entrypoint-initdb.d/ddl.sql
      - mysql-data-volume:/var/lib/mysql
  client:
    build: ./client
    environment:
      <<: *common-variables
      NODE_PATH: src
    expose:
      - 3000
    ports:
      - ${CLIENT_PORT}:3000
    volumes:
      - ./client/src:/app/src
    stdin_open: true
    command: npm start
  server:
    build: ./server
    depends_on:
      - mysql
    expose:
      - ${REACT_APP_SERVER_PORT}
    environment:
      <<: *common-variables
      MYSQL_HOST_IP: mysql
    ports:
      - ${REACT_APP_SERVER_PORT}:${REACT_APP_SERVER_PORT}
    volumes:
      - ./server:/app
    links:
      - mysql
    command: ["./run_server.sh"]
volumes:
  mysql-data-volume:

More details on the variables used here can be found in the github repo for the app.

To build the containers run $ docker-compose build from the same directory as “docker-compose.yml”, and to start the services run $ docker-compose up -d.

This will build the docker images and run the containers locally, so navigating to “localhost:<CLIENT_PORT>” in your browser after executing these commands should display the running app.

Final Notes

There are a number of missing pieces and unsolved problems to the approach taken above. The most obvious is how sparse and unequally distributed the DBpedia data is; there are likely more comprehensive data sources out there, but I wasn’t able to find any readily available public alternatives with a more satisfactory size and level of detail. (If you’re aware of any please do reach out and let me know.)

Another weakness is that I haven’t yet accounted for B.C.E. history, or the fact that as history stretches further back in time, our record of events is spread thinner. A log scale of some sort for the input dates would likely help with this.

With all that said, putting this app together was a great intro to Mapbox and the public Wikipedia databases, as well as a fun exercise in React, and I hope that others will find these resources as fun to build with as I did.