Adventures in Amateur Cartography

or, Making a map I couldn’t find anywhere else

A few months ago I moved to a different neighborhood and started exploring more. My list of preferred ways of getting around the city (in order) goes:

  1. Walk
  2. Bike
  3. Take public transit
  4. Taxi/Lyft/Uber
  5. Ride from a friend going that way
  6. Hitchhike with a stranger
  7. Run/jog
  8. Drive a car

For much of my day to day travels, walking or biking just aren’t as much of an option as I’d like them to be. I usually want to get where I’m going within a reasonable amount of time, and I almost always don’t want to show up sweaty and holding a bike helmet. I end up taking a bus, trollybus, streetcar, trollycar, lightrail, or cable car (herein after referred to as “bus”) most everywhere I go.

I’ve lived in San Francisco for several years, but always within a couple blocks of the initial friend’s-couch-to-crash-on apartment I stayed in when I first got here. I thought I knew the MUNI system pretty well, but it turned out that I just knew the MUNI busses that passed through my old neighborhood.

When I use the internet to look up directions I will look at the map to see the whole route rather than the turn-by-turn directions. I like being able to see the whole route. I like having a map in my head of the area. I like being able to go to a bus stop, look at the route listing, and know if any of the busses listed can get me closer to my destination.

For instance, if I’m waiting at Van Ness and Market street and I want to get to Haight-Ashbury, a quick check of Google Maps might tell me to wait for the 6. If I have a better sense of the bus system I would know I could take the 6 or the 7, and depending on where I’m headed, I can take an N train walk an extra 3 or 4 blocks and the time saved by going underground might get me there faster.


I set out to learn the bus system by trying to find a map that would let me easily explore a bus line’s entire route, in context with other bus routes and streets. Of course, the canonical way is with a MUNI map, and a great copy can be downloaded from their website:

https://www.sfmta.com/maps/muni-system-map
Aside: When I first moved to San Francisco, I bought a paper copy of the MUNI map and hung it by the door to the apartment as decoration. In retrospect, that was a major part of how I learned my way around the city, and I highly recommend that to anyone else who is not adverse (and/or whose partner is not adverse) to the whole maps-as-decor aesthetic.

The limitation with the MUNI system map twofold: to find an unfamiliar route one has to search the entire map and to see an individual route from start to finish one has to trace it. The MTA does provide a map of individual routes (two, actually):

https://www.sfmta.com/getting-around/transit/routes-stops/27-bryant
https://www.sfmta.com/getting-around/transit/routes-stops/27-bryant

The first is, to be frank, pretty much useless as it provides almost zero context. Unless the question you’re asking is “does this bus go from this general area of the city to that general area of the city?” or you have an amazing ability to overlay a mental street map on top of a landmass outline, this map will not help.

The second is much more helpful, but lacks the feature that allows one to see the entire transit network at once and jump quickly to another route. To see another route map, one has to navigate through the MUNI website, which can be clunky.

I couldn’t find exactly what I wanted on the internet, so I decided to make it. I ended up with something that looks like the following:

https://thfield.github.io/sf-bus/

I can see the entire system in context with streets, zoom and pan, highlight bus routes by name, and see the entire route from start to finish.


The first step in creating this transit route map is finding the data. Luckily for this project, the SFMTA provides data on the entire transit system in a standardized format, the General Transit Feed Specification.

We want to use this data to create GeoJSON linestrings for each SFMTA bus route, then draw and provide some interaction with those linestrings on a web map using Mapbox and D3.

Downloading the GTFS zip file from SFMTA provides several text files (comma separated) with data describing every scheduled trip, every stop, every fare, every route name, every prescribed path for the entire system. The three files that are most important to this map are “routes.txt”, “shapes.txt”, and “trips.txt”.

routes.txt

“routes.txt” identifies each route with a unique key and lists the route names we are familiar with: 1-California, 22-Fillmore, J-Church, etc. We want to get the route_id, route_short_name, androute_long_name.

shapes.txt

“shapes.txt” contains lists of coordinates representing every different path that a bus will drive while making a trip. A bus on the same route might drive a different path depending on the time of day or day of the week. For example, on weekdays during the day the 5-Fulton route goes between the Transbay Terminal and 7th Avenue/Fulton St, but in the evenings and on weekends the 5-Fulton route goes between the Transbay Terminal and Ocean Beach; same route, different path. Another reason paths might differ is because of one-way streets. This is obvious in the difference between inbound and outbound paths for the 1BX-California:

The properties we’ll use from “shapes.txt” are shape_id, shape_pt_lon, and shape_pt_lat.

trips.txt

“trips.txt” is a list of every single time a bus is scheduled to make a trip from start to finish. We can use that data to figure out the “most common” trip to represent the path of the bus route. We’ll need to make note of the properties route_id, direction_id, and shape_id.

We’ll also need to note from “calendar.txt” that service_id can be 1, 2, or 3, meaning the trip schedule applies on weekdays, Saturdays, or Sundays respectively.


(Most of the following describes the script I wrote to parse the GTFS data files and transform them into GeoJSON files. That script can be found on Github.)

Using “routes.txt” we can define a Map between the route_id and route_short_name/route_long_name. This will allow us to assign a human readable name to each route.

By using “trips.txt” we’ll count how many times a shape_id is used for a trip, at the same time keeping track of the route_id and sorting each trip by its calendar schedule (weekday/weekend) and direction (inbound/outbound or East/West or North/South). Then we’ll set up another Map between route_id and the path, or shape_id, of the most common path for a route on a given calendar schedule going a given direction. This will give us a representative path for both directions of each route on weekdays and weekends.

Finally, using “shapes.txt”, we will create reate a dictionary between route_id and path, which is described by an array of coordinates (shape_pt_lon and shape_pt_lat).

With the two Maps and dictionary in hand, we then create a GeoJSON string for each route by joining 1) human readable name and the 2) path for the most common trip 3) to the path coordinates to the using route_id and shape_id.

As a result, we get a series of GeoJSON files that look something like the following:

{"type":"Feature","properties":{"shortName":"27","longName":"BRYANT","direction":"A"},"geometry":{"type":"LineString","coordinates":[[-122.422993,37.794017],[-122.423061,37.79394],[-122.422884,37.793055],[-122.421243,37.793262],[-122.419592,37.793472],[-122.417948,37.793688],[-122.417772,37.792808],[-122.417678,37.792343],[-122.417595,37.791927],[-122.417404,37.790978],[-122.417217,37.79005],[-122.417029,37.789115],[-122.415384,37.789324],[-122.413741,37.789534],[-122.413554,37.788603],[-122.413425,37.787963],[-122.413367,37.787668],[-122.413307,37.787388],[-122.413174,37.786734],[-122.412982,37.785794],[-122.41216,37.785898],[-122.411338,37.786001],[-122.409696,37.786207],[-122.40951,37.785288],[-122.409322,37.784354],[-122.408506,37.784457],[-122.408539,37.784358],[-122.408079,37.783996],[-122.407495,37.783534],[-122.407197,37.783297],[-122.407028,37.783162],[-122.406492,37.782736],[-122.405944,37.782301],[-122.405479,37.78193],[-122.404946,37.781504],[-122.404416,37.781081],[-122.403933,37.780704],[-122.4034,37.78027],[-122.402853,37.779833],[-122.402717,37.779724],[-122.402532,37.779576],[-122.40239,37.779463],[-122.401856,37.779037],[-122.402397,37.778609],[-122.402974,37.778154],[-122.403542,37.777704],[-122.404083,37.777277],[-122.404623,37.776851],[-122.404909,37.776625],[-122.405489,37.776167],[-122.406309,37.775519],[-122.406381,37.775461],[-122.406841,37.775098],[-122.407418,37.774642],[-122.40778,37.774356],[-122.408529,37.773764],[-122.409696,37.772842],[-122.410089,37.772531],[-122.410691,37.772057],[-122.41129,37.771583],[-122.412492,37.770634],[-122.410944,37.769416],[-122.410844,37.769236],[-122.410826,37.769058],[-122.410743,37.768182],[-122.410622,37.766908],[-122.410499,37.76561],[-122.410376,37.764315],[-122.410257,37.763034],[-122.410131,37.761764],[-122.41001,37.76049],[-122.409933,37.759571],[-122.409783,37.759199],[-122.409748,37.759113],[-122.409668,37.758451],[-122.409581,37.757731],[-122.409601,37.7576],[-122.40948,37.756089],[-122.409446,37.756003],[-122.409362,37.754968],[-122.409266,37.754403],[-122.409251,37.754266],[-122.409279,37.754228],[-122.409142,37.7528],[-122.40899,37.751205],[-122.408838,37.749605],[-122.408719,37.748357],[-122.409659,37.748339],[-122.410559,37.748321],[-122.411497,37.748303],[-122.412596,37.748281],[-122.413681,37.74826],[-122.414932,37.748235],[-122.415875,37.748217],[-122.418234,37.74817],[-122.418919,37.748163],[-122.418964,37.748216]]}}

Now that the schedule data has been transformed into GeoJSON linestrings representing the most common path in both directions for each route, the remaining task it to draw those linestrings on a map.

I used D3, Leaflet, and Mapbox for this because 1) I’ve used them before 2) I like open source libraries and 3) Mapbox tiles are very pretty.

That task is accomplished with a script that functions as a front-end app, which can be found here. “map.js” requests a list of bus routes and creates a list item for each route and direction. It attaches some event handlers which provide user interaction. Then “map.js” starts up a Leaflet map, requests tiles from Mapbox, uses D3 to download the GeoJSON linestrings for the bus routes, draws them on the page, and attaches some additional event handlers.

The result is a fairly simple map that allows one to explore different bus routes easily and quickly: exactly what I wanted to be able to do.


My map is live at https://thfield.github.io/sf-bus/

The source code can be found on Github.

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.