Measuring New York City Transit: A Brief Overview and Critique of MTA Metrics
New Yorkers are not a shy bunch when it comes to talking about their transportation. Candid, pointed reviews of transit performance come with the territory of living here. But these reviews, which are a constant indicator of the MTA’s performance on any given day, do not correlate with how the MTA, in fact, measures its own performance and reports it to riders.
Most mornings after I arrive to the office, I gather with co-workers to make my morning coffee and overview our commutes. Was the trip smooth? Were trains and buses overcrowded? Were there delays? Was someone lucky enough to get a seat?
This overview is a daily ritual and revolves around a singular subject: quality and reliability of transit service, two essential indicators of transit performance. The availability of public transit services, and their quality and consistency, is a constant factor of life for New Yorkers. It is no small task to ensure that transit keeps the city moving; the MTA runs a mammoth-sized transit system, and the city’s day-to-day life depends on it. Metrics and key performance indicators are one way of ensuring that it does. The MTA, for its part, considers the following metrics for subway and bus services provided:
Subways: On-Time Performance, Subway Wait Assessment, Elevator Availability, Total Ridership, Mean Distance Between Failures
Buses: % of completed Trips — MTA Bus, Bus Passenger Wheelchair Usage, Mean Distance Between Failures
In assessing these metrics, it becomes clear that there exists a gap between how my co-workers and I measure the quality of our commutes, and how the MTA measures its performance and reports on it to the public. These metrics, for the most part, simply overview whether or not transit has been operational, with little to no information on much else. They don’t describe the state of the system, or the quality of services provided. Overcrowding on a train or bus, for example, or anything related to a passenger’s experience inside a subway or bus, is not relevant to these performance metrics. Riders are left, instead, to take leaps of logic regarding how their transit experience might somehow be deduced from the operational measurements included in the screenshot above.
Metrics and performance indicators, ideally, should be used as accountability benchmarks and as goals to strive towards. The MTA, for all of these metrics, only sets rough goals for each one, indicating whether it wants an “increase” or “decrease” for each indicator: on-time performance should “increase” and mean distance between failures should “decrease”. The question then is to what extent? And what is the MTA’s next step or plan of action if an indicator is showing under-performance?
Though these metrics show the system is operational on a fundamental level, they unfortunately don’t enlighten the public on much else, including whether improvements to the system are having a positive impact. To that end, the agency is hardly idle: it is currently pushing forward a number of new services, among them Select Bus Service, new stations (Fulton Terminal, for example), and an increasing range of digital services like free wifi in subway stations and real-time announcements of train arrivals. The agency also regularly assesses schedules to determine whether subway and bus frequency is appropriate given demand and ridership, and increases or decreases it accordingly. But given the agency’s current metrics, it’s impossible for the public to measure the impact of these changes, or whether larger-scale roll-outs of like services, service changes, or new technology might result in marked improvements to the system’s performance.
The agency would do well to re-examine how it measures its performance and reports on it to the public, and to set more specific targets for the metrics it already has. Doing so would benefit both the MTA, which could point towards the impact of specific improvements, as well as the general public, which would benefit from being better able to assess the state of transit in New York City.
