Using Computer Vision Model to Detect and Count Yellow Taxis in Manhattan
This project was developed by Urwa Muaz, Junjie Cai, and Manrique
Abstract
Governments spend large amount of their budget to maintain infrastructure every year [1]. This spending can be done much more efficiently if they understand and predict the needs of the city dwellers. This study uses computer vision model to automatically detect and count taxis in Manhattan, New York City. It provides “accounting” tools for local governments to implement local regulations, a method that be expanded to extensive fields.
Introduction
As part of a Street Data Collaborative projects lead by ARGO, our group of urban scientists from New York University’s Centre of Urban Science and Progress (NYU-CUSP) used computer vision to count taxis and street signs in New York City.
Throughout three weeks, we extracted street-level images and trained a Custom-Vision model. Scripts were developed to extract-load-and-transfer images from Google-Street-View and Open-Street-Camera and uploaded them to the Custom-Vision pre-trained model. This allowed us to count the number of taxis in determined locations.
Data
Google Street View
We extracted street-level images of Manhattan through Google Street View API. First, street shapefiles were fetched from NYC Open Data. The endpoint coordinates of every street segment are calculated from the street shapefile. Then we requested Google Street View API to download the images of the coordinates from 4 horizontal angles.
Open Street Camera
We extracted images from Open Street Camera given specific coordinates and the corresponding Track IDs. The endpoint coordinates from street shapefile are used to grasp nearby Open Street Camera tracks as a list of sequence IDs. Then we fetched all images in that track from Open Street Camera and filtered the images by coordinates. As a result, we got a large number of images for every street segment towards the roadway.
Methodology
Our model uses the extracted imaged and trains the computer vision model that detects objects.
Training
Training images from Google Street View and Open Street Camera were uploaded to the Custom vision platform. Then We manually selected the regions of taxis in images. Finally, we trained the model to identify yellow taxis. The model had a reliable performance with parameters as precision 97.4%, recall 84.4%, and mAP 95.9%.
Prediction
Through computer vision prediction API, we uploaded the images and got prediction results as JSON files that showed the tags of detected objects and probabilities. We set the probability threshold value as 50%, which means that the algorithm decides the detected object be a target if the probability is more than 50%. The image below in figure 1 shows a quick test of detection capabilities of the custom-vision model:
Object counting
With the prediction result, we can count the number of taxis in each image. We randomly selected 50 images from every neighborhood in Manhattan and counted the number of taxis per neighborhood.
Conclusion
As expected, the area with largest commercial activity, Upper-East-Side Carnegie Hall in Midtown Manhattan, has the largest ratio of taxis in Manhattan.
Limitations
It is difficult to obtain all the images from all the streets since many locations do not have images. Also, it is likely that taxis are repeated in different images. For this reason, the absolute count of yellow taxis cannot be counted, but instead, a “relative” number of taxis was compared them across neighborhoods.
Future Work
The development of this project can be applied in other cities to have a better understanding of our streets. This can provide “accounting” tools for local governments to implement local regulations for the cities.
Links
Follow more Streets-Data-Collaborative initiatives through our official website.
Bibliography
[1] Large economic gains can come from mundane improvements in policy
