Beyond Infrastructure Mapping — Finding Vehicles with Solaris

Jake Shermeyer
Aug 1, 2019 · 4 min read
Image for post
Image for post
Image for post
Image for post

A previous blog post introduced you to our new open-source framework Solaris, which aims to bridge the gap between difficult geospatial problems and new computer vision techniques. In this post we showcase how Solaris can be used for car segmentation (and detection/localization by proxy) using one of the models featured in SpaceNet 4. Although SpaceNet is focused on foundational infrastructure mapping challenges, the lessons learned and code developed can now easily be transferred to take on new problems (like vehicle detection) all thanks to Solaris.

If you wish to follow along with this blog or run this for yourself, a full jupyter notebook for analysis is provided here. We work under the assumption that you have installed Solaris and any dependencies as listed in the imports section of the notebook.

As in a few previous blogs [1, 2, 3] we work with the Cars with Overhead Context (COWC) dataset that spans 6 cities and has a spatial resolution of 15cm. The difference between this blog and previous blogs is that we use Solaris and a segmentation network instead of object detection to localize vehicles. After downloading the dataset, we must do some light preprocessing: This includes some data reorganization; tiling the images and masks; converting images and masks to GeoTiffs; and creating some csvs to document our file structure.

COWC masks identify cars with a single pixel with a value of 255, however this is a really small target for a neural network to identify. We want to enlarge the area identified as a vehicle a bit to increase visibility and provide a larger object that is easier for a neural network to detect. We use a simple square dilation operation with a size of 9 to accomplish this.

Image for post
Image for post
An example of an input image and its training mask after dilation.

We then split our data into two sets: 5 cities for training (randomly holding out 20% for validation), and 1 city for testing (Salt Lake City, Utah). This will help to showcase how transferrable our model is to a new location. We also go through the process of z-scoring our imagery. Z-scoring standardizes and normalizes our imagery across all of our locations, potentially helping to improve performance and model transferability.

Image for post
Image for post
Go away shadows.

Once you have your data preprocessed, create a yml file pointing to everything along with some settings for augmentations (among other things), you’re ready to train your net.

# Train a neural net with 3 lines of code.
config = sol.utils.config.parse('/path/to/yml/vehicleDetection.yml')
trainer = sol.nets.train.Trainer(config)
trainer.train()

We work with SpaceNet 4 participant XD_XD’s U-Net model with VGG16 encoder paired with two loss functions: jaccard (good for identifying small objects and imbalanced classes) and binary cross entropy with logits. Training time is about 7 hours on 2 NVIDIA Titan XP GPUs. Ultimately, inferencing is just as simple as training, with a few tweaks to the yml file. Inference speed for images 512 x 512 pixels in size is roughly 1,000 images per minute. Following inferencing, we binarize our model outputs into a car/not car map.

#Inference time
inf_df = sol.nets.infer.get_infer_df(config)
inferer = sol.nets.infer.Inferer(config)
inferer(inf_df)
Image for post
Image for post
Image for post
Image for post
Our image (Left), the ground truth (Middle), and our prediction from the segmentation network (Right).

We can also convert our binarized mask into some more palatable geospatial vectors using Solaris’ mask_to poly_geojson function. Finally, using our vectorized predictions and ground truth, we can score our results. Using the F1 metric (also known as the SpaceNet metric) with an IoU of 0.25 (acceptable for small objects) we score a solid 0.92.

We can then pull the centroid from each polygon using geopandas centroid function, and save the predictions as points. With a final touch of GIS symbology magic to convert the points into bounding boxes, we can show off our results.

Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Image for post
Our model does surprisingly well with dark vehicles, shadows, and even mild occlusions from tree cover.

Thanks for reading, be sure to check out our companion code for this blog and stay tuned for some more CV-geo action with Solaris from CosmiQ. Follow us on twitter @CosmiQWorks and @jshermeyer for more!

The DownLinQ

Welcome to the official blog of CosmiQ Works, an IQT Lab…

Thanks to Adam Van Etten and Nick Weir

Jake Shermeyer

Written by

Research Scientist at CosmiQ Works

The DownLinQ

Welcome to the official blog of CosmiQ Works, an IQT Lab dedicated to exploring the rapid advances delivered by artificial intelligence and geospatial startups, industry, academia, and the open source community

Jake Shermeyer

Written by

Research Scientist at CosmiQ Works

The DownLinQ

Welcome to the official blog of CosmiQ Works, an IQT Lab dedicated to exploring the rapid advances delivered by artificial intelligence and geospatial startups, industry, academia, and the open source community

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store