Aerial Visual Search of the Continental USA in 0.1 Seconds

Synced
Synced
Feb 26 · 4 min read
Image for post
Image for post

When a falcon spies a tiny quail or other prey in the distance it can dive from the clouds at a speed of over 300 kilometres per hour. Although humans are not naturally endowed with either of these skills, technology has brought us both — we can fly faster than the speed of sound, and now, thanks to advances in computer vision tech, we can visually search huge areas from the sky in the blink of an eye.

Visual search is the ability of an AI model to find images that are visually similar to a query image — a form of pattern recognition applied to the task of image retrieval. The scope and speed of a new visual search model for aerial images introduced by New Mexico’s Descartes Labs would make even a falcon envious.

“This system enables real-time visual search over the surface of the earth,” boasts the paper Visual Search Over Billions of Aerial and Satellite Images. The proposed system is capable of searching the continental United States at 1 -meter pixel resolution, corresponding to approximately 2 billion images, in around 0.1 seconds. As convolutional neural networks (CNN) trained on large image datasets have provided promising outcomes on extracting rich feature representations from photographic imagery, researchers used a CNN in their content-based image retrieval for visual search system.

To reduce data and compute requirements and ensure the visual search system could run in real-time, researchers defined visual similarity using 512 abstract visual features generated by a CNN that had been trained on aerial and satellite imagery. The conversion of these features into binary values played an essential role in the process because of the reduced data footprint.

Researchers chose to use a convolutional neural network with a 50-layer ResNet architecture that was pretrained on ImageNet. They noticed the last few layers of the ImageNet-trained network could deliver surprisingly good similarity search results for satellite imagery even though they were trained largely on photographic images of animals, plants and vehicles.

Image for post
Image for post

The team used two image datasets in the study: Aerial over USA and Landsat 8 over Earth. The former consists of aerial images from the National Agriculture Imagery Program and the Texas Orthoimagery Program. The latter was built by adapting data from one of the powerhouses for NASA’s satellite-based earth observation program — Landsat 8. Both datasets include a wide range of object types and landscapes ranging from industrial infrastructure such as wind turbines to natural features such as ponds.

Image for post
Image for post

The visual search model performed well on query objects from the classes used during the fine-tuning stage, such as pictures of wind turbines from the NAIP dataset. Although less impressive, the results on generic images that were not part of the supervised learning phase were also judged good enough to be leveraged for applications such as training downstream computer vision models.

The team also identified some shortcomings. For example although some low-quality search results include images that are visually similar to the query images, they failed to contain the query object class. The delicate balance between offering good search results for common object classes and generic class-agnostic search remains a challenge for further studies.

The researchers believe adding features such as multi-scale search, geospatial filtering and temporal filtering to the visual search system can provide an even more efficient tool for thoroughly searching and interpreting visual information in large collections of aerial and satellite imagery.

Visual Search Over Billions of Aerial and Satellite Images is available on ScienceDirect, and the system interactive demo is available at Descartes Labs.

Journalist: Fangyu Cai | Editor: Michael Sarazen

Thinking of contributing to Synced Review? Synced’s new column Share My Research welcomes scholars to share their own research breakthroughs with global AI enthusiasts.

Image for post
Image for post

We know you don’t want to miss any story. Subscribe to our popular Synced Global AI Weekly to get weekly AI updates.

Image for post
Image for post

Need a comprehensive review of the past, present and future of modern AI research development? Trends of AI Technology Development Report is out!

2018 Fortune Global 500 Public Company AI Adaptivity Report is out!
Purchase a Kindle-formatted report on Amazon.
Apply for Insight Partner Program to get a complimentary full PDF report.

Image for post
Image for post

SyncedReview

Synced

Written by

Synced

AI Technology & Industry Review — syncedreview.com | Newsletter: http://bit.ly/2IYL6Y2 | Share My Research http://bit.ly/2TrUPMI | Twitter: @Synced_Global

SyncedReview

We produce professional, authoritative, and thought-provoking content relating to artificial intelligence, machine intelligence, emerging technologies and industrial insights.

Synced

Written by

Synced

AI Technology & Industry Review — syncedreview.com | Newsletter: http://bit.ly/2IYL6Y2 | Share My Research http://bit.ly/2TrUPMI | Twitter: @Synced_Global

SyncedReview

We produce professional, authoritative, and thought-provoking content relating to artificial intelligence, machine intelligence, emerging technologies and industrial insights.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store