Super-Resolution and Object Detection: A Love Story- Epilogue

Jake Shermeyer
Jun 26, 2019 · 3 min read

Jake Shermeyer & Adam Van Etten

Image for post
Image for post
The effects of super-resolving 30cm and 60cm imagery to 15cm in Brasília, Brazil. Imagery courtesy of DigitalGlobe.

A Brief Overview:

Image for post
Image for post
Figure 1: Object detection performance as a function of sensor resolution for unenhanced native resolution imagery (SSD and YOLT) for five unique classes of objects.

We then applied super-resolution with two methods across these five resolutions. We used a standard computer vision baseline and neural network- Very Deep Super-Resolution (VDSR) and computationally inexpensive CPU only technique- Random Forest Super-Resolution (RFSR). We enhanced imagery 2x and 4x across each of these resolution ranges and then tested the performance tradeoffs. Our findings showed that super-resolution was most effective in the finest resolutions and became less effective as imagery becomes coarser. Overall the technique is a computationally inexpensive (~0.16 ms per 544x544 pixel image) and an effective pre-processing step for object detection applications (Figure 2).

Image for post
Image for post
Figure 2: The performance boost from 2x Super-Resolution for YOLT (change in mAP versus the blue line in Figure 1). Note that the performance boost is only valuable in the finest resolutions.

Some Fresh Findings:

Image for post
Image for post
Figure 3: The benefit of enhancing 30 and 60cm imagery to 15cm.

In Figure 3 we have three subplots: On the leftmost plot we display object detection performance with 30cm (Green) and 60cm (Red) native resolution imagery. On the center and right plots we present the performance boost of enhancing imagery from 30 to 15cm (Teal) and 60 to 15cm (Orange) with VDSR and RFSR respectively.

The results of our study showcase several interesting bits of information:

A) When super-enhancing 60cm imagery to 15cm, object detection performance is improved by 11% vs. native 30cm imagery and by 20% vs. native 60cm imagery.

B) When super-enhancing 30cm imagery to 15cm, object detection performance is improved by 14% vs. the native 30cm imagery.

C) Once super-resolved to 15 cm, object detection performance for 30 and 60 cm imagery is equivalent (within errors)

D) Both a neural network (VDSR) and a simpler less computationally expensive method (RFSR) can provide similar enhancements to object detection performance when enhancing data from 60 or 30cm to 15cm.

Conclusions:

The DownLinQ

Welcome to the official blog of CosmiQ Works, an IQT Lab…

Jake Shermeyer

Written by

Research Scientist at CosmiQ Works

The DownLinQ

Welcome to the official blog of CosmiQ Works, an IQT Lab dedicated to exploring the rapid advances delivered by artificial intelligence and geospatial startups, industry, academia, and the open source community

Jake Shermeyer

Written by

Research Scientist at CosmiQ Works

The DownLinQ

Welcome to the official blog of CosmiQ Works, an IQT Lab dedicated to exploring the rapid advances delivered by artificial intelligence and geospatial startups, industry, academia, and the open source community

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store