Eye in the sky: How satellite imagery can be used to solve real business problems

Using Satellite Imagery + AI to predict the financial health of a company

Gaurav Kaila
Aug 5, 2019 · 6 min read
Parking lot photo from Unsplash

To evaluate the financial health of a company, four main factors can be examined: liquidity, solvency, profitability and operating efficiency. Out of these 4 metrics, profitability is the best measurement of an institution's financial health. To survive in the long run, a company must attain and maintain profitability.

One of the key indicators of increasing profits is the number of active customers. For e.g., a brick and mortar store such as McDonald's would have high profitability and increased net margin (assuming operating costs remain the same) if more customers visit the store and buy its product. This is true for any customer-facing store such as Walmart, Decathalon, Starbucks, etc.

Predicting the number of customers for such stores could help us forecast the growth of the company and make better investment decisions.


Now there are many ways to predict the number of customers for a store:

  1. Manual survey of the number of people anytime inside the store
  2. Counting people remotely using in-store CCTV video stream

or doing what Sam Walton, the founder of Walmart, did:

It is rumored that Walton would count the number of cars outside the Walmart stores using aerial images taken by a plane flying over his stores to predict how well his stores were doing. Now, this could be real or fiction, but what is real is that in 2010 Neil Currie, an analyst at UBS, analyzed the satellite images of Walmart’s parking lot and suggested that its stocks are undervalued. Given this was 2010 and the resolution of satellite imagery wasn’t as good as it is now and neither did we have powerful computing, both from hardware and software perspective, Currie would have had to manually count the number of cars in the parking lot.

With the advances in satellite imagery and software methods, especially AI, there is now a way to automatically count the number of cars in a parking lot to predict how well the company is doing.


The motivation for this article came from this research paper from Berkeley Haas which describes the negative effects of using such technology to predict the financial health of a company that can influence the stock price.

This article will briefly describe the resources available to the general public to experiment in this domain. The reader can take inspiration from this article and use it as a starting point to develop such a system or pivot to other use-cases that can be solved using this technology.


Data

Cars Overhead with Context Dataset (COWC)

This dataset consists of ~33,000 unique cars from six different regions. There are both grayscale and RGB datasets available. Data is collected via aerial platforms, but at a nadir view angle such that it resembles satellite imagery. The imagery has a resolution of 15 cm GSD that is approximately twice as good as the current best resolution of commercial satellite imagery (30 cm GSD for DigitalGlobe). In order to make the COWC dataset resemble satellite images at 30cms GSD, you can downsample the images by a factor of 0.5 to match the current state-of-the-art satellite image resolution.

Now you have a dataset available where the location of each car is marked as a non-zero pixel. Assuming the mean car size of 3M and creating a bounding box around the non-zero pixel, you can generate training data that is ready to be ingested into the machine learning model described below.

Dataset for Object Detection in Aerial images (DOTA)

This dataset is collected from multiple sensors and platforms (e.g. Google Earth) with multiple resolutions, with a total of 2806 images and 188,282 annotated instances of the following classes: plane, ship, storage tank, baseball diamond, tennis court, swimming pool, ground track field, harbor, bridge, large vehicle, small vehicle, helicopter, roundabout, soccer ball field and basketball court. An added advantage of this dataset is that it splits up the vehicle class into two separate classes: large and small, leading to a more granular classification system.

Vehicle Detection in Aerial Imagery (VEDAI)

This is a benchmark dataset used for evaluating the model developed for counting the target object. It contains 1210 images with the following annotated classes: plane, boat, camping car, car, pick-up, tractor, truck, van, and other. It also provides an evaluation protocol to benchmark the developed model.

Overhead Imagery Research DataSet (OIRDS)

This dataset contains 900 annotated images. It contains five classes of vehicles: truck, pick up, car, van and unknown.

There are other open-source satellite imagery datasets available as well which could be used for this analysis but would require the manual labeling effort.

Algorithms

Object Detection

There are open-source algorithms available that have achieved reasonably good performance in the field of object localization such as RCNN and YOLO family of models. However, both of these family of models fail to identify really small objects in very large images such as cars in satellite images.

Luckily for us, there is an object detection model available that can alleviate this problem — YOLT2. YOLT2 is specifically built for identifying small objects in aerial imagery and has shown good performance in use-cases such as localization of buildings, boats, and aircraft in satellite images.

Another algorithm that has shown promise in this area in the Single Shot Detector (SSD). It is a uniform one-stage model that utilizes the feature maps at different scales. This algorithm and modifications of it have yielded successful results in recognizing really small objects in very large images

Semantic Segmentation

Identifying individual objects close to each other can be a difficult task for object localization models and can lead to poor performance especially when the end goal is to perform object counting. To alleviate this challenge, semantic segmentation is a method that helps in recognizing and understanding what is in the image at a pixel level. Variety of non-AI and AI-based methods are present to achieve this, but one of the more popular AI-based models to achieve this is U-Net, that has shown successful results in the biomedical image segmentation field. Modifying and training such a model for counting cars could lead to an increase in performance compared to solely using object detection methods as described above.

Existing solutions in the market

With the availability of high-resolution aerial imagery, computing power, and open-source AI algorithms, companies are capitalizing on this opportunity to provide advanced analytics solutions that can predict the financial health of the institution and many similar use-cases. Some of the companies are mentioned below:

1. Orbital Insight

2. RS Metrics

3. Picterra


The potential of Satellite Imagery is promising

The use of satellite imagery for addressing use-cases such as above is heating up. Agriculture departments all over the world are using such images to predict crop yield and analyze animal health. The extent of natural disasters is being studied by analyzing satellite imagery. National security use-cases such as counting cars at the border and analyzing weapon stockpiles in terrorist invested regions could benefit from such technology.

With advances in AI algorithms (and being open-source) and computing infrastructure (read: cloud), the development of the above use-cases is within the reach of the general public. This article intends to introduce the topic of combining satellite images with advanced AI methods for one such use-case and serves as a starting point for the development of many such applications.

Call for comments and feedback

I encourage the readers to join the conversation with me on IdeaChain where I have pinned down the premise of this idea. As a community, we can share our thoughts, collaborate and add more innovative ideas using the IdeaChain platform.


ideachain

A hub for ideas, discussion and collaboration…

Thanks to George Krasadakis

Gaurav Kaila

Written by

Data Science Manager @EY and Chief Data Scientist @IdeaChain; A hub for ideas, discussion and collaboration -http://ideacha.in

ideachain

ideachain

A hub for ideas, discussion and collaboration -http://ideacha.in

More From Medium

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade