Searching the World Wide World

Today, we’re unveiling a technology demonstration of machine learning at global scale, which we call GeoVisual Search. This release makes the global composite images we released last month a bit smarter.

Click anywhere on the map and we’ll scan the US, China, or the entire globe to find things that look similar. Click on a baseball diamond, a wind turbine, or even a house with a moat and GeoVisual Search will find thousands of other matching objects.

Though GeoVisual Search is just a demo, not a product, we wanted to release it because it will hopefully inspire you to think about what’s possible when computers can recognize patterns in satellite imagery. Plus, it’s really fun to play with.

Go ahead, try it. We’ll be here when you come back to tell you more.

Baseball diamond, wind turbine, and a house with a moat

Finding every haystack on the planet

Visual search attempts to find similar things in a set of images. An example of this is when Facebook automatically identifies who is in your pictures. You don’t look the same in every picture, but there are visually similar elements that make up your likeness. Humans are particularly skilled at recognizing patterns in the world. Thanks to recent improvements in technology, computers are getting better at this, too.

Last year, a team at Carnegie Mellon University applied the principles of visual search to seven cities around the world in a demo called Terrapattern. We were impressed with their work and wondered: could we do this not just for a few cities, but for the entire globe?

With a lot work, ingenuity, and pixels, we were able to build GeoVisual Search for three maps using three different imagery sources:

  • The continental United States — This uses aerial imagery at 1m per pixel from the National Agriculture Imagery Program (NAIP) in the US. Because the imagery is high-resolution, you’ll be able to find solar farms and orchards.
  • China — This map uses satellite imagery at 4m resolution from our friends at Planet. Though the resolution isn’t as high as the NAIP map over the US, Planet’s satellites will soon be providing daily pictures of the globe. In this map, you’ll be able to find solar farms and stadiums.
  • The entire world — This uses a composite that Descartes Labs released a few weeks ago uses Landsat 8 and is at 15m resolution. Though much coarser than our other maps, you’ll still be able to find lots of interesting larger scale phenomena, like pivot irrigation and suburbs.

In building GeoVisual Search, the first problem to tackle is training the computer to figure out what “similarity” means. Since the human brain is really good at this, that’s a good place to start. We can simulate a simple human brain in the computer using what we understand about the physiology of the brain and the structure of neurons. We build a “neural network” in the computer and teach it hundreds of “features” like shadows, colors, and edges (this is commonly called deep learning). We then use those features to look at the entire earth and look for other images that share similar features. The more the images have in common, the higher they go on the list. Effectively, we’ve taught the computer how to recognize patterns. Sometimes the results are surprising (and sometimes they’re just plain wrong), but they’re almost always interesting.

The second problem is not nearly as sexy — there’s no simulation of neurons involved — but in many ways much harder. The challenge is that the entire Earth is many times larger than even the largest city: 150,000,000 square kilometers of landmass in total. And, the search needs to be done in “real time” — we need to be able to scan the entire globe as quickly as possible to return you results. This requires a sophisticated infrastructure.

When you click on something in GeoVisual Search, notice how it only takes a few seconds? In this time, we’ve searched through all of the potential matching images, filtered them down to only the best, and returned the results back to you. Broad area searches like this would simply be impossible with humans and would take too many people too much time. This kind of performance is only possible with a lot of computers and a considerable amount of ingenuity.

What’s next?

That’s a high-level overview of how GeoVisual Search works, meant to give you a flavor of how challenging this problem is. If you want to geek out on the details, we’ve written a technical blog post on our new Descartes Tech Blog.

GeoVisual Search is a demo, not quite a product yet. But, it’s an important step toward better understanding our planet through satellite imagery. Ultimately, we want to be able to search the globe for any object you define and track how that object changes over time.

For example, imagine that you were trying to better understand renewable energy to make a policy recommendation to the United Nations. In addition to other data sources you could gather, you might want to pinpoint the location of every wind turbine and solar panel on the planet and watch how the number has grown and at what rate over the last ten years. The underlying infrastructure we’ve built for GeoVisual Search lays the foundation for exactly this type of technical analysis.

We’ve got a lot of projects in the works, and more exciting announcements coming soon, so follow Descartes Labs and keep track of how we are changing the way we look at our world. (Twitter, Facebook, Medium, LinkedIn).

But for now, just have fun exploring the Earth, the US, or China with GeoVisual Search.

(Oh, and if you’re a developer and looking for a new challenge, drop us a line: we’re hiring!)

--

--

--

Explore posts from the Descartes Labs team

Recommended from Medium

Design Thinking for Artificial Intelligence Projects

What is data science? How Data Scientists Work?

Textual Analysis

Product Analytics for Subscription Businesses

Big Data Analytics Solutions and Services for the Enterprises Resources

COVID-19 Daily Analysis

Cricket analysis: The art of chasing in ODIs, ft. The World Champions

Predict Red Wine Quality with SVC, Decision Tree and Random Forest

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Descartes Labs

Descartes Labs

Descartes Labs is building a data refinery for geospatial data.

More from Medium

Protests spot report 01–07 December 2021

Flood influence analysis using satellite imagery

AI-based Super resolution and change detection to enforce Sentinel-2 systematic usage

Animate urban expansion with Google Earth Engine