Earth observation and the value of open, accessible data
By Simon Ilyushchenko, Software Engineer, Google Earth Engine
For the last several years, I have been working on the Google Earth Engine team. Earth Engine is a cloud platform for petapixel-scale analysis of geospatial data. (If this sounds like buzzword soup, the translation is “cloud magic for processing Earth observation data extremely fast.”) My responsibility is downloading and ingesting a huge amount of data (200+ datasets, 20 TB/day) so that it can be instantly accessed in Earth Engine. Thanks to the many providers of accessible open data, we were able to build a remote sensing data library unrivaled by anything else this side of Jupiter (or at least anything publicly available).
Why is Google doing this? We believe that open and accessible data is good for everyone.
Over the last few years, we’ve used publicly available imagery to develop an incredibly accurate depiction of Earth and its changes over time, which has benefited Google Earth and Maps. Most recently, we used Landsat and Sentinel imagery data to create Earth Timelapse, a global, zoomable video that lets you see how the Earth has changed over the past 32 years. Scientists using Earth Engine also rely on these datasets for pioneering new approaches in flood risk mapping, agriculture, infectious disease prevention, Arctic mapping, forest monitoring, land use change, and more. In September 2016, journalists at the Center for Investigative Reporting even used Landsat 8 imagery and Earth Engine for their Wet Prince of Bel Air investigation into residential water usage during the ongoing California drought. A few months earlier, journalists at the Berliner Morgenpost compared the area of green spaces in various German cities.
How did we do all of this?
In many cases we rely on data from Landsat and Sentinel 2, two of the most comprehensive collections of public, cost-free satellite imagery.
Landsat, a joint program of the USGS and NASA, offers an unparalleled 40+ year record of our changing planet. The Landsat satellites have observed the Earth continuously from 1972 through the present day. The entire Earth’s surface is imaged about once every two weeks. Sentinel 2 is part of the European Commission and European Space Agency’s ambitious Copernicus Earth observation program. It has already generated over half a petabyte of imagery since its launch in 2015, with much more to come.
Since 2010, we have ordered, downloaded, and added 5 million Landsat scenes to the Earth Engine catalog. We could not have done this without the cooperation of teams across the U.S. Geological Survey’s Earth Resources Observation and Science (EROS) Center, the federal organization which maintains the Landsat long-term archive. We are immensely grateful to the engineering, policy, and science staff at USGS EROS and at the European Union’s Copernicus program, whose assistance has made the Earth Engine and Cloud public data catalogs possible.
To make Landsat and Sentinel 2 datasets even more accessible, we partnered with the Google Public Datasets program to set up a public mirror of these two datasets in Google Cloud. In addition, Google Cloud Storage also provides a variety of other public datasets that anyone can integrate into their applications and use as inputs for analytical processing (this includes machine learning pipelines like TensorFlow). These datasets can be accessed with Google’s Cloud Platform Console or Cloud Storage API. Earth Engine and Cloud Storage provide several petabytes of the world’s public satellite imagery data in our catalogs so you can spend more of your time using the data to build great products and services for your customers and users.
If you would like to suggest more public datasets to be mirrored in Google Cloud, please contact gcp-public-data@google.com.
