Why Radiant.Earth Invests in Data Collaborative Innovation
By Radiant.Earth Editorial Team
Data collaborative innovation — that is, a group of actors from different data domains working together toward solutions — might hold the key to finding solutions for some of the global challenges that the world faces.
In October 2017, Radiant.Earth’s Technical Fellow Chris Holmes rallied developers working with imagery and software to develop modern geospatial specifications and standards that facilitate the sharing and searching of imagery natively on the cloud. The effort began with a conference and hackathon in Boulder, CO, known as the Boulder Sprint, attended by 25 people representing 14 different organizations in both the for-profit and non-profit sectors. Their enthusiasm and willingness suggest that the time is finally right for open source industry-wide collaboration on this essential technology.
The SpatioTemporal Asset Catalog (STAC) is an outcome of the Boulder Sprint. Available via GitHub, the STAC specification presents a normalized format for all remotely-sensed imagery, whether from satellites, drones, airplanes, or even balloons. The STAC specification provides a way to make Earth observation raster data more accessible, especially if a significant number of imagery providers adapt their data to this specification.
Moreover, we have invested in support of the development of Cloud Optimized GeoTIFFs (COGs), which rely on two complementary pieces of technology. The first is the ability of GeoTIFFs to not just store an image’s raw pixels, but also to organize them in particular ways. The second is HTTP GET range requests that let client software ask for just the portions of a file that they need.
The first piece of COG technology organizes the GeoTIFF so that the second COG technology can easily select the part of the file that requires processing. COGs enable efficient streaming of data, embedded with some smart technology to allow fully cloud-native geospatial workflows. Online imagery platforms, such as Planet, DigitalGlobe’s GBDX, and Radiant.Earth, have begun to use these two parts of COG technology toprovide imagery that gets processed on the fly. COG-aware software can stream just the portions of data that it needs, improving access times.
Pretty cool!
Why is this important?
One word: Speed.
To address real-world problems, data and resulting insights need to flow to developers, analysts, and decision-makers in near-real-time, especially in the event of disasters and humanitarian relief efforts. Therefore, the speed of data and insights streaming to market is critical.
Developers have been clamoring for this type of technology because it is a challenge to work with large amounts of imagery data scattered around the world, stored in different archives, file structures, and formats. Now, developers can essentially “stream” the data on demand. COGs make working with massive amounts of data easier. As an outcome, analysis becomes more efficient, and it reduces duplication of data.
Not surprisingly, organizations are quickly adopting COGs and the STAC specification.
Digital Globe, for instance, followed up on the work done at the Boulder Sprint by converting their IKONOS imagery catalog and NOAA’s Suomi NPP VIIRS Night Lights data to Cloud Optimized GeoTiffs. Other organizations, such as OpenAerialMap, Planet, and Radiant.Earth are also in the process of building a migration path to using Cloud Optimized GeoTIFFs and SpatioTemporal Asset Catalogs. As Sean Gorman, DigitalGlobe’s Platform Director, states in his blog detailing how they have gone about it, “[DigitalGlobe is] super excited about the potential of Cloud Native Geospatial. The concepts behind COG are truly opening the door to an exciting new chapter for the geospatial industry.”
You can use Radiant.Earth’s COG validator here, or host your own.
COGs open up many possibilities for data scientists, particularly for analysts and those interested in AI and machine learning, to explore new methodologies, models, and data cube integration. That said, new solutions to global development challenges are the real opportunities for Radiant.Earth and its partners to tackle next.
The need for neutrality
To build STAC and COG, we had to tap into the knowledge base of experts — across industry and domains. Much the same way as the Linux Foundation provides a framework for the development of core Linux and supporting technologies, the geospatial industry requires a collaborative environment that is focused on solutions for some of the global challenges that the world faces, that adopts modern open specifications and technology, and that does not favor any one vendor, government, or standards body.
This is why Radiant.Earth invests in thought leadership summits, hackathons, and technical fellowships. As a non-profit founded with the goal of harnessing Earth imagery and geospatial data to meet the world’s critical challenges, we pride ourselves on being a neutral resource that leverages ongoing efforts from both the public and private worlds. Our mandate is simple: to enhance industry collaboration, provide support for open source development through our Technical Fellows program, ignite innovation, and expand the global knowledge base in the use of remotely sensed data to positively impact the developing world’s foremost problems.
“COG and STAC have the potential for lowering technology friction and allowing one to search, acquire, and analyze imagery across multiple archives”
What does it mean for the global development community?
The new modern geospatial specifications and standards enable cost-effective discovering of the right Earth observation imagery and data to enhance our understanding of how we humans are impacting our planet. It also provides the technology and strategies to understand and build scalable and reproducible predictive models to real-world issues in the future, such as food security, deforestation, climate change, poaching, and conflict, to name a few.
The difficulty in finding and accessing the right imagery at the right time is perhaps the leading geospatial barrier facing the global development community. COG and STAC have the potential for lowering technology friction and allowing one to search, acquire, and analyze imagery across multiple archives, as well as to significantly enhance program outcomes and reach, lower barriers to entry, and reduce costs.
