OpenNEX Climate Data Access Tool
I’m excited to share that today we released our OpenNEX Climate Data Access Tool to help improve access to NASA Earth Exchange (NEX) downscaled climate projections.
Climate information is important not only to highly technical scientists, researchers, and numerical modellers, but also to non-technical local community members and decision makers who are responsible for developing response plans. These downscaled and bias-corrected climate data products provide a better representation of regional climate patterns and are used to determine specific impacts to numerous real-world applications such as crop productivity, flood risk, energy demand, and human health.
However due to their immense size, archive structure and file format, acquiring and incorporating downscaled climate projection data into regional analyses can require significant technical knowledge. The OpenNEX Climate Data Access Tool was designed to remove these technical barriers and allow quick, easy access to these valuable climate data by both scientists and policy planners.
“As climate change is increasingly affecting our everyday life, we are seeing communities requesting easier ways to ingest climate data, both historical and projections, to their work,” says Dr. Sangram Ganguly, Senior Research Scientist at Bay Area Environmental Research Institute and NASA Ames Research Center.
“We are excited to collaborate with Planet OS to help communities with the complexities of analyzing space-time bound climate projections datasets. Planet OS’s approach provides a scalable easy-to-use data discovery and integration framework for these big datasets without having to spend days to obtain the data and perform analytics on top of it.”
The OpenNEX Project
In late 2013 NASA Earth Exchange, in partnership with Amazon Web Services (AWS), published a number of earth science datasets within the AWS S3 infrastructure. The goal of the OpenNEX project was to make the data accessible to a wider audience of full-time researchers, students, and citizen scientists.
Among the datasets published to S3 were two downscaled climate projections: NASA Earth Exchange Downscaled Climate Projections (NEX-DCP30) and NASA Earth Exchange Global Daily Downscaled Projections (NEX-GDDP). These datasets provide a set of bias-corrected climate change projections and include maximum temperature, minimum temperature, and precipitation for the periods from 1950 through 2005 (Retrospective Run) and from 2006 to 2099 (Prospective Run).
To complement the data, a number of resources were created to assist users in accessing OpenNEX data, including Amazon Machine Images, tutorials, and software development kits. These tools are quite useful for tech savvy users, but can be confusing for those with limited technical knowledge.
Additionally, the binary NetCDF format in which the data is provided can also pose technical challenges that make working with the data difficult. This is particularly true when users wish to work with a limited temporal or spatial extent of the data. The work needed to accomplish this is not trivial and grossly inefficient given the archive file structure, which requires users to download the full spatial extent of the data regardless of how much is actually being used in the analysis.
The OpenNEX Climate Data Access Tool
The OpenNEX Climate Data Access Tool was produced by Planet OS in collaboration with the NASA Earth Exchange team with the intent of improving access to data for climate assessment. This effort builds upon the work of the OpenNEX project by providing intuitive, web-based access interfaces to NEX-DCP30 and NEX-GDDP data.
Using the OpenNEX Climate Data Access Tool researchers can build custom subsets of the climate projection data and access the selected data in either CSV or NetCDF format. The tool requires no programming knowledge and produces files that can be used for climate science work performed in Python, R, Matlab, and even Excel.
Some of the key features of the tool include:
- Create Custom Datasets — Create a custom dataset by selecting the time, region, parameter, climate model and scenario that matches your needs. Spatial and temporal slicing enables you to download data only within your area and time of interest.
- Deploy via Docker — Run the provided bash script to deploy a Docker container that will acquire and serve the data you selected. This container can be run on a local machine or deployed to a remote instance.
- CSV or NetCDF Format — Download your data in NetCDF or CSV format. The availability of a CSV option means data can be loaded directly to applications without NetCDF support, such as Excel.
- Reproducible — Deploy your container to an EC2 instance and expose the access endpoint to allow others direct access to your custom dataset. You can also share the unique dataset permalink with colleagues, who can launch their own containers to replicate your dataset.
Examples & Additional Resources
We’ve created a number of code samples and Jupyter notebooks to highlight some basic use cases that utilize the OpenNEX Climate Data Access Tool.
Our Senior Technology Advisor Tom Faulhaber used NEX-DCP30 precipitation data to investigate how San Francisco’s water supply could be affected by climate change.
NEX-DCP30 data is also used to explore maximum temperatures in the Chicago area according to a single climate model (CESM1-CAM5) under various RCP scenarios. The analysis above is done in R, however a corresponding Jupyter notebook that uses Python and Pandas is available on GitHub.
For more details including sample code and information about the available data, check out the documentation. Or just jump in and get started with the OpenNEX Climate Data Access Tool. You can also join us for a webinar on September 21 at 9:00 AM US Pacific Time.
If you have suggestions on how we might improve the tool or would like to share a use case with the Planet OS community, we’d love to hear from you! Please send your thoughts and code samples to firstname.lastname@example.org.
Originally published at planetos.com on September 1, 2016.