Systematic Data Transformation to Enable Web Coverage Services (WCS) and ArcGIS Image Services within ESDIS Cumulus Cloud
Principal Investigator (PI): Jeff Walter, NASA's Langley Research Center
NASA's Earth Science Data and Information System (ESDIS) Project has recently launched the Cumulus prototype which provides a scalable cloud-based platform to ingest, archive, distribute, and manage Earth Science data within the Amazon cloud environment ("Earth Science Data," 2017). One of the goals for the Cumulus project is to increase the flexibility and effectiveness of NASA Earth Science data access and distribution to end users. Earth science data is geospatial in nature. However, according to the Geospatial Data Abstraction Library (GDAL) Enhancements for ESDIS (GEE) Assessment ("GDAL Enhancements," 2016), many Earth science data products (e.g. Measurement of Pollution in the Troposphere[MOPITT], Clouds and the Earth's Radiant Energy System [CERES], to name a few) are difficult to access and use within commercial off-the-shelf (COTS) and open source geographic information systems (GIS) software, such as Esri's ArcGIS and the open source QGIS software.
According to the 2017 American Customer Satisfaction Index (ACSI) survey, ArcGIS is the most used software tool/package at 64%, followed by QGIS (37%), ENVI (32%) and Excel (27%) to work with NASA Earth science data ("ACSI Reports," 2018). We propose to develop geospatial data transformation plugins that could be used within the ESDIS Cumulus environment to serve out transformed MOPITT and CERES data product(s) as OGC Web Coverage Services (WCS) and Esri ArcGIS Image Services. These services will then be easily consumed into COTS GIS software such as ArcGIS and QGIS. These plugins will perform the transformation to fix the data issues (e.g. incorrect image sizes, orientation, multidimensional variable interpretation, georeferenced metadata recognition, etc.) and this extensible framework can be further expanded to add new plugins as additional issues are identified and addressed.
In addition, ArcGIS Image Services and Web Coverage Services provide easy access to actual data values, not just static images, and alleviate the need to navigate complex and storage-intensive processes of raw data downloads. The importance of delivering geospatially enabled web services is emphasized through initiatives like the establishment of NASA's Geospatial Web Services Working Group (GWSWG) (part of the ESDSWG), NASA's Big Earth Data Initiative (BEDI), and the ESDIS Cumulus Seamless 360 Degrees of Services that includes the implementation of a common user-facing API such as WCS. By developing these geospatial data transformation plugins and subsequently providing the corrected data as OGC and Esri web services, it will remove the barrier of transforming each NASA data product after download since it will be correctly served, leading to more easily accessible NASA data products by the Earth Science community.
This proposal will utilize Amazon Web Services (AWS) Step Functions and Lambda Functions as also utilized by the ESDIS Cumulus prototype, though if the ESDIS Cumulus system is not operationalized, our project can be decoupled and operate independently within a commercial cloud environment. We will approach this problem in three phases. Phase one, planning, will focus on the AWS workflow structure. Phase two, development and implementation, will deploy the AWS Step Functions and Lambda Functions used to orchestrate a workflow of customized micro-services executing GDAL transformations in order to geospatially enable and serve a new cloud-optimized MetaRaster Format (MRF) product as an OGC Web Coverage Service and ArcGIS Image Service. Phase three, testing, will allow us to measure the discoverability of NASA Earth Science data web services within major data catalogs (e.g. Earthdata Search, GeoPlatform, Atmospheric Science Data Center [ASDC] ArcGIS Portal, etc.), and their use, speed, accessibility and analytical capabilities within QGIS, ArcGIS, custom web mapping applications, and Data Cubes.
Last Updated: Aug 5, 2019 at 2:49 PM EDT