Data Product Development Guide for Data Producers

Summary

This Data Product Development Guide (DPDG) for Data Producers was produced for the Earth Observing System Data and Information System (EOSDIS) by the DPDG Working Group, one of the Earth Science Data System Working Groups (ESDSWGs) to aid in the development of NASA Earth Science data products.

The DPDG is intended for those who develop Earth Science data products and are collectively referred to as “data producers.” This guide is primarily intended for developers of Earth Science data products derived from remote sensing data, and particularly for the development of Level 1B through Level 4 products (see Data Processing Levels). However, developers of related data products including Level 0 and 1A satellite data, airborne and ground-based data products, and modeling data will also find useful guidance.

Status

The Data Product Development Guide (DPDG) for Data Producers was approved as a Suggested Practice for NASA Earth Science Data Systems in July 2020.

Suggested Practice Document

Data Product Development Guide (DPDG) for Data Producers

NASA Earth Science Community Recommendations for Use

Strengths

The Data Product Development Guide (DPDG) draws upon existing resources such as standards, conventions, best practices, and data format manuals to assemble a step-by-step process for data producers to design and develop Earth Science data products. From requirements and design, to data formats, metadata, and compression, to tools, Digital Object Identifiers (DOIs) and data publication, this document provides a comprehensive set of guidelines and recommendations for data producers working with designated Distributed Active Archive Centers (DAACs) to generate and deliver data products that best serve end user communities. Inputs were collected from several members of the NASA Earth Science community, especially those participating in the Earth Science Data Systems Working Groups (ESDSWGs), as well as DAACs, and experienced data producers, in the making of this document.

Weaknesses

This guide is a broad snapshot bringing together best practices, standards, data formats, and metadata utilized today in the design and development of Earth Science data products. With the evolution of these protocols in the community, especially with cloud computing, big data, and analysis-ready data, the challenge will be to stay abreast of these conventions, documenting and recommending them for use by the Earth Science community, and updating this guide as warranted.

Cloud computing is not specifically addressed, however this document is intended to be updated with some regularity so that this can be considered in future versions of this document.

Applicability

This guide is primarily intended for developers of Earth Science data products derived from remote sensing data, and particularly for the development of Level 1B through Level 4 products. However, developers of related data products including Level 0 and 1A satellite data, airborne and ground-based data products will also find useful guidance.

Much of the document is based on the experience data producers have gained via the use of netCDF as a file format and use of CF Metadata Conventions for metadata. There are, however, strong lessons to be learned from this that do apply to the entire data product development process.

Many aspects of the data development process, including requirements, design, selecting the appropriate data format(s), metadata, data compression, data quality, and DOIs, can certainly be utilized for developing data products derived from non-remote sensing sources.

Limitations

This guide provides a thorough process for designing and developing Earth Science data products that require essentials such as organizational commitment, staff resources, and computing power that some data producers may not be able to launch and maintain. It may be advantageous for data producers to establish a cohesive relationship with the DAACs to leverage some of these essentials in order to create and sustain a process for producing such products.

Page Last Updated: Aug 4, 2020 at 11:19 AM EDT