ESDS Program

Data Product Development Guide for Data Producers

Summary

This Data Product Development Guide (DPDG) for Data Producers was produced for the Earth Observing System Data and Information System (EOSDIS) by the DPDG Working Group, one of the Earth Science Data System Working Groups (ESDSWGs) to aid in the development of NASA Earth Science data products.

The DPDG is intended for those who develop Earth Science data products and are collectively referred to as “data producers.” This guide is primarily intended for developers of Earth Science data products derived from remote sensing data, and particularly for the development of Level 1B through Level 4 products (see Data Processing Levels). However, developers of related data products including Level 0 and 1A satellite data, airborne and ground-based data products, and modeling data will also find useful guidance.

Status

The Data Product Development Guide (DPDG) for Data Producers was approved as a Suggested Practice for NASA Earth Science Data Systems in July 2020. A minor update (Version 1.1) to the original version was published in October 2021.

The Quick Start Guide to the EOSDIS Data Product Development Guide for Data Producers was published by the ESO as a companion document in June 2021 and updated in April 2022.

Suggested Practice Document

Data Product Development Guide (DPDG) for Data Producers V1.1

Previous Version(s) Data Product Development Guide (DPDG) for Data Producers V1.0
DOI https://doi.org/10.5067/DOC/ESO/RFC-041VERSION1
Suggested Citation Ramapriyan, H. K., and P. J. T. Leonard. 2020. Data Product Development Guide (DPDG) for Data Producers version1. NASA Earth Science Data and Information System Standards Office, 21 October 2021. https://doi.org/10.5067/DOC/ESO/RFC-041VERSION1

Quick Start Guide

A companion to the complete Data Product Development Guide for Data
Producers, which, as a condensed version, serves as a quick start guide
for novice data producers before delving into the details of the full
document.


Quick Start Guide to the EOSDIS Data Product Development Guide for Data Producers V1.1

Previous Version(s) Quick Start Guide to the EOSDIS Data Product Development Guide for Data Producers V1.0
DOI https://doi.org/10.5067/DOC/ESO/DPDG_QSG_VERSION1
Suggested Citation Ramapriyan, H. K., and P. J. T. Leonard. 2021. Quick Start Guide to the EOSDIS Data Product Development Guide for Data Producers version 1.1. NASA Earth Science Data and Information System Standards Office, Publication Date 2022-04-08.https://doi.org/10.5067/DOC/ESO/DPDG_QSG_VERSION1

NASA Earth Science Community Recommendations for Use

Strengths

The Data Product Development Guide (DPDG) draws upon existing resources such as standards, conventions, best practices, and data format manuals to assemble a step-by-step process for data producers to design and develop Earth Science data products. From requirements and design, to data formats, metadata, and compression, to tools, Digital Object Identifiers (DOIs) and data publication, this document provides a comprehensive set of guidelines and recommendations for data producers working with designated Distributed Active Archive Centers (DAACs) to generate and deliver data products that best serve end user communities. Inputs were collected from several members of the NASA Earth Science community, especially those participating in the Earth Science Data Systems Working Groups (ESDSWGs), as well as DAACs, and experienced data producers, in the making of this document.

Weaknesses

This guide is a broad snapshot bringing together best practices, standards, data formats, and metadata utilized today in the design and development of Earth Science data products. With the evolution of these protocols in the community, especially with cloud computing, big data, and analysis-ready data, the challenge will be to stay abreast of these conventions, documenting and recommending them for use by the Earth Science community, and updating this guide as warranted.

Cloud computing is not specifically addressed, however this document is intended to be updated with some regularity so that this can be considered in future versions of this document.

Applicability

This guide is primarily intended for developers of Earth Science data products derived from remote sensing data, and particularly for the development of Level 1B through Level 4 products. However, developers of related data products including Level 0 and 1A satellite data, airborne and ground-based data products will also find useful guidance.

Much of the document is based on the experience data producers have gained via the use of netCDF as a file format and use of CF Metadata Conventions for metadata. There are, however, strong lessons to be learned from this that do apply to the entire data product development process.

Many aspects of the data development process, including requirements, design, selecting the appropriate data format(s), metadata, data compression, data quality, and DOIs, can certainly be utilized for developing data products derived from non-remote sensing sources.

Limitations

This guide provides a thorough process for designing and developing Earth Science data products that require essentials such as organizational commitment, staff resources, and computing power that some data producers may not be able to launch and maintain. It may be advantageous for data producers to establish a cohesive relationship with the DAACs to leverage some of these essentials in order to create and sustain a process for producing such products.

Last Updated