Standards and Practices

ESD-Approved Standards and Practices for use in NASA Earth Science Data Systems

ESO coordinates standards policy and activities within ESDIS. ESO follows a Standards Process for identification of appropriate standards and practices and their subsequent adoption for use in NASA Earth science data systems.

ESDIS Mission Requirements

The ESDIS Mission Requirements have been moved to this page.

Data Formats

Toggle Data Formats
Name Description
Data Product Development Guide for Data Producers
NEW
The Data Product Development Guide (DPDG) draws upon existing resources such as standards, conventions, best practices, and data format manuals to assemble a step-by-step process for data producers to design and develop Earth Science data products. The DPDG is intended for those who develop Earth Science data products and are collectively referred to as “data producers.”
(Approved in July 2020)
GeoTIFF GeoTIFF is based on the TIFF format and is used as an interchange format for georeferenced raster imagery. The GeoTIFF file format is in widespread use worldwide. There is strong software support in both the open source and commercial GIS and spatial data analysis software products.
(Approved in September 2019)
Dataset Interoperability Recommendations Dataset Interoperability Recommendations for Earth science embody best practices to reduce and bridge gaps between geoscience dataset formats widely used at NASA and elsewhere. The Dataset Interoperability Working Group (DIWG) generated over 20 recommendations, including maximizing HDF5/netCDF4 interoperability via API accessibility, CF attributes, packing conventions, “balanced” chunking for 3-D datasets, time coordinates in swath structured data, georeference information with geospatial coordinates, and adopting dataset release identifiers.
(Approved in July 2016 for part 1; Approved in April 2019 for part 2)

ASCII File Format Guidelines The ASCII File Format Guidelines for Earth Science Data recommend practices for formatting and describing ASCII encoded data files, such that the files will be self-describing and adhere to common conventions. These guidelines address a number of common-sense practices that can make data files encoded in ASCII more widely useful.
(Approved in May 2016)

OGC KML OGC KML was adopted as a NASA ESDS community standard with which to code and share visual geographic content in web-based outline maps and 3D geospatial browsers, such as Google Earth. KML makes Earth science data easily accessible to a wide range of end-users; used in openly available data streams and tools which are easy to use and allow overlaying of information (e.g. Google EarthTM).
(Approved in March 2014)

NetCDF-4 The netCDF-4/HDF5 file format enables the expansion of the netCDF model, libraries, and machine-independent data format for geoscience data. NetCDF-4 is straightforward to use relative to HDF5, has a lower learning curve, and has many tools available to enhance ease of use.
(Approved in March 2011)

NetCDF Classic NetCDF Classic provides a data model, software libraries, and machine-independent data format for geoscience data. NetCDF classic fostered data interoperability and exchange through its self-describing file format, platform independent architecture, and robust access methods.
(Approved in September 2008)

HDF 5 HDF 5 is a data model, file format and I/O library designed for storing, exchanging, managing and archiving complex data including scientific, engineering, and remote sensing data. HDF 5 is widely used for NASA Earth science data in that data users read only the data that they need, not the whole file, while data producers can put images, tables, multidimensional arrays, etc. into the same file.
(Approved in January 2007)

HDF-EOS 5 HDF-EOS 5 is a software library designed to support NASA Earth Observing System (EOS) science data. HDF-EOS5 is built on the more feature-rich HDF5 format, and includes software libraries and application programming interfaces (APIs) widely used in NASA Earth Science Data Systems.
(Approved in January 2007)

Metadata

Toggle Metadata
Name Description
Data Product Development Guide for Data Producers
NEW
The Data Product Development Guide (DPDG) draws upon existing resources such as standards, conventions, best practices, and data format manuals to assemble a step-by-step process for data producers to design and develop Earth Science data products. The DPDG is intended for those who develop Earth Science data products and are collectively referred to as “data producers.”
(Approved in July 2020)
ISO 19115 NASA's Earth Science Division (ESD) International Organization for Standardization (ISO) 19115 Geographic Metadata Standard - Implementation Requirement and Guidance document describes the ESD-approved implementation guidelines for required metadata to be included in science data products. (See also: ESDIS Mission Requirements)
UMM NASA's Unified Metadata Model (UMM) is an extensible metadata model which provides a cross-walk for mapping between the Common Metadata Repository (CMR) supported metadata standards. UMM describes metadata related to key EOSDIS data components using UMM metadata “profiles” for collections, granules, services, variables, visualizations, and elements common to multiple UMM component models.
GCMD Keywords Global Change Master Directory (GCMD) Keywords are a hierarchical set of controlled Earth Science vocabularies that help ensure Earth science data, services, and variables are described in a consistent and comprehensive manner and allow for the precise searching of metadata and subsequent retrieval of data, services, and variables. GCMD Keywords are peer-reviewed and are updated based on community needs.
NetCDF CF Metadata Conventions The NetCDF Climate and Forecast (CF) Metadata Conventions are intended to promote interoperability among data providers, data users, and data services by providing a clear and unambiguous standard for representing geolocations and times of Earth science data, physical quantities that the data represent, and other ancillary information useful in interpreting the data or comparing it with data from other sources. CF is very practical, functional, and its attributes are both machine and human readable.
(Approved in April 2010)

GCMD DIF The GCMD Directory Interchange Format (DIF) content metadata provides a specific set of attributes for describing Earth science data at the collection level. The DIF has evolved over its 30+ year existence to enhance interoperability with the Federal Geographic Data Committee (FGDC) Content Standard for Digital Geospatial Metadata (CSDGM) and the International Organisation for Standardisation (ISO) metadata standard 19115.
(Approved in September 2008)

Discovery and Access

Toggle Discovery and Access
Name Description
PDR Mechanism The Polling with Delivery Record (PDR) mechanism is used for electronic transfer of data and related information. PDR is a protocol used extensively across NASA data systems.
(Approved in March 2017)

CEOS OpenSearch Committee on Earth Observation Satellites (CEOS) OpenSearch is a collection of simple specifications for a simple search syntax and the sharing of Earth science data search results. CEOS OpenSearch is generally implemented with very simple search semantics, can work on a wide variety of data, and is widely implemented in the U.S. and by many international agencies.
(Approved in January 2016; Updated in June 2017)

Data Access Protocol 2.0 Data Access Protocol (DAP) 2.0 is a data transmission protocol designed specifically for science data. DAP can be installed on a variety of platforms and can be used on a variety of data sets.
(Approved in November 2005; Updated in August 2011)

OGC WMS v1.1.1 The OGC Web Map Service v1.1.1 uses HTTP and defines several operations that allow a client to discover the functions a server is capable of providing, request a specific map, and optionally request information about individual features shown on a map. WMS 1.1.1 has been deployed in a wide variety of settings, including many U.S. Government agencies, foreign government agencies, universities and research organizations, and commercial organizations where it can be installed on a variety of platforms and used on a variety of data sets.
(Approved in June 2007)

Data Quality

Toggle Data Quality
Name Description
Reuse Readiness Assessment of DQ Software Products
NEW
The Reuse Readiness Assessment of Data Quality Software Products document offers insight into some of the implementation issues that should be considered when planning to adopt software products, as well as a guide for software developers to produce reusable software. Products such as metadata editors, compliance and converters, data filters and analyzers, help desk software, user forum and quality search services were evaluated for improving data quality.
(Approved in September 2019)

Comprehensive Data Quality Recommendations
NEW
The Comprehensive Recommendations for Data Producers and Distributors document summarizes the approach and outcomes of the Data Quality Working Group (DQWG) where 16 use cases were analyzed and associated with phases of the data quality information management lifecycle for focusing on 4 primary areas of data quality. The DQWG generated 93 recommendations, grouped into 7 categories, as well as proposed solutions and implementation strategies.
(Approved in September 2019)

High Priority Data Quality Recommendations The High Priority Data Quality Recommendations for Data Producers and Distributors highlights a subset of high-priority recommendations for ESDIS to plan and coordinate concrete actions to be taken by data producers and distributors. The DQWG produced 8 Prioritized Recommended Implementation Actions (PRIAs) that support quality of information, metadata creation and validation, user support, guidance, instruction and dissemination, and facilitates DAAC-PI communication.
(Approved in April 2019)

DMP Guidance Data Management Plan (DMP) Templates for both data producers and DAACs were updated in early 2019 to include recommendations by the DQWG on how to better describe the quality of the subject data products. These templates will help producers in assembling DMPs for projects in a consistent and useful format, while also assisting DAACs on developing the contents of their plans, which may include describing their data operations.
(Approved in February 2019)
Data Preservation Content Specification This document defines the contents of data, metadata and associated documentation to be preserved beyond the life of missions funded by NASA’s Earth Science Division (ESD). The purpose of the document is to identify all the content items that need to be preserved to ensure their availability to support future investigations in long-term global change research. (See also: ESDIS Mission Requirements)

Data Usability

Toggle Data Quality
Name Description
Data Product Development Guide for Data Producers
NEW
The Data Product Development Guide (DPDG) draws upon existing resources such as standards, conventions, best practices, and data format manuals to assemble a step-by-step process for data producers to design and develop Earth Science data products. The DPDG is intended for those who develop Earth Science data products and are collectively referred to as “data producers.”
(Approved in July 2020)
Dataset Interoperability Recommendations Dataset Interoperability Recommendations for Earth science embody best practices to reduce and bridge gaps between geoscience dataset formats widely used at NASA and elsewhere. The Dataset Interoperability Working Group (DIWG) generated over 20 recommendations, including maximizing HDF5/netCDF4 interoperability via API accessibility, CF attributes, packing conventions, “balanced” chunking for 3-D datasets, time coordinates in swath structured data, georeference information with geospatial coordinates, and adopting dataset release identifiers.
(Approved in July 2016 for part 1; Approved in April 2019 for part 2)

Deprecated Standards and Practices

Toggle Deprecated Standards and Practices
Name Description
HDF 4 HDF 4 is being deprecated in favor of HDF 5 and is therefore not recommended for new missions. Although support will still be offered for HDF 4, it will be primarily of a sustainment nature.
HDF-EOS 2 HDF EOS 2 is being deprecated in favor of HDF EOS 5 and is therefore not recommended for new missions. Although support will still be offered for HDF EOS 2, it will be primarily of a sustainment nature.
ECHO Metadata Standard The original metadata model for the EOSDIS Core System (ECS), NASA's Earth Observing System ClearingHOuse (ECHO) metadata standard is being deprecated in favor of NASA's Unified Metadata Model (UMM), and is therefore not recommended for new missions. Although support will still be offered for ECHO, it will be primarily of a sustainment nature.

References

  • NASA Earth system science remote sensors: An overview and listing of Remote sensing instruments (active and passive).
  • Processing levels: EOSDIS data products are processed at various levels ranging from Level 0 to Level 4. Read through the definitions of each level.
  • Earth science disciplines: A list of the various Earth science disciplines served by NASA's DAACs with various data products, data information services, and tools unique to a particular user community.
  • ESDIS acronym list: A comprehensive listing of acronyms used within EOSDIS.
  • EOSDIS Glossary: A glossary of phrases with their definition, context, and usage within EOSDIS.

Document Lists

Page Last Updated: Aug 4, 2020 at 11:18 AM EDT