NASA's ESDIS Standards Coordination Office (ESCO) coordinates standards policy and activities within NASA's Earth Science Data and Information System (ESDIS) Project. These standards and practices are approved by NASA's Earth Science Division (ESD).
ESCO follows a Standards Process for identification of appropriate standards and practices and their subsequent adoption for use in NASA Earth science data systems. Information about ESDIS mission requirements is also available.
ESD-approved Standards and Practices
The URLs provided within the tables below were correct at the time the document(s) were published.
Data Formats
| Name | Description |
|---|---|
| Data Product Development Guide for Data Producers | The Data Product Development Guide (DPDG) draws upon existing resources such as standards, conventions, best practices, and data format manuals to assemble a step-by-step process for data producers to design and develop Earth Science data products. The DPDG is intended for those who develop Earth Science data products and are collectively referred to as “data producers.” (Updated in August 2024) |
| GeoTIFF | GeoTIFF is based on the TIFF format and is used as an interchange format for georeferenced raster imagery. The GeoTIFF file format is in widespread use worldwide. There is strong software support in both the open source and commercial GIS and spatial data analysis software products. (Approved in September 2019) |
| Dataset Interoperability Recommendations | Dataset Interoperability Recommendations for Earth science embody best practices to reduce and bridge gaps between geoscience dataset formats widely used at NASA and elsewhere. The Dataset Interoperability Working Group (DIWG) generated over 20 recommendations, including maximizing HDF5/netCDF4 interoperability via API accessibility, CF attributes, packing conventions, “balanced” chunking for 3-D datasets, time coordinates in swath structured data, georeference information with geospatial coordinates, and adopting dataset release identifiers. (Approved in July 2016 for part 1; Approved in April 2019 for part 2) |
| ASCII File Format Guidelines | The American Standard Code for Information Interchange (ASCII) File Format Guidelines for Earth Science Data recommend practices for formatting and describing ASCII encoded data files, such that the files will be self-describing and adhere to common conventions. These guidelines address a number of common-sense practices that can make data files encoded in ASCII more widely useful. (Approved in May 2016) |
| OGC KML | Open Geospatial Consortium (OCG) KML was adopted as a NASA ESDS community standard with which to code and share visual geographic content in web-based outline maps and 3D geospatial browsers, such as Google Earth. KML makes Earth science data easily accessible to a wide range of end-users; used in openly available data streams and tools which are easy to use and allow overlaying of information (e.g. Google EarthTM). (Approved in March 2014) |
| NetCDF-4 | The netCDF-4/HDF5 file format enables the expansion of the netCDF model, libraries, and machine-independent data format for geoscience data. NetCDF-4 is straightforward to use relative to HDF5, has a lower learning curve, and has many tools available to enhance ease of use. (Approved in March 2011) |
| NetCDF Classic | NetCDF Classic provides a data model, software libraries, and machine-independent data format for geoscience data. NetCDF classic fostered data interoperability and exchange through its self-describing file format, platform independent architecture, and robust access methods. (Approved in September 2008) |
| HDF5 | HDF5 is a data model, file format and I/O library designed for storing, exchanging, managing and archiving complex data including scientific, engineering, and remote sensing data. HDF5 is widely used for NASA Earth science data in that data users read only the data that they need, not the whole file, while data producers can put images, tables, multidimensional arrays, etc. into the same file. (Approved in January 2007) |
| HDF-EOS5 | HDF-EOS5 is a software library designed to support NASA Earth Observing System (EOS) science data. HDF-EOS5 is built on the more feature-rich HDF5 format, and includes software libraries and application programming interfaces (APIs) widely used in NASA Earth Science Data Systems. (Approved in January 2007) |
| Zarr Storage Specification Version 2 | Zarr is a specification for storage of and access to multi-dimensional array data. Its development targeted data in cloud environments (i.e., in object store), and the specification is optimized for cloud access via capabilities such as metadata consolidation and ability to chunk across any dimension. |
| Cloud Optimized GeoTiff | The Cloud Optimized GeoTIFF (COG) file format is an extension of the mature, well-established, and heavily used GeoTIFF file format, accepted as a NASA standard in 2019. |
| RINEX | RINEX is a data interchange format for raw satellite navigation system data and is the current standard format for GNSS data received, archived, and distributed by NASA’s Crustal Dynamics Data Information System (CDDIS). |
Metadata
| Name | Description |
|---|---|
| Data Product Development Guide for Data Producers | The Data Product Development Guide (DPDG) draws upon existing resources such as standards, conventions, best practices, and data format manuals to assemble a step-by-step process for data producers to design and develop Earth Science data products. The DPDG is intended for those who develop Earth Science data products and are collectively referred to as “data producers.” (Updated in August 2024) |
| ISO 19115 | NASA's Earth Science Division (ESD) International Organization for Standardization (ISO) 19115 Geographic Metadata Standard - Implementation Requirement and Guidance document describes the ESD-approved implementation guidelines for required metadata to be included in science data products. (See also: ESDIS Mission Requirements) |
| UMM | NASA's Unified Metadata Model (UMM) is an extensible metadata model which provides a cross-walk for mapping between the Common Metadata Repository (CMR) supported metadata standards. UMM describes metadata related to key EOSDIS data components using UMM metadata “profiles” for collections, granules, services, variables, visualizations, and elements common to multiple UMM component models. |
| GCMD Keywords | Global Change Master Directory (GCMD) Keywords are a hierarchical set of controlled Earth Science vocabularies that help ensure Earth science data, services, and variables are described in a consistent and comprehensive manner and allow for the precise searching of metadata and subsequent retrieval of data, services, and variables. GCMD Keywords are peer-reviewed and are updated based on community needs. |
| NetCDF CF Metadata Conventions | The NetCDF Climate and Forecast (CF) Metadata Conventions are intended to promote interoperability among data providers, data users, and data services by providing a clear and unambiguous standard for representing geolocations and times of Earth science data, physical quantities that the data represent, and other ancillary information useful in interpreting the data or comparing it with data from other sources. CF is very practical, functional, and its attributes are both machine and human readable. (Approved in April 2010) |
| GCMD DIF | The GCMD Directory Interchange Format (DIF) content metadata provides a specific set of attributes for describing Earth science data at the collection level. The DIF has evolved over its 30+ year existence to enhance interoperability with the Federal Geographic Data Committee (FGDC) Content Standard for Digital Geospatial Metadata (CSDGM) and the International Organisation for Standardisation (ISO) metadata standard 19115. (Approved in September 2008) |
| Data Processing Levels | Data archived at NASA’s Earth Science Data Systems (ESDS) data repositories are processed at various levels ranging from Level 0 to Level 4. Level 0 products are raw data at full instrument resolution. At higher levels, the data are converted into more useful parameters and formats. |
| Data Maturity Levels | The Data Maturity Levels provide guidance on a product’s suitability for use in scientific research and applications and provide a user with a degree of confidence in the accuracy of data as applied to their scientific research. |
Discovery and Access
| Name | Description |
|---|---|
| PDR Mechanism | The Polling with Delivery Record (PDR) mechanism is used for electronic transfer of data and related information. PDR is a protocol used extensively across NASA data systems. (Approved in March 2017) |
| CEOS OpenSearch | Committee on Earth Observation Satellites (CEOS) OpenSearch is a collection of simple specifications for a simple search syntax and the sharing of Earth science data search results. CEOS OpenSearch is generally implemented with very simple search semantics, can work on a wide variety of data, and is widely implemented in the U.S. and by many international agencies. (Approved in January 2016; Updated in June 2017) |
| Data Access Protocol 2.0 | Data Access Protocol (DAP) 2.0 is a data transmission protocol designed specifically for science data. DAP can be installed on a variety of platforms and can be used on a variety of datasets. (Approved in November 2005; Updated in August 2011) |
| OGC WMS v1.1.1 | The Open Geospatial Consortium (OGC) Web Map Service (WMS) v1.1.1 uses HTTP and defines several operations that allow a client to discover the functions a server is capable of providing, request a specific map, and optionally request information about individual features shown on a map. WMS 1.1.1 has been deployed in a wide variety of settings, including many U.S. Government agencies, foreign government agencies, universities and research organizations, and commercial organizations where it can be installed on a variety of platforms and used on a variety of datasets. (Approved in June 2007) |
| SpatioTemporal Asset Catalogs (STAC) | The STAC specification is a common language to describe geospatial information, so it can more easily be leveraged, indexed, and discovered. It is a standardized way to expose collections of spatiotemporal data and provides a common structure for describing and cataloging spatiotemporal assets. (Approved in June 2023) |
| Cloud Notification Mechanism (CNM) | The Cloud Notification Mechanism (CNM) is standardized methodology for the notification of transfer of data between a cloud-based or an on-premises Earth Observing System Data and Information System (EOSDIS) platform and the Earthdata Cloud. (Approved September 2023) |
| Science Data Transfer Protocol (SDTP) | The Science Data Transfer Protocol (SDTP) is a secure mechanism for transfer of science data using Hypertext Transfer Protocol Secure (HTTPS) between a data provider and a data subscriber. (Approved September 2023) |
Data Quality
| Name | Description |
|---|---|
| Reuse Readiness Assessment of DQ Software Products | The Reuse Readiness Assessment of Data Quality Software Products document offers insight into some of the implementation issues that should be considered when planning to adopt software products, as well as a guide for software developers to produce reusable software. Products such as metadata editors, compliance and converters, data filters and analyzers, help desk software, user forum and quality search services were evaluated for improving data quality. (Approved in September 2019) |
| Comprehensive Data Quality Recommendations | The Comprehensive Recommendations for Data Producers and Distributors document summarizes the approach and outcomes of the Data Quality Working Group (DQWG) where 16 use cases were analyzed and associated with phases of the data quality information management lifecycle for focusing on 4 primary areas of data quality. The DQWG generated 93 recommendations, grouped into 7 categories, as well as proposed solutions and implementation strategies. (Approved in September 2019) |
| High Priority Data Quality Recommendations | The High Priority Data Quality Recommendations for Data Producers and Distributors highlights a subset of high-priority recommendations for ESDIS to plan and coordinate concrete actions to be taken by data producers and distributors. The DQWG produced 8 Prioritized Recommended Implementation Actions (PRIAs) that support quality of information, metadata creation and validation, user support, guidance, instruction and dissemination, and facilitates DAAC-PI communication. (Approved in April 2019) |
| OSDMP Guidance | Open Science and Data Management Plan (OSDMP) Templates for both data producers and DAACs were updated in early 2019 to include recommendations by the DQWG on how to better describe the quality of the subject data products. These templates will help producers in assembling OSDMPs for projects in a consistent and useful format, while also assisting DAACs on developing the contents of their plans, which may include describing their data operations. (Approved in February 2019) |
| Data Preservation Content Specification | This document defines the contents of data, metadata and associated documentation to be preserved beyond the life of missions funded by NASA’s Earth Science Division (ESD). The purpose of the document is to identify all the content items that need to be preserved to ensure their availability to support future investigations in long-term global change research. (See also: ESDIS Mission Requirements) |
Data Usability
| Name | Description |
|---|---|
| Data Product Development Guide for Data Producers | The Data Product Development Guide (DPDG) draws upon existing resources such as standards, conventions, best practices, and data format manuals to assemble a step-by-step process for data producers to design and develop Earth Science data products. The DPDG is intended for those who develop Earth Science data products and are collectively referred to as “data producers.” (Updated in August 2024) |
| Dataset Interoperability Recommendations | Dataset Interoperability Recommendations for Earth science embody best practices to reduce and bridge gaps between geoscience dataset formats widely used at NASA and elsewhere. The Dataset Interoperability Working Group (DIWG) generated over 20 recommendations, including maximizing HDF5/netCDF4 interoperability via API accessibility, CF attributes, packing conventions, “balanced” chunking for 3-D datasets, time coordinates in swath structured data, georeference information with geospatial coordinates, and adopting dataset release identifiers. (Approved in July 2016 for part 1; Approved in April 2019 for part 2) |
Emerging Standards
| Name | Description |
|---|---|
| COG (Cloud Optimized GeoTIFF) | The COG data format builds on the established GeoTIFF format that has been accepted as an ESCO standard. |
| STAC (SpatialTemporal Asset Catalogs) | The STAC specification is a common language to describe geospatial information, so it can more easily be leveraged, indexed, and discovered. |
| Zarr | Zarr is a cloud-based data format specifically for improving access to N-Dimensional arrays. It is an effective way to store large N-dimensional data in the cloud and access the data in predefined chunks. |
Deprecated Standards and Practices
| Name | Description |
|---|---|
| HDF 4 | HDF 4 is being deprecated in favor of HDF 5 and is therefore not recommended for new missions. Although support will still be offered for HDF 4, it will be primarily of a sustainment nature. |
| HDF-EOS 2 | HDF EOS 2 is being deprecated in favor of HDF EOS 5 and is therefore not recommended for new missions. Although support will still be offered for HDF EOS 2, it will be primarily of a sustainment nature. |
| ECHO Metadata Standard | The original metadata model for the EOSDIS Core System (ECS), NASA's Earth Observing System ClearingHOuse (ECHO) metadata standard is being deprecated in favor of NASA's Unified Metadata Model (UMM), and is therefore not recommended for new missions. Although support will still be offered for ECHO, it will be primarily of a sustainment nature. |
References and Related Technical Information
References
- NASA Earth system science instruments: An overview and listing of remote sensing instruments (active and passive)
- Processing levels: EOSDIS data products are processed at various levels ranging from Level 0 to Level 4. Read through the definitions of each level
- Earth science disciplines: A list of the various Earth science disciplines served by NASA's DAACs with various data products, data information services, and tools unique to a particular user community
- EOSDIS acronym list: A comprehensive listing of acronyms used within EOSDIS
- EOSDIS Glossary: A glossary of phrases with their definition, context, and usage within EOSDIS
Related Technical Information
- Search Relevance Recommendations for Earth Science
- Guidelines for a Common File Format for Aura Instruments
- Creating File Format Guidelines: The Aura Experience
Document Lists
- List of ESCO documents
- Documents that are superseded or obsolete are moved to the Retired Documents list