Printer-friendly version Add This

Generating Metadata

Tip: To download an xsd or dtd file, right-click and select "Save Link As..."

Overview

The topics covered on this page include information regarding the generation of metadata files in the Extensible Mark-up Language (XML) format. These metadata files are used to describe additions, removals, and changes to a provider's data holdings. It is the responsiblity of the Data Partner to regularly publish metadata reflecting the ongoing changes to its Earth Science data.

The information provided is broken up into the following topic areas:

Metadata Organization

Data providers are responsible for generating metadata files in the (XML) format that comply with the published ECHO Data Model. The following two metadata constructs are utilized by the ECHO system:

  • Collection - A grouping of science data that all come from the same source, such as a modeling group or institution. Collections have information that is common across all the granules they contain and a template for describing additional attributes not already part of the metadata model.
  • Granule - The smallest aggregation of data that can be independently managed (described, inventoried, and retrieved). Granules have their own metadata model and support values associated with the additional attributes defined by the owning collection.

Data Providers may generate a full metadata record which will be processed as an insert or full replacement. Providers may generate metadata to update or delete specific fields within a metadata record. Providers may also generate metadata to delete an entire metadata record from ECHO.

Note that deleting a collection will cascade to delete all associated granules. However, a cascaded or explicit granule deletion will not cascade to delete associated browse records.

For a complete overview of metadata organization, please refer to section 3 of the ECHO 10.0 Data Partner User's Guide

ECHO 10.0 Schemas

Metadata submitted by new ECHO partners must conform to the following XML Schema that corresponds to the particular messages involved. These schemas are subject to change through regular ECHO releases. For more information regarding upcoming changes, contact ECHO Operations at echo@echo.nasa.gov or view the following links which refer to the mode-specific Ingest documentation for Ingest Schemas, DTDs, and sample metadata files.

  • Operations Ingest Schemas, DTDs, & Sample Metadata - (HTML)
  • Partner Test Ingest Schemas, DTDs, & Sample Metadata - (HTML)
  • Testbed Ingest Schemas, DTDs, & Sample Metadata - (HTML)

Requirements and Recommendations

The ECHO Data Partner's User Guide outlines the ECHO metadata requirements and recommendations for metadata Ingest into the ECHO system. For each metadata type, the minimum metadata fields required to validate against the ECHO Ingest schema are outlined. In addition to this, we have provided a list of recommended metadata fields that should be included in data Ingested into ECHO. These additional fields will facilitate the searching and ordering of data by the Earth Science community.

Special Considerations

The following items outline some common areas which Data Partners should be aware of while generating their metadata. For a complete overview of ECHO Ingest Business Rules, please refer to:

Dates and Times

Providers utilizing the ECHO 10.0 XML schema format must conform to the XML Schema Specification. Providers utilizing the ECHO 8.0/9.0 DTD format must coordinate with ECHO Operations to define a provider-specific format to be used for all dates and times in generated metadata. Unless otherwise specified in the provided date/time value, ECHO will process all date and time fields in the GMT timezone.

Temporal Ranges

The collection and granule metadata both contain temporal ranges for the associated item. If a collection specifies a temporal range, ECHO will verify that each ingested granule falls within the specified range. If a collection does not specify a temporal range, then the verification will be skipped. If a provider performs a collection update to modify the temporal range, and granules exist in ECHO which invalidate the new range, the update will be rejected.

Spatial Types

ECHO supports three spatial data types (Orbit, Global, Geometry). Within the Geometry spatial type, ECHO supports both the Geodetic and Cartesian coordinate systems. Spatial information can be associated with collections and granules, however the data types do not need to be the same. A collection's spatial representation specifies how the extent of the entire collection will be specified. A granule's spatial representation specifies how the extent of each granule will be specified.

For example, a collection may have a spatial type of Global to designate that data within that collection will cover the whole earth. However, the associated granule's spatial type may be Cartesian to designate that each granule's spatial coverage will be described with points on the Cartesian coordinate system.

ECHO's holdings are contained within an Oracle database and must conform to the restraints inherent in the Oracle spatial model (for more information see Oracle's Spatial User's Guide and Reference). Existing spatial restrictions within ECHO include the following list. For a full listing of restrictions, see the Data Partner's User's Guide.

  • A spatial area specified in the Geodetic coordinate system cannot cover more than half of the earth.
  • A spatial area specified in the Cartesian coordinate system cannot cross the international dateline.
  • Geometric points must be specified in a clockwise direction.

Unique Values

ECHO requires specific metadata fields in order to uniquely identify collection, granule, and browse records within its holdings. These fields are validated for uniqueness during ingest and used during updates, replacements, and deletions in order to uniquely a target record. The fields which are validated for uniqueness are found below.

  • Collection - DatasetID
  • Collection - ShortName and VersionID
  • Collection - LongName
  • Granule - GranuleUR
  • Browse - ProviderBrowseId

Metadata Inheritance

Specific metadata fields have been designated to require an enforced 'inheritance' validation between collections and associated granules. This means that values for these fields which are specified at the collection level include a superset of all values which may be specified at the granule level. For example, if a collection specifies Campaigns A, B, and C, all associated granules may have any subset of these campaigns. However, if a granule specifies a Campaign D, it will be rejected. Collection and granule updates will always perform this validation check to ensure the inheritance is valid. Affected fields include the following:

  • Campaign
  • Platform
  • Instrument
  • Sensor
  • Additional Attributes