Common Metadata Repository (CMR)
What is CMR?
The Common Metadata Repository (CMR) is a high-performance, high-quality, continuously evolving metadata system that catalogs all data and service metadata records for the EOSDIS system and will be the authoritative management system for all EOSDIS metadata. These metadata records are registered, modified, discovered, and accessed through programmatic interfaces leveraging standard protocols and APIs.
The CMR system is composed of:
- CMR itself (formerly the ECHO system)
- Global Change Master Directory (GCMD)
- International Data Network (IDN)
- All related tools (internal and external)
- All metadata — including the Unified Metadata Model (UMM) concepts, the GCMD Keywords Controlled Vocabulary, and other controlled vocabularies.
NASA's Earth Science Data Information System (ESDIS) program is responsible for providing access to Earth Observation data and services to an ever-growing user community including — but not limited to — Earth Scientists, Educators, other government agencies, decision makers, and the general public. As data archives grow and more data becomes accessible online — cataloging, searching, and extracting relevant data from these archives becomes a critical part of Earth Science research.
Formerly, NASA metadata providers were required to contend with multiple, disparate systems; each requiring different formats and different mechanisms for submitting and updating data entries. As an end user or application developer, this inconsistency reduced the value of the metadata and complicated finding and using earth science data.
In response to these challenges, NASA set out to design a system that would reduce the burden on metadata providers and improve the metadata quality, consistency, and usability for end users by:
- Serving as a middleware replacement for the ECHO and GCMD’s backend. Note: Users of GCMD should see no impact - as the GCMD frontend will remain as is.
- Handling metadata at the Concept level; including Collections, Granules, Visualizations, Variables, Documentation, Services, and more.
- Managing hundreds of millions of metadata records; making them available through performing, standards based - temporal, spatial, and faceted search.
- Incorporating both human and machine metadata assessment features that work to ensure the highest quality metadata possible.
- Supporting multiple metadata standards using an evolvable Unified Metadata Model (UMM).
How does CMR do that?
The CMR is designed to handle metadata at the Concept level. Collections and Granules are common metadata concepts, but this can be extended out to Visualizations, Variables, Documentation, Services, and more. The CMR provides a flexible ingest system with metadata libraries which can handle multiple metadata record formats, multiple metadata record concepts, and relationships and validations between them. As new formats are introduced, new translations can be written for the CMR’s metadata libraries to provide ingest, validation and search support. The libraries also provide format conversions for backward compatibility.
Benefits of CMR
The CMR builds on the work done by ECHO and the GCMD to provide a unified repository for NASA's Earth Science metadata. This benefits providers by only needing to ingest their metadata into one system. It also benefits users by being able search one system and retrieve both ESDIS and IDN data.
Modern Earth Science applications strive to provide end users with nearly immediate access and interactivity across massive stores of Earth Science data. That data is discovered, navigated, and often interrogated through science metadata. As the range of applications grow and more and more information moves from the underlying science data to metadata, the challenges of navigating even just the metadata increases. CMR is designed to handle hundreds of millions of metadata records; striving to make them available in under 1 second. CMR is able to do this by a using sophisticated data indexing and caching strategies built on top of a performance infrastructure.
High performance access to metadata is only part of the problem. To be useful to a broad range of Earth Science applications, the metadata must be of high quality, complete, and consistent. The CMR incorporates both human and machine metadata assessment features that work to ensure the highest quality metadata possible. During ingest, automated metadata scoring rubrics are applied giving data providers insight into how to make their data more discoverable or usable by end users. Science Coordinators and review teams can review metadata that fails verification or lacks required information to help providers make their metadata more consistent and complete.
Consistent Metadata Representation
- The CMR's ingest framework supports services which validate distinct metadata standards such as ECHO10, GCMD DIF, and ISO19115 against a common set of core metadata elements described in the Unified Metadata Model (UMM).
The UMM describes the metadata related to key EOSDIS concepts (such as collection or granule) using UMM metadata “Profiles” (such as UMM-C for collection and UMM-G for granule).
- The CMR provides services for mapping metadata records to and from any of the CMR-supported metadata standards, through the UMM. Thus, the CMR is able take metadata records of various source formats, and convert non-ISO19115 metadata into robust and standards compliant ISO19115 representation for interested clients. This can be seen in the diagram below:
For example, for collection metadata records, a UMM-C record may be mapped to any other supported collection metadata standard such as ISO 19115-1, ISO 19115-2, ECHO 10, DIF 9, or DIF 10. Records created using any of those supported standards may be mapped to any other of those supported standards by first mapping to the UMM-C.
- As additional metadata concepts are introduced to the CMR, new ingest services will provide verification and search indexing capabilities across diverse metadata such as visualization and variable information.
- CMR Client and Developer Information
- Unified Metadata Model (UMM) information
- EOSDIS Tools Information
Contact Earthdata Support for questions, comments or technical issues.
Last Updated: Nov 22, 2016 at 4:35 PM EST