• Letter from the Program Executive
• Introduction
• Acquire and Process
• Deliver Data and Imagery
• Open Science: Setting the Standard for Open Data
• Investing in Earth Science Capabilities and Data
• Engaging a Diverse Community

Letter from the Program Executive

Welcome to the 2019 highlights of the Earth Science Data Systems (ESDS) Program. The staff, programs, projects, and initiatives that make up the program have made significant progress over the past year. Our efforts have enabled new science, explored new commercial and international partnerships and collaborations, and encouraged the development of new applications and advances in research. All of our work is done in a manner that ensures that NASA's Earth science data—while a national asset of the U.S.—are available to users around the world for research and to the benefit of our global society.

2019 marked 25 years of NASA's groundbreaking open data policies and ESDS focused on our continuing commitment to provide free and open data and tools through open systems and services. We invested in the development, testing, and deployment of new systems and services to extend our capabilities for managing an ever-growing archive as we prepare for the incoming surge of data that will be the result of future missions. We look to utilize the growing commercial cloud landscape to handle these data volumes as well as providing a forward-looking platform through which to serve our growing numbers of users.

Our competitive programs continue to yield innovative tools and techniques necessary for thinking about data in the 21st century. We have embarked on investigations into the use of commercial data to augment our own collections. We are reaching new audiences for our products and systems. We are inspired by the challenges that are before us.

We look forward to building on these successes in 2020.

Kevin Murphy

Program Executive

Introduction

Over the past 25 years, tens of millions of users around the world have benefited from NASA's groundbreaking open data policy. From scientists to citizens, the world depends upon the continuous flow of data to inform decisions and improve lives. Behind the modern flow of data are policies, developed by organizations such as NASA, that have been essential in making data available to more users than ever previously imagined. The adoption of open data policies leads to advances in research, the development of new applications, and the exposure of citizens around the globe to information essential for understanding our planet, managing the environment, and encouraging commercial innovation.

Since 1994, NASA Earth science data have been free and open to all users for any purpose, and since 2015, all data systems software developed through NASA research and technology awards have been made available as Open Source Software (OSS). It is the vision of NASA's Earth Science Data Systems (ESDS) Program to make NASA's free and open, Earth science data interactive, interoperable, and accessible for research and societal benefit from today to the future.

The ESDS Program oversees the life cycle of NASA's Earth science data—from acquisition through processing and distribution—maximizing the scientific return from NASA's Earth-observing missions and experiments for research and applied scientists, decision makers, and society at large. Within ESDS are a number of programs, projects, and initiatives that collectively strive to achieve common goals:

• Set the standard for the efficient production, archiving, and delivery of science-quality data related to planet Earth.
• Invest in open and adaptable capabilities for the next generation of missions and data sources for groundbreaking Earth science.
• Lead the research and development of technology for management and analysis of complex Earth science data.
• Harness the diversity of the global Earth science community to develop a world-class program.

The Earth Science Data and Information System (ESDIS) Project manages and oversees the Earth Observing System Data and Information System (EOSDIS). EOSDIS is the core operational system that provides end-to-end capabilities for managing NASA Earth science data from various sources—satellites, aircraft, field measurements, and various other programs. It is comprised of twelve discipline-specific Distributed Active Archive Centers (DAACs) that are relied upon by millions of users each and every year for access to NASA Earth science data.

The Interagency Implementation and Advanced Concepts Team (IMPACT) is an interdisciplinary team that works to maximize the scientific return of NASA's missions and experiments through developing interagency collaborations, assessing and evaluating ESDS processes, and deploying strategic and technical expertise for rapid prototyping, development, and testing of advanced concepts in data and information systems for Earth observations.

Other ESDS program components include the Data System Evolution Program, which includes Competitive Programs and the Earth Science Data Systems Geographic Information Systems Team (EGIST); collaborative projects such as the Multi-Mission Algorithm and Analysis Platform (MAAP); and new data-acquisition initiatives such as the Commercial SmallSat Data Acquisition (CSDA) Program. ESDS also supports user working groups and the development of standards for data quality, formats, metadata, discoverability, and usability.

Together we:

• ACQUIRE AND PROCESS Earth-observing instrument data to create Earth System Data Records (ESDRs).
• DELIVER NASA's Earth science data and actively manage those data as a national asset.
• OPEN AND FREELY SHARE all data, tools, and ancillary information to all users.
• INVEST in the development of data system capabilities optimized to support rigorous science investigations and the unique needs of various scientific disciplines.
• ENGAGE members of the Earth science community in the evolution of data systems.

Acquire and Process

Data is at the heart of what we do. For over half a century, the data we acquire and process have provided a critical resource for scientists studying the Earth and its systems. NASA's Earth science data are continuously gathered from satellite, airborne, and in-situ instruments and processed to create high quality long-term ESDRs. ESDRs are critical to understanding Earth system processes and in assessing variability, long-term trends, and changes in the Earth system. ESDS and its components support the missions, investigations and campaigns that produce data and are responsible for ensuring those data's quality, from acquisition through archival and distribution.

New Missions and Datasets

Following satellite and sensor launches in 2018, ESDIS worked closely with the science and ground systems teams of new on-orbit missions to support the routine collection and archiving of new datasets.

• The GRACE Follow-On (GRACE-FO) mission made its first science and gravity field data available for analysis from the Physical Oceanography DAAC (PO.DAAC) in May 2019.
• The ECOsystem Spaceborne Thermal Radiometer Experiment on Space Station (ECOSTRESS) was launched in 2018 to measure evaporation and transpiration from plants. From 2018–2019, the instrument teams, along with the Land Processes DAAC (LP DAAC), worked through several scenarios for data acquisition, making more data available to the public.
• The Ice, Cloud and land Elevation Satellite-2 (ICESat-2), launched in 2018, began releasing global data measuring the relative height of Earth's surface features, especially glaciers, ice sheets, and sea ice in May 2019 via the National Snow and Ice DAAC (NSIDC DAAC). Data from NASA's Operation IceBridge, also available from NSIDC DAAC, were used to validate ICESat-2's measurements.
• After launching to the International Space Station (ISS) in November 2018, the Global Ecosystem Dynamics Investigation (GEDI) released its first global, high resolution observations of the forest vertical structure in November 2019 from LP DAAC.

Airborne Missions

The Airborne Data Management Group (ADMG), part of the IMPACT program, provided support to the five Earth Venture Sub-orbital 3 (EVS-3) projects as they planned and prepared for project implementation. ADMG assisted with assigning projects to DAACs, provided a Data Management Plan (DMP) example and an airborne data manager guide to help with using the new DMP template developed last year by the Earth Science Data Systems Working Group (ESDSWG), and identified roles and expectations of all parties that handle future EVS-3 data. ADMG continues to provide support and advice to the DAACs as they prepare for receiving EVS-3 investigation data in 2020. The DAACs will publish the data and provide data support and outreach to data users.

Addressing Earth Observation Satellite Needs

The IMPACT program continued to support assessments by the Satellite Needs Working Group and worked to develop responses to identified needs. One identified need was to improve the frequency of global surface reflectance observations in order to support land monitoring applications research. To that end, IMPACT initiated the production of the Harmonized Landsat and Sentinel (HLS) data product which harmonizes observations from multiple observation platforms to improve temporal coverage.

Making Earth System Data Records for Use in Research Environments (MEaSUREs)

The MEaSUREs projects, an element of Research Opportunities in Space and Earth and Science (ROSES), provide an opportunity for the research community to participate in the development and generation of data products that complement and augment the Earth science already available to the research community. The MEaSUREs projects are focused on product generation, availability, and the utility of ESDRs. In FY 2019, data from 24 MEaSUREs projects were assigned to DAACs for eventual archive and distribution.

Deliver Data and Imagery

Investments in technology, infrastructure, new data capabilities, and mission support come together to achieve one of our most visible goals: the delivery of data and imagery to the widest possible audience. Every year we see increases in the raw metrics related to our data and imagery downloads. More users than ever are seeking out and using NASA Earth science data and our systems and services must continuously evolve to take advantage of improved technology to meet the increasing demands of data providers and users.

Ongoing Missions

Twelve DAACs are located across the U.S. at centers of discipline-specific expertise. Through these 12 DAACs EOSDIS maintains, archives, and distributes NASA Earth science data. In 2019, the DAACs served a combined total of over 1.3 million users who downloaded NASA Earth science data.

Average archive growth in FY 2019 was 20 TB per day, with over 7.2 PB of data (298 million files) added to the EOSDIS collection over the course of the year. DAACs added more than 650 new and reprocessed datasets to the EOSDIS collection and assigned over 8,000 Digital Object Identifiers (DOI) to datasets, enabling users and data providers to meet publication guidelines established by the user community for data citations. By the end of FY 2019, DAAC holdings included approximately 12,000 datasets with a total volume of 34 PB.

Near Real-Time Data

LANCE

2019 marked the tenth year of operations for the Land, Atmosphere Near Real-time Capability for EOS (LANCE), NASA's conduit for the rapid delivery of satellite data. The products, services, and data distribution strategies developed by the LANCE team have helped transform not only how Earth observing data are used, but also the worldwide accessibility of these data.

New near real-time (NRT) products were added to LANCE: the joint NASA/NOAA Suomi National Polar-orbiting Partnership (Suomi NPP) Ozone Mapping and Profiler Suite (OMPS) Aerosol Index (PyroCumuloNimbus) imagery product; the Moderate Resolution Imaging Spectroradiometer (MODIS) Multi-Angle Implementation of Atmospheric Correction (MAIAC), which supports the air quality and climate/atmospheric modeling communities; the MODIS Thermal Alert System (MODVOLC), a thermal volcano monitoring dataset; and several new JPSS-1/NOAA 20 Level 1 products. LANCE distributed over 90 million files from 11 instruments to over 350,000 users in FY 2019.

The Fire Information for Resource Management System (FIRMS) is a component of LANCE that distributes NRT active fire data within three hours of satellite observation from the MODIS and the Visible Infrared Imaging Radiometer Suite (VIIRS).

In 2019, FIRMS added the MODIS burned area dataset into the map viewer; added fire detections from the Joint Polar Satellite System (JPSS), also known as NOAA-20; and improved the fire imagery generation process, speeding up web server performance and reducing time between acquisition and delivery.

FIRMS offers email alerts notifying subscribers of fires in their area of interest. In 2019, an average of 15,000 rapid, daily, and weekly email alerts were delivered each week to users in over 160 countries, and the FIRMS website was viewed by 540,000 users through 1.7 million page views. In the fall of 2019, FIRMS provided the media with data about fires in the Amazon, staff fielded questions about the data and FIRMS imagery appeared in articles on CNBC, the Washington Post, CNN, and Euronews.

Worldview and Global Imagery Browse Services

The Worldview data visualization application allows users to interactively browse global, full-resolution, satellite imagery layers and download the underlying data. The imagery in Worldview is served by the Global Imagery Browse Services (GIBS). In 2019, Worldview/GIBS added the capability to browse geostationary imagery provided by NASA's Marshall Space Flight Center Short-term Prediction Research and Transition Center (SPoRT) from the Geostationary Operational Environmental Satellite-East (GOES-East), GOES-West, and Himawari-8 satellites. Users can interactively browse these imagery, create playback loops, export videos, and combine with more than 900 other imagery layers provided by GIBS.

Worldview's new “Tour Story” feature allows users to browse stories about events going on around the world, including interesting facts about the imagery, imagery tutorials, and information on how to use Worldview. A story is added every month to promote new imagery products, features, and natural events.

Sea Level Data and Tools

Established in 2014 through a ROSES competed grant, the Sea Level Change Portal is a one-stop source for datasets relevant to the study of sea level and is also the platform for the NASA Sea Level Change Team to publish datasets they produce. Eight new datasets were added to the portal in 2019.

The Sea Level Change Data Analysis Tool (DAT) is a publicly available interactive tool for studying sea level. The tool allows users to view and compare data layers, work with a region of interest, and analyze and compare time series. The Amazon Web Services (AWS)-hosted back end and version 2.0 of the tool were released in December, 2019. DAT 2.0 expands on the previous version to include a time slider and the ability to visualize data in 3-D.

A new Flooding Days Projection Tool was also released in December, 2019. This interactive infographic tool provides annual and decadal projections of flooding days for a selection of coastal cities.

Open Science: Setting the Standard for Open Data

2019 marked the 25th anniversary of NASA making all its data free and open to research and applications communities, private industry, academia, and the general public. To enable the efficient production, archival, and delivery of data, the ESDS Program and its projects participate in and lead efforts to develop data standards, create and curate metadata, and build tools to enable search and discovery which ensure interoperability with other systems, agencies, and organizations.

Data and Metadata Standards

In 2019, the ESDIS Project led an international effort to develop the ISO standard 19165-2 "Geographic information—Preservation of digital data and metadata—Part 2: Content specifications for Earth observation data and derived digital products." Once finalized, the standard will define how Earth science data and associated knowledge will be documented to facilitate future reuse. There has never been a standard in this area before.

In addition, the ESDIS Standards Coordination Office (ESCO) published a valuable suite of documents related to capturing, conveying, and enabling the use of data quality information. Data quality flags and processes ensure that the data captured by Earth science missions are used correctly and appropriately in a way that is common across science disciplines. The documents were developed within NASA's Earth Science Data System Working Groups (ESDSWG) framework by the Data Quality Working Group.

ESCO also released several standards after conducting community reviews, including key enhancements to the Unified Metadata Model. Recommendations on search relevance (PDF) and further community conventions for dataset interoperability (PDF) were also released, enabling the construction of more tool- and user-friendly formats.

The approval this year of GeoTIFF as an ESCO standard format will enable the storage of EOSDIS standard data products (e.g., Synthetic Aperture Radar) in this tool-friendly format, particularly in its “Cloud-optimized” form. This approval followed the approval of the GeoTIFF specification by the Open Geospatial Consortium, a specification to which EOSDIS personnel contributed significant content.

The ESDIS Project team, working with the NASA Office of the Chief Information Officer (OCIO) (curator of data.nasa.gov), succeeded in enhancing the metadata records for NASA Earth science in data.gov, exposing a rich set of links for data services and related information. In particular, the metadata in both data.gov and data.nasa.gov was enriched to include links to browse images, data access services, and documentation pages. The benefits of this enhancement are shown by the typically high ranking of data.gov results in data search tools such as Google Dataset Search.

IMPACT's Analysis and Review of Common Metadata Repository (ARC) was established to improve the discoverability, accessibility and usability of NASA's Earth science data holdings by ensuring all NASA collection and granule level metadata records in NASA's Common Metadata Repository (CMR) meet a minimum standard of quality. Since its founding in 2018, the ARC team has developed novel methodologies, automated processes, and operational tools to substantially improve more than half of NASA's metadata records. Their improvements have enhanced the ability of an end user to find NASA Earth science data and ensured that data is accessible to the user.

The ARC team, lead by Jeanné le Roux, received NASA Marshall Space Flight Center's Group Achievement Honor Award in 2019 in recognition of their unceasing commitment to innovation, collaboration, and teamwork which have enhanced the quality of NASA data systems in furtherance of Marshall's Earth science mission.

Keyword Curation

The IMPACT team supported the U.S. Global Change Research Program's (USGCRP) Global Change Information System (GCIS) team in the curation of new keywords related to climate change and resiliency. The IMPACT team collaborated with the GCIS team and the Global Change Master Directory (GCMD) team to include over 50 new climate-relevant keywords in the GCMD keyword list. The GCIS team has been including GCMD keywords on all resources documented for the Fourth National Climate Assessment, Volume II report.

Algorithm Publication

The IMPACT project team led the effort to develop the Algorithm Publication Tool (APT). The APT eases the development of algorithm theoretical basis documents (ATBDs), the key documents needed to describe the physical and mathematical background of the algorithms that are used to generate data products. The APT prototype supports the modernization of ATBDs by standardizing the structure and content provided in these documents; providing an interface to author and publish ATBDs; generating ATBDs as searchable web pages and PDF documents; supporting the generation of rich text within ATBDs including equations, figures and tables; and creating a centralized repository for search and discovery.

Advancing Use of GIS Technology

The Earth Science Data Systems GIS Team (EGIST) was chartered in March 2019 to enable the appropriate use and adoption of GIS technology, provide guidance, evaluate GIS technology, and engage the community in developing program-wide strategy.

Over the last year, EGIST engaged with 10 DAACs to complete an inventory of geospatial web services and GIS software deployed and used by the DAACs. EGIST updated the progress made by the Geospatial Web Services Working Group (GWSWG), part of ESDSWG, in 2017, with the goal of maintaining an inventory of GIS capabilities across Earth Science components and the Distributed Active Archive Centers (DAACs) to encourage development, coordination, maturation and evaluation of GIS at ESDS. The inventory was captured and summarized on the Earthdata wiki for reference as well as for users who could identify end points and catalogs available to access data through primary GIS tools.

The EGIST team also engaged ESDIS, IMPACT, and GIBS to discuss support for the Open Geospatial Consortium and recommended short-term solutions to allow harvest, discovery, and use of NASA Earth science data and services in platforms such as GeoPlatform and Esri's Living Atlas. EGIST also proposed new processes to better prioritize and track the status of feedback from GIS users to primary GIS vendors such as Esri.

Collaborating on Data Sharing and Processing

Pilot Phase development of the Multi-Mission Algorithm and Analysis Platform (MAAP), a collaborative project between NASA and the European Space Agency (ESA), was concluded in September 2019. MAAP brings together relevant data, algorithms, and computing capabilities in a common environment in order to address the challenges of sharing and processing data from field, airborne, and satellite measurements related to ESA and NASA missions. The pilot and Version 1 platforms were released to a limited number of users in the User Working Groups (UWG) for use and testing, marking the beginning of the Pilot Operations Phase. Feedback from the UWG will inform the development and expanded capabilities of MAAP. The platform will be released to the public with expanded capabilities (MAAP Version 1) in 2021.

Investing in Earth Science Capabilities and Data

As our systems continue to evolve through the development of new technologies and infrastructure, we must also pursue rigorous support of existing operational missions and constantly strive to support new missions and new datasets. Trends in the development of the long-term data records utilized by NASA scientists—whether due to new sensors, new collaborative partnerships, or innovative interdisciplinary research—require us always to pursue new ideas and develop new capabilities.

Leading Technology Research and Development

The ESDS Program is taking great strides to research and develop new technologies that enhance our ability to maintain a robust and open archive of data, while exploring new approaches to the ingest and distribution of data using cutting edge tools and infrastructure. We actively enhance our capabilities through partnerships with government and commercial organizations, by encouraging new types of data access through our competitive programs, and by reaching new communities through the support of citizen science.

Cloud Evolution

By 2022, the ingest rate of data into the EOSDIS archive is projected to grow to as much as 47.7 PB per year. As this ingest rate increases, the volume of data in the EOSDIS archive is also expected to grow—from nearly 32 PB today to more than 37 PB by 2020; by 2025, the volume of data in the EOSDIS archive is expected to be more than 246 PB.

As the volume of mission data increases, so too does the need for technologies that can adapt and scale. In 2019, the ESDIS Project completed several major developments focusing on the use of the commercial cloud to address these data archival and processing needs via a system known as Earthdata Cloud. These developments will enable new ways to connect to NASA's collection of rich Earth science datasets while improving the efficiency of data systems operations, increasing user autonomy, maximizing flexibility, and offering shared services and controls.

Earthdata Cloud consists of the NASA-Compliant General Application Platform (NGAP), a custom-built cloud optimized platform, and Cumulus, a cloud-based framework for data ingest, archive, distribution, and management.

The development of the fully operational NGAP was completed in 2019. NGAP now provides the underlying platform and framework used by ESDIS to set up multiple data archive instances. NGAP resides on Amazon Web Services (AWS) and provides highly flexible cloud native infrastructure, NASA-compliant IT Security controls, networking services, and business cost controls and is available through the OCIO Enterprise Management Cloud Computing (EMCC) environment.

Significant new features and capabilities were also added to Cumulus, including:

• auto-scaling functionality to support large ingest loads;
• support for bulk re-ingest of data and an ingest prioritization mechanism;
• enhanced Cumulus Operations Dashboard capabilities based on feedback from designers and operational users; and
• thorough online documentation of ingest and distribution procedures, including training materials for new developers and integrators.

EOSDIS System Architect Katie Baynes won the Engineering Excellence award from NASA's Goddard Space Flight Center Software Engineering Division for her work on Cumulus.

Significant cloud-related milestones involving current datasets and future missions were met during 2019. Initial software was completed for cloud systems in support of the NASA/Indian Space Research Organisation SAR (NISAR) and Surface Water and Ocean Topography (SWOT) missions due to launch in 2022. The PO.DAAC and Cumulus teams successfully completed 100% of technical qualification requirements for the SWOT archive and distribution, and more than 400 datasets (comprising 32 TB of data) from NASA's Global Hydrometeorology Resource Center DAAC (GHRC DAAC) were moved to the cloud, making it the first DAAC to make all datasets available from a commercial cloud.

New cloud-based data services were added, enabling users to access and manipulate large quantities of data without requiring the transfer of data files to the user's workstation. A high-performance, cloud-capable version of OPeNDAP was developed and the OPeNDAP Server was tested with popular end-user data tools, including Panoply, the Geospatial Data Abstraction Library (GDAL) and Jupyter notebooks with xarray (part of the Pangeo suite of technologies). The Harmony project, a trans-EOSDIS framework to implement data transformation services in the Earthdata Cloud, was inaugurated with several prototypes. These prototypes have the ability to execute both off-the-shelf transformation tools (e.g., GDAL) and DAAC-provided code (e.g., PO.DAAC's Level 2 Subsetter).

Commercial SmallSat Data

In 2017, NASA initiated a program called the Private-Sector Small Constellation Satellite Data Product Pilot Project to evaluate how observations derived from Earth-orbiting, small-satellite constellations can provide a cost-effective means to augment observations from the agency's fleet of orbiting Earth science missions.

Under the pilot program, NASA awarded contracts to three companies (Planet Labs, Inc., Maxar (formerly DigitalGlobe, Inc.), and Spire Global, Inc.) that met the criteria within the public Request for Information (RFI). Data from selected vendors was evaluated by 34 principal investigators (PIs) selected through the ROSES solicitation. The pilot evaluation program took place from January 1, 2019 to December 16, 2019; final summary reports will be delivered in mid-2020.

Having access to high resolution commercial data allowed the PIs to disaggregate NASA Earth science satellite data to a finer scale, providing better spatial understanding of the phenomena being studied across Earth science's research and analysis and applied science disciplines.

Data acquired during the evaluations are available, as are data from the Teledyne Brown Engineering DLR Earth Sensing Imaging Spectrometer (DESIS). A web-based tool, the SmallSat Data Explorer (SDX), has been developed to allow PIs to search, discover, and access Planet data. These products are available at no cost to PIs and are subject to scientific use licenses.

In 2019, the Commercial SmallSat Data Acquisition Program (CSDA)was established to continue the program and open it to other qualified vendors.

As the capabilities of commercial satellite vendors grow, NASA's Earth Science Division will continuously monitor the development of these companies and acquire relevant data to complement NASA's Earth observation data.

Competitive Programs

ESDS competitive programs focus on the development of new Earth science data products and innovative technology by engaging external researchers and software developers through peer-reviewed competition. Current programs include Advancing Collaborative Connections for Earth System Science (ACCESS) and Citizen Science for Earth Systems Program (CSESP).

Advancing Collaborative Connections for Earth System Science (ACCESS)

The goal of the ACCESS program's funded projects is to develop and implement technologies for effectively managing, discovering, and utilizing NASA Earth observing data for scientific research and applications. These new technologies enable ESDIS to evaluate innovative systems, applications, and programs for integration with EOSDIS systems.

ACCESS 2017 projects, which kicked off in 2018, summarized their first year of work in midterm reports and one-year technical reviews. Over the next year, ACCESS teams will be finishing their projects and integrating with EOSDIS systems. All ACCESS projects are members of the Community Systems Integration Working Group (CSIWG), which was initiated in 2019 to link ACCESS PIs to the EOSDIS community in order to facilitate the integration of ACCESS technologies into EOSDIS.

• The Community Tools for Analysis of NASA Earth Observation System Data in the Cloud project is facilitating the Geoscience community's transition into cloud computing by building cloud-compatible data discovery and processing capabilities on top of the growing Pangeo ecosystem of open-source tools.
• The Data Access and the ECCO Ocean and Ice State Estimate is improving NASA's Estimating the Circulation and Climate of the Ocean (ECCO) model with automated preprocessing and transformation. During the project's first year, the team refined the overall ECCO-Cloud architecture and established a fully automated cloud-based, serverless data processing interface between the AWS Cloud and NASA Ames Research Center's Pleiades.
• The Multi-Temporal Anomaly Detection for SAR Earth Observations team is using machine learning to detect anomalies in SAR data. The project set up an AWS-based SAR analysis system using open source software and is testing the system with use cases that include floods, landslides, and volcanic uplift.
• STARE: SpatioTemporal Adaptive-Resolution Encoding to Unify Diverse Earth Science Data for Integrative Analysis uses numerical indices to coalign diverse data. STARE data variety harmonizing technology is now ready for testing.
• The Systematic Data Transformation to Enable Web Coverage Services (WCS) and ArcGIS Image Services within ESDIS Cumulus Cloud team is developing geospatial data transformation plugins that can be used within the ESDIS Cumulus environment to serve data products that are compatible with commercial off-the-shelf and open-source geographic information system software.

Two ACCESS 2015 projects are continuing work under supplemental funding: Terra Data Fusion and OpenAltimetry.

Terra Data Fusion moved the 2.4 PB file-based dataset generated from all the data collected by the five instruments on the Terra satellite from 2000-2015 to the AWS cloud. A partial dataset of Terra Data Fusion is available to users through Amazon's public cloud. The full dataset is expected to be available to researchers in 2020.

Elevations from the ICESat-2 Ocean (ATL12) and sea ice products (ATL07, ATL10) are now available in OpenAltimetry, along with Global Geolocated Photon Data (ATL03), Land Ice Height (ATL06), and Land and Vegetation Height (ATL08).

OpenAltimetry, a web application for map-based discovery and visualization of ICESat and ICESat-2 data, provides on-demand access to underlying datasets.

The OpenAltimetry application team has made several improvements over the past year in response to user requests. New features include the ability to zoom to specific coordinates, improved plotting capabilities, and interactivity with Jupyter notebooks.

NASA Citizen Science for Earth Systems Program (CSESP)

CSESP is focused on developing and implementing projects that harness contributions from members of the general public to advance our understanding of the Earth as a system.

The six projects selected for the CSESP Implementation phase completed their first year of work and are successfully moving into their second year.

• The Lake Observations from Citizen Scientists and Satellites (LOCSS) team expanded their network of lake height gauges across the U.S. and around the world.
• The Citizen Science Campaign to Validate Snow Remote Sensing Products team developed a new, more accurate method for predicting snow water equivalent from snow depth.
• The Can Citizen Science and Low-Cost Sensors Help Improve Earth System Data? team developed a comprehensive sensor installation toolkit to help citizen scientists to better place sensors to measure air quality.
• The Citizen-Enabled Aerosol Measurements for Satellites (CEAMS) team upgraded the Aerosol and Optical Depth instrument version 2 (AMODv2) device which allows scientists to measure fine particle air pollution.
• Soundscapes to Landscapes (S2L) completed a year-one field campaign in June 2019 that is helping researchers develop machine algorithms for bird identification.
• The Using Citizen Science to Understand Thirty Years of Change in Global Kelp Cover by Expanding the Zooniverse to NASA Satellite Imagery project has resulted in a substantial dataset of kelp classifications for the Falklands Islands (85% complete), including observations by 5,538 citizen scientists who have performed 129,891 classifications on the site.

Deep Learning and Artificial Intelligence

Advancements in deep learning and artificial intelligence (AI), combined with the ever-increasing volume of Earth science data, have created opportunities for new techniques to address the challenges of data discovery and analysis. Machine learning techniques can also augment and improve data systems operations via automation and search and discovery services. A recent example is the application of AI capabilities to enhance the detection of smoke plumes from geostationary satellite measurements undertaken by the IMPACT team.

In a recent study, IMPACT data scientists used a large number of satellite images with confirmed smoke plumes to train a convolutional neural network (i.e., a machine learning algorithm) to predict the probability that a new incoming shortwave reflectance pixel contains smoke.

IMPACT also executed a Space Act Agreement with Google, LLC on behalf of NASA to address data discovery, access, and use challenges specifically related to large volumes of NASA science data. IMPACT team members continue to work closely with Google and NASA's Frontier Development Lab (FDL), an applied AI research partnership with Google which focuses on interdisciplinary problem solving to develop future AI applications in data-intensive areas. Activities under the agreement include, but are not limited to: collaborations related to transferring and storing large volumes of data, improving data discovery, demonstrating discoveries possible through big data, and capturing lessons learned.

Engaging a Diverse Community

NASA Earth science data, services, and tools serve a diverse international community. ESDS meets the needs of these audiences via strategic domestic and international collaborations and partnerships as well as directly engaging with data users via the Earthdata.nasa.gov website, user working groups, and other opportunities for outreach.

International Collaboration

Under the NASA Open Data Policy, NASA Earth science data have been distributed to every country of the world. The countries accessing the most data from EOSDIS include China, United Kingdom, Germany, Japan and France. The ESDIS Project hosts NASA's Sentinel Gateway in support of the NASA-European Space Agency (ESA) bilateral agreement to mirror Sentinel data from ESA's Copernicus Program in the United States. NASA's Sentinel Gateway moved over 2 million files (over 2.7 PB) of data from Sentinel-1A and 1B, Sentinel 3, and Sentinel 5P data to the DAACs for user distribution. During this period, EOSDIS distributed over 8.9 million Sentinel data files (almost 9 PB) to user communities.

By enabling this Gateway, ESA saves resources that would have otherwise been required to serve the worldwide user community and NASA scientists get fast unlimited access to valuable data. ESDIS also continued discussions with the Japan Aerospace Exploration Agency on the distribution of their most popular datasets, including the Advanced Land Observing Satellite (ALOS-2) and the Advanced Microwave Scanning Radiometer-2 (AMSR-2).

The ESDIS Project continues to provide significant support for the Committee on Earth Observation Satellites (CEOS) supporting the Working Group on Climate (WGClimate) and the Working Group on Information Systems and Services (WGISS) among others. ESDIS shared and discussed technical solutions and approaches to common problems and identified opportunities for further inter-agency cooperation.

ESDIS focused particularly on ways to improve the interoperability of various data systems. These efforts included new advancements in OGC standards like NASA's adoption of a Spatiotemporal Asset Catalog (STAC) interface for the Common Metadata Repository (CMR) that utilizes the OGC API functionality, best practices for data citation and use of Digital Object Identifiers (DOIs) were developed, and Open Data Cube (ODC) initiatives were launched across multiple agencies. ESDIS staff presented tool demonstrations to CEOS WGISS meetings in 2019 highlighting improvements to shared CEOS resources such as NASA's new International Directory Network (IDN) search and discovery portal.

The IMPACT team, in collaboration with the Geoplatform.gov team, released the Resilience Community in Geoplatform.gov. The Resilience Community is an interactive, topically-focused web portal to share web content, datasets, services, maps, applications, and tools relevant to environmental change and climate resilience. The teams also collaborated to incorporate key climate relevant datasets from the Climate Data Initiative (CDI) into Geoplatform.gov, including relevant Earth observation data from NASA.

NASA continued to partner with Google Earth to create and publish interactive stories that leverage NASA Earth observations merged with Google's expertise in online data and imagery visualization platforms. These stories are featured on Google Voyager and enable users to learn about and see the uses of NASA Earth science data, reaching a wider audience. The GLOBE Observer story, Stopping the Spread of Zika, was published on February 15, 2019 and was a collaboration between Google, the GLOBE program, and the ESDS Program on a project that was funded by both GLOBE and the U.S. Department of State.

ESDS participation in the NASA-Google partnership was recognized through the NASA Team Excellence Award in 2019.

User Engagement and Communications

Engaging with Users

2019 saw an increase in EOSDIS data and service users: 3.5 million users accessed EOSDIS for information, data, and services. Of those 3.5 million users, 1.3 million downloaded almost 2 billion data files. The flow of science products to users averaged 102 terabytes per day. Earthdata.nasa.gov, the on-line gateway to EOSDIS data centers and services, also saw an increase in access with well over 3.5 million visits.

ESDIS and the DAACs engage with users in order to determine priorities, the needs for new capabilities, changes in support, and general communication. User Working Groups (UWGs) are composed of over 200 subject-matter experts from academia, industry, and other areas who provide guidance and direction for the DAAC they serve.

The DAACs further serve the community by responding to user questions, supporting science team meetings, and, along with ESDS and ESDIS staff, meeting with users at major science conferences such as the annual meetings of the Ecological Society of America, American Meteorological Society, and the American Geophysical Union.

ESDIS also oversees an independent annual survey of EOSDIS and DAAC users through the CFI Group American Customer Satisfaction Index (ACSI). The ACSI is a leading national indicator of customer satisfaction for more than 300 companies. For the past 15 years, the ESDIS Project has published the results of this survey of thousands of users. The 2019 score, 78, reflects the continued stable trend of high user satisfaction in EOSDIS systems and services, despite substantial system changes and evolution, and represents a world class score. It also serves as a metric for use by the Office of Management and Budget on the performance of NASA Earth Science Data Systems.

Communications

Communications efforts in 2019 were active across the whole of the ESDS Program. The EOSDIS webinar program presented 17 Earth science data discovery and data access webinars and three specialty GIS webinars for NASA or NASA-affiliated data users. Just over 1,900 participants from more than 200 countries actively participated in the live presentations. The recorded webinars garnered close to 65,000 additional views on the NASA Earthdata YouTube channel.

In addition to the webinars, more than 50 new articles were published on the Earthdata website in 2019. These articles covered a wide range of topics—from short announcements about new EOSDIS and DAAC products and services to feature-length articles. Numerous articles about mission milestones were published, including the first release of ICESat-2 data, new data from instruments on the ISS, a coordinated LP DAAC/Terra Science Team/EOSDIS announcement for the next version of the Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) Global Digital Elevation Model, the first release of GRACE-FO data, and a feature promoting 10 years of LANCE. The Earthdata Data User Profile series, now in its sixth year, continues to showcase the breadth of work enabled by NASA Earth observing data.

The ESDS communications teams supported Earth Science Division-wide communications campaigns by developing Toolkits, a new content type that aggregates and describes data and other relevant resources on specific topics. Toolkits were developed on Wildfires and Freshwater Availability.

Another set of helpful resources developed and published in 2019 are Data Pathfinders. Pathfinders are designed to guide new NASA data users through the process of selecting and using applicable datasets. Pathfinders were developed on Wildfires, Health and Air Quality, Water Quality, and Agriculture and Water Resources.

Read our Highlights from 2018.