- Open Science
Furthering open science is the cornerstone of all ESDS activities. The program is committed to making NASA Earth science data available and useable by a more inclusive and diverse community as early in the scientific process as feasible. These actions form the core of an evolving scientific paradigm called open-source science. During FY 21 ESDS supported, contributed to, and laid the foundation for numerous open-source science initiatives.
ESDS open science policies (such as ESDS' Open-Source Software Policy) were incorporated into a new policy from NASA’s Science Mission Directorate (SMD) regarding the openness and accessibility of data, software, papers, and ancillary information. SMD Policy Document-41 (SPD-41), released in August 2021, is based on recommendations in NASA’s Strategy for Data Management and Computing for Groundbreaking Science 2019-2024 and is the first comprehensive policy on information management for all SMD-funded efforts.
ESDS is leading a new SMD open science effort: the Open-Source Science Initiative (OSSI). OSSI is a coordinated program of activities to help move science and the scientific process towards greater openness. Along with supporting OSSI development, ESDS is designing systems, websites, programs, and outreach as part of this agency-wide activity.
A major OSSI component is Transform to Open Science (TOPS), which will help agencies, organizations, and communities to evolve to an inclusive culture of open science. From 2022 to 2027, TOPS will act as a catalyst to jump-start a suite of coordinated activities to rapidly transform science, including the designation of 2023 as the Year Of Open Science (YOOS). ESDS is playing a key role supporting the three primary TOPS goals:
- Increase understanding and adoption of open science principles and techniques in mission and research communities
- Accelerate major scientific discoveries through supporting the adoption of open science
- Broaden participation by historically excluded communities
Another ESDS-supported open-source science effort is Openscapes. This NASA-funded initiative kicked off in March 2021 to support mentors from NASA's Distributed Active Archive Centers (DAACs) developing resources for helping researchers migrate their workflows to the Earthdata Cloud. Five DAACs are participating in the first Openscapes Cohort, and mentors from these DAACs will lead workshops to foster skills for working collaboratively with EOSDIS data in the cloud. The first of these Openscapes Champions opportunities is scheduled for spring 2022.
During FY 21, ESDS completed work leading to the public release of a revolutionary open-source platform for collaborative research—the Multi-Mission Algorithm and Analysis Platform (MAAP). MAAP is a joint NASA/European Space Agency (ESA) project that features significant involvement of NASA’s Interagency Implementation and Advanced Concepts Team (IMPACT). IMPACT is an ESDS component focused on improving data acquisition, management, analysis, and exchange.
MAAP brings together data, algorithms, and computing capabilities in a common cloud environment to facilitate the sharing and processing of data from field, airborne, and satellite measurements. Key features of MAAP are full and open access to mission data, the use of open-source code, and unrestricted access to data and ancillary information. The public release of MAAP Version 1.0 was announced jointly by NASA and ESA in October 2021, with MAAP Version 2.0 scheduled for joint NASA/ESA release in the spring of 2022.
- Data Innovation and Supporting User Needs
ESDS oversees the life cycle of NASA’s Earth science data—from acquisition through processing and distribution. Over FY 21, the volume of data in the EOSDIS archive (including cloud-based data) grew from 41.86 petabytes (PB) to 59.24 PB.
Processing and archiving this vast collection and developing the data architecture to efficiently deliver these data is the responsibility of the Earth Science Data and Information System (ESDIS) Project. This ESDS component continued its outstanding service to data users, who once again indicated their high satisfaction with EOSDIS data, products, and services.
In the annual American Customer Satisfaction Index (ACSI) survey of users of EOSDIS data over the past fiscal year, EOSDIS received an aggregate score of 81 out of a possible 100—the highest score received in the 18 years NASA has performed this independent review and an increase from the 2020 score of 79. The EOSDIS score remains well above the latest reported U.S. federal government score of 65. EOSDIS uses feedback from these annual surveys to continually evolve the system to meet ongoing community needs.
Migration of EOSDIS data to the Earthdata Cloud continued through FY 21. This work is an ESDS priority and will provide the scalability and efficiency to deal with the expected tremendous growth of the EOSDIS archive from upcoming high-data-volume missions. At the end of FY 21, 15.11 PB of EOSDIS data have been migrated, and the entire data collection is on track to be migrated over the next two to three years. Work also continued on migrating the Global Imagery Browse Services (GIBS) visualization collection to the Earthdata Cloud. At the end of FY 21, 80% of the GIBS visualization archive had migrated, with the entire system on track to be migrated by the fall of 2022.
Other ESDS cloud accomplishments over FY 21 include the launch of NASA’s first cloud-native mission (Sentinel-6 Michael Freilich) and the release of the first major EOSDIS dataset hosted in the Earthdata Cloud (Harmonized Landsat Sentinel-2 [HLS]).
Sentinel-6 Michael Freilich is a joint U.S./European satellite that launched November 21, 2020, with data archiving and distribution coordinated by NASA’s Physical Oceanography DAAC (PO.DAAC). Sentinel-6 Michael Freilich is a radar altimeter mission designed to provide high-precision measurements of global sea-level.
HLS is a joint effort of IMPACT, NASA’s Land Processes DAAC (LP DAAC), and the HLS science team. HLS harmonizes data from the Operational Land Imager (OLI) aboard the joint NASA/USGS Landsat 8 satellite for use with data from the Multi-Spectral Instrument (MSI) aboard ESA’s Sentinel-2A and -2B satellites to provide low-latency global imagery with 30-meter spatial resolution. Data from the OLI-2 aboard Landsat 9 (launched on September 27, 2021) are expected to be integrated into the HLS data stream by the spring of 2022.
Along with its HLS efforts, IMPACT technical innovations are making work with NASA Earth science data more efficient. Highlights of IMPACT FY 21 work include the web-based Catalog of Archived Suborbital Earth Science Investigations (CASEI), innovations to improve Earth science metadata, and the Worldview Image Search Pipeline:
- Developed by IMPACT’s Airborne Data Management Group (ADMG), CASEI provides quick access to detailed information about NASA’s airborne and field investigations along with links to associated data products in a single, intuitive, interconnected web interface.
- Metadata are data describing data, and a key element in making data discoverable and usable. IMPACT developed a number of metadata enhancements over FY 21. The pyQuARC Python code package performs automated quality checks for metadata standards supported by NASA’s Common Metadata Repository (CMR), which is the authoritative management system for EOSDIS data. The IMPACT Global Change Master Directory Keyword Recommender (GKR) application uses artificial intelligence (AI) to optimize keyword selections. Both pyQuARC and GKR improve data stewardship and support efforts to standardize EOSDIS metadata.
- IMPACT embraced an open science approach in partnership with the SpaceML initiative to create the Worldview Image Search Pipeline. This generalizable package of machine learning components supports a set of open science tools to simplify the use of GIBS imagery for machine learning.
Improvements and updates to NASA Worldview further enhanced the utility of this satellite imagery exploration tool. The Worldview team extended the capability to download data via Earthdata Search to almost all available Worldview layers, and users can now embed Worldview into a web page, StoryMap, or other web-based product. Finally, the Worldview team overhauled the design and deployment pipeline to make it easier and faster to update reference layers, such as place labels and coastlines. These layers now display down to 15-meter resolution, which makes it easier to work with 30-meter resolution HLS imagery.
The Land, Atmosphere Near real-time Capability for EOS (LANCE) team also continued their work enhancing the provision of near real-time (NRT) EOSDIS data. LANCE partnered with the USDA Forest Service to create the Fire Information for Resource Management System (FIRMS) US/Canada. Based on the existing FIRMS, FIRMS US/Canada provides salient NRT information for ongoing large fires in both countries, including incident locations, links to incident reports, and improved fire classification. In addition, LANCE introduced the MODIS NRT Global Flood Product, which builds on a legacy NRT flood mapping product and includes more robust production and lower latency. The product is available through GIBS for viewing in Worldview, which allows users to browse imagery and data more easily than with the legacy product.
Finally, the ESDS Commercial SmallSat Data Acquisition (CSDA) Program worked with NASA Headquarters to finalize End User License Agreements (EULAs) to expand access of purchased commercial SmallSat products from Planet Labs Inc. (Planet) and Spire Global (Spire) to U.S. federal civil agencies and government-funded researchers for scientific purposes. CSDA identifies, evaluates, and acquires commercial remote sensing imagery and data that support NASA’s Earth science research and application activities. The CSDA team also introduced the SmallSat Data Explorer (SDX) tool. SDX enables approved users to search, discover, and access CSDA-acquired products from Planet and Spire as well as EarthDEM digital elevation models (DEMs).
- Engagements
Throughout FY 21, ESDS communications teams excelled at supporting the program’s open science efforts, describing program accomplishments and technical innovations, and contributing to agency-wide campaigns.
The ESDS and EOSDIS communications teams published 86 articles on the Earthdata website, including 12 Data User Profiles, four Data Chats, six articles highlighting the release of new DAAC data products, and quarterly EOSDIS newsletters about data activities. Writers from IMPACT and GIBS contributed 33 publications through the IMPACT blog and the GIBS blog, while members of the ESDS Geographic Information Systems Team (EGIST) contributed to a chapter about mapping extreme events from space to the Esri book GIS for Science: Maps for Saving the Planet, Volume 3.
ESDS Data Pathfinders, Toolkits, and Backgrounders continue to be valuable resources for data users. Over the past fiscal year, five Pathfinders, five Toolkits, and a Nighttime Lights Backgrounder were published on the Earthdata website. New data resource pages for Sea Level Change, Wildfires, Tropical Cyclones, and Crops, Soils, and Agronomy also were published. The ESDS communications team continued its collaboration with NASA’s Earth Observatory for joint publication of articles and graphics.
The EOSDIS communications team continued its popular webinar series, producing and hosting 15 data discovery and data access webinars for 2,071 participants in more than 100 countries. Webinar recordings garnered 101,700 additional online views (up 14.6% over FY 20), 7,700 hours of watch time, and 1.4 million impressions on the NASA Earthdata YouTube channel (up 46.2% over FY 20).
Along with webinars and tutorials, new features were added to the channel, including a GIS playlist, NASA Data Made Easy Workshops, and 12 special feature videos and short data tutorials.
The ESDS and EOSDIS communications teams contributed to multiple agency-wide campaigns, including the Sea Level Rise campaign and NASA at Your Table, a food and drink campaign. The program’s multi-pronged communications efforts included the development of new data learning resources (such as special ArcGIS StoryMaps for Earth Day and the NASA at Your Table campaign) and campaign-specific social media. During the food and drink campaign, users were guided to cross-DAAC data recipes and tutorials through Data Recipe Cards that garnered more than 500,000 impressions on NASA Earthdata social media accounts.
ESDS’s participation in hackathons connected global audiences with NASA Earth science data. More than 4,300 participants from 132 countries and territories participated in the joint NASA/ESA/Japan Aerospace Exploration Agency Earth Observation (EO) Dashboard Hackathon to create solutions for dealing with COVID-19 using data from all three agencies openly available through the tri-agency EO Dashboard. IMPACT helped coordinate the 2021 Competition on Flood Detection that advanced efforts to bring machine learning and AI to the remote detection of floods. Almost 140 participants produced 460 submissions during the competition. Lastly, ESDS contributed challenges requiring the use of EOSDIS data and resources to the 9th NASA International Space Apps Challenge. More than 26,000 participants in 3,800 global teams created more than 2,300 solutions to challenges related to life on Earth and in space in this all-virtual event.
Finally, the Earthdata Forum enables data users to ask questions about NASA Earth science data and interact with peers, fellow data users, and DAAC subject matter experts. During FY 21, NASA’s Ocean Biology DAAC (OB.DAAC), Level 1 and Atmosphere Archive and Distribution System DAAC (LAADS DAAC), and the Global Change Master Directory (GCMD) joined the seven DAACs that already were part of the forum.
- Building on ESDS Accomplishments: The Year Ahead
ESDS efforts to further open-source science will continue to be a major component of ESDS activities throughout FY 22. Along with leading OSSI and TOPS efforts, ESDS is supporting development of NASA's Earth System Observatory (ESO), in which open science is a foundational element.
ESDIS is co-leading the Open-Source ESO Science Mission Data Processing Study with NASA’s Jet Propulsion Laboratory. The study has identified a diverse set of members for a System Architecture Study Working Group. The working group is charged with identifying and assessing potential architectures that can meet the ESO mission science processing objectives, enable data system efficiencies, seek opportunities that support Earth system science, and promote open science principles. ESDIS also is leading the ESO Data Latency and Ground Segment study team that is evaluating flight hardware and ground system architectures to minimize product latency and support cross-ESO science product generation. Both studies are expected to be completed during FY 22.
A redesigned Earthdata website will go live in spring 2022 with all ESDS components under one program-managed website. The updated website will provide paths to help users of all ability levels quickly find the NASA Earth science data, tools, and supporting resources they need. Additionally, migration of EOSDIS data to the Earthdata Cloud will continue. All datasets archived at PO.DAAC and the entire GIBS visualization archive are on track be migrated by the end of FY 22.
In the fall of 2021, CSDA onboarded two new vendors for synthetic aperture radar (SAR) and optical imagery data evaluation, and these data evaluations will take place during FY 22. CSDA also plans to issue a request for information (RFI) to survey the market and see what new data are available. Finally, search and discovery of CSDA-acquired NASA commercial data holdings will be possible using Earthdata Search, with access to these commercial datasets subject to dataset EULAs.