CSDA @AGU2023
Posters
Publishing MAXAR Data to NASA Earthdata Cloud using the Cumulus Framework
IN13B-0568: Monday, December 11, 2023, 14:10–18:30 PST
The poster discusses NASA’s efforts to enhance accessibility to MAXAR satellite imagery data through cloud-based architecture and data pipelines within the Earthdata Cloud. Utilizing the NASA Commercial SmallSat Data Acquisition program, the petabyte-scale MAXAR data archive is being migrated to AWS cloud and incorporated into the ESDIS Earthdata Cloud environment using the Cumulus framework. This initiative aims to improve access for the broader NASA scientific community.
SREDD: Framework to facilitate MLopS related to discovery of commercial satellite data of multiple resolution
GC11E-0887: Monday, December 11, 2023, 08:30–12:50 PST
This poster presents Super Resolution Event Detection Dashboard (SREDD), an architecture designed to automate the ML model inference process. The architecture leverages Amazon S3, Elastic Cluster Service (ECS), Managed Workflows for Apache Airflow (MWAA), and other cloud-based services that efficiently handle geospatial data management. We collected daily Planet imagery of 20 lakes across the USA, Canada and Europe. Attention U-Net was used to train over 15 lakes on four bands, Red, Green, Blue, and NIR, and tested over five lakes.
eLightning
Planet Utilities: A Python Package for Efficient Processing and Analysis of PlanetScope Satellite Data
IN24B-10: Tuesday, December 12, 2023, 16:27–16:30 PST
This session introduces “Planet Utilities,” a Python package aimed at simplifying the processing and analysis of high-resolution PlanetScope satellite data. It offers a range of functions for metadata extraction, data reading, image plotting, reprojection, supervised classification, data validation, and feature extraction. The package accommodates users of varying expertise levels, providing both a user-friendly interface for beginners and customization options for advanced users, supported by extensive examples and tutorials. Overall, it serves as a valuable tool for researchers, students, and practitioners in fields like environmental monitoring and land cover classification, fostering collaboration and data-driven discoveries through its open-source nature on GitHub.
Presentations
Evaluation of Computer Vision Based Flood Mapping using High Resolution Satellite Imagery
H11A-04: Monday, December 11, 2023, 09:00–09:10 PST
Computer vision techniques with high-resolution satellite imagery are pivotal for swift and precise mapping of flood-hit areas, offering an alternative to time-consuming traditional remote sensing methods. Initially, a Machine Learning (ML) model was developed and trained on a single flood event, showcasing its effectiveness in predicting affected areas on a small dataset. Now, an updated investigation involves refining the ML model using a larger training dataset encompassing about 5000 samples from multiple major flood events, aiming to assess its performance across diverse spatial and temporal conditions. This evaluation includes comprehensive accuracy assessments based on machine learning and remote sensing standards, alongside metrics like processing speeds and scalability.
NASA’s Commercial SmallSat Data Acquisition (CSDA) Program Data System Services
IN34B-05: Wednesday, December 13, 2023, 16:40–16:50 PST
The NASA Commercial SmallSat Data Acquisition (CSDA) program aims to acquire and manage commercial satellite data that aligns with NASA’s Earth sciences missions. They’ve developed a scalable data system ensuring efficient management of acquired commercial data, focusing on improving end user services for data access and information about archived products. The presentation outlines recent additions to the CSDA data, enhancements to search and download functions via the SmallSat Data Explorer, integration with the Earth Observing System Data and Information System, efforts for long-term preservation of commercial datasets, and new user services for better dissemination of acquired data information.
GHG @AGU2023
Poster
Utilization of Multi-Agency Collaboration and a Comprehensive and Modern Data Management Approach to Achieve an Impactful Greenhouse Data Exploration and Analysis System
IN41C-0612: Thursday, December 14, 2023, 08:30–12:50 PST
The complexity of data management is amplified in multi-agency environments, necessitating collaborative efforts to develop a tailored data governance strategy. The NASA MSFC Information System Development and Integration team is constructing a multi-agency data system for the U.S. Greenhouse Gas Center, aiming to offer visualization and analysis tools for authoritative greenhouse gas data from partners like EPA, NASA, NOAA, and NIST. Their efforts involve enhancing metadata consistency, accommodating unique stewardship needs, employing modern data handling methods, ensuring open access, and implementing a governance framework with automated checks to maintain data integrity and compliance with standards.
HLS @AGU2023
Poster
An Update on NASA’s Harmonized Landsat and Sentinel-2 Project
IN43C-0644: Thursday, December 14, 2023, 14:10–18:30 PST
The HLS project harmonizes Landsat and Sentinel-2 data into interchangeable 30-meter datasets (L30 & S30) to meet the need for more frequent satellite revisit times. NASA initiated the release of global HLS products in 2021, offering access via cloud-based services and direct download from LP DAAC, accumulating a sizable 10-year archive starting from 2013 totaling nearly 4 PB in size. This poster updates the HLS product status, showcases its applications, and outlines future plans and derived products catering to user requirements.
eLightning
Assessment of Atmospheric Correction Methods on Harmonized Landsat and Sentinel-2 (HLS) Products
EP34B-06: Wednesday, December 13, 2023, 16:17–16:20 PST
The HLS project generates global surface reflectance data from Landsat-8/9 and Sentinel-2A/2B satellites with high temporal frequency. It utilizes the LaSRC algorithm for atmospheric correction, currently reliant on MODIS data, but transitioning to VIIRS due to MODIS reaching end-of-life. This presentation assesses the impact of newer LaSRC versions and VIIRS data on HLS surface reflectance, comparing it to alternative atmospheric correction algorithms across diverse test sites globally.
Presentation
The Harmonized Landsat and Sentinel-2 Version 2.0 Surface Reflectance Data Set
IN42B-05: Thursday, December 14, 2023, 11:02–11:12 PST
The Harmonized Landsat and Sentinel-2 (HLS) version 2.0 dataset is now complete, encompassing global land areas excluding Antarctica and utilizing measurements from Landsat OLI and Sentinel-2 MSI. This collaborative project between NASA and USGS ensures 30-meter surface reflectance data comparability by applying algorithms like LaSRC for reflectance derivation, Fmask for cloud detection, and adjustments for spectral and view angle differences. Evaluation using same-day observation pairs from globally distributed sites reveals atmospheric correction’s impact on reducing reflectance differences, highlighting directional characteristics uncovered by this correction, while subsequent BRDF correction effectively minimizes between-sensor differences, and bandpass adjustment has a limited effect.
VEDA @AGU2023
Poster
Advancing Climate Research Through the U.S. Greenhouse Gas Center and VEDA Open Science Platform
SY21C-0824: Tuesday, December 12, 2023, 08:30–12:50 PST
The U.S. Greenhouse Gas Center (GHG Center) emerges as a collaborative hub integrating data from agencies like the EPA, NASA, NIST, and NOAA to enable informed decision making in combating climate change. This initiative emphasizes seamless interagency collaboration, open data accessibility, and user-friendly tools facilitated by the VEDA open-science platform. The GHG Center’s beta version, powered by VEDA, centralizes and curates greenhouse gas data, emphasizing dataset curation, validation, and user-friendly features for effective implementation and strategic decision making.
NASA Kiosk Demo
Visualization, Exploration, and Data Analysis (VEDA): A Pathfinder System to Support Open Science
Demo Kiosk 1: Wednesday, December 13, 2023, 10:00–11:00 PST
VEDA enables discovery, accessibility, and visualization of NASA Earth science data for a broad user community via its visualization and exploration dashboard, the VEDA spatiotemporal asset catalog (STAC), visualization and configuration APIs, as well as an open analytical hub for advanced researchers to perform collaborative analysis and research on cloud-based datasets using cloud computing. All the services provided by VEDA are designed to lower the barrier to entry for science enthusiasts and new researchers, support the transition of legacy workflows to a modernized cloud-based workflow, and improve the efficiency of scientific research.
Presentations
An Open Platform for Cloud-Based, Cloud-Optimized Data Exploration, Visualization, and Analysis
IN11A-02: Monday, December 11, 2023, 08:45–08:55 PST
NASA’s VEDA platform, designed for Earth science data exploration, offers a suite of services like visualization dashboards, JupyterHub for analysis, and a data catalog, supporting various user expertise levels. Utilizing cloud-native tools and adhering to STAC and OGC standards, VEDA facilitates efficient data access via APIs and promotes interoperability with similar platforms. Its modular, open-source design has attracted initiatives like the U.S. Greenhouse Gas Center and IEEE GRSS, showcasing adaptability for specific data needs and aiming to engage a broad user and development community in Earth science exploration.
Lowering the Barrier to Entry for NASA Earth Science Data in the Cloud
ED21A-02: Tuesday, December 12, 2023, 08:43–08:53 PST
NASA’s Earth Science Data Systems (ESDS) face challenges due to growing archive volumes, projected to exceed 600 PB by 2029, further complicated by data migration to the cloud. The VEDA platform aims to simplify user access to this data by abstracting complexities, offering dynamic visualization compatible with GIS services, and providing data stories with Jupyter notebooks for both novice and experienced users. With features like a dashboard for exploration and a JupyterHub for cloud-based execution, VEDA aims to enable educational workshops, support science in the cloud, and promote open science through accessible data exploration and analysis.
Enhancing Data Quality: Leveraging Profiling Tools and Validation Frameworks
IN21A-06: Tuesday, December 12, 2023, 09:20–09:30 PST
NASA’s VEDA platform aims to democratize access to scientific datasets by focusing on cloud-optimized data, intending to simplify access to this new data paradigm. Converting legacy data to cloud-optimized formats, essential for platforms like VEDA, can challenge existing validation workflows, prompting a need for automated validation techniques. The presentation will discuss automated validation methods, employing tools like pyQuARC and Great Expectations, crucial for ensuring data quality and metadata compliance with STAC specifications, supporting data standardization and interoperability among cloud-based platforms like VEDA.
Next-Gen Zarr Web Map Visualization (Invited)
IN42A-01: Thursday, December 14 2023, 10:20–10:30 PST
This presentation demonstrates the importance of visualizing Earth science data in web browsers and highlights challenges due to the scale of geospatial data and the demand for fast rendering. As Zarr gained popularity for large-scale data analysis, users sought browser-based visualization tools, leading to the creation of two options: a dynamic tile server and a direct client. The authors contributed to both approaches, offering insights and testing results in a “Zarr Visualization Cookbook” to help users understand tradeoffs, preprocessing requirements, and performance testing for delivering Zarr data in web browsers, aiming to enhance the adoption of this format for understanding large-scale environmental data.
eoAPI: an open source, configurable and innovative cyberinfrastructure for delivering the SpatioTemporal Asset Catalog (STAC) and data services
INV53A-06: Friday, December 15, 2023, 15:30–15:42 PST
This presentation introduces eoAPI, an open-source, cloud-computing infrastructure focusing on Earth observation data, combining STAC data ingestion, hosting, querying services, raster, and vector services. It powers various applications like NASA’s VEDA Open Science Platform and AWS ASDI’s data catalog, leveraging the STAC specification for data discovery, visualization, machine learning model training, and Earth data science. While numerous open tools facilitate working with EO data at scale, the flexibility of STAC poses challenges in selecting interoperable projects; eoAPI simplifies this by offering a pre-built, opinionated blueprint for a cloud-native EO infrastructure, making it accessible to both STAC-curious and serious users.
VEDA: Empowering Open Science for Earth Observation Data
INV51D-13: Friday, December 15, 2023, 16:00–16:25 PST
NASA’s VEDA platform embodies open science by promoting inclusivity and accessibility in scientific research through its commitment to openness and integration of community-standard open-source software. VEDA’s innovative components, like STAC storage, OGC-standard dynamic tiling, and a customizable dashboard, enhance data discovery and analytics, while its cloud-native approach ensures efficient data access and exploration. The presentation aims to demonstrate the seamless transition from the dashboard to the JupyterHub environment, showcasing how researchers can engage in open science by exploring, visualizing, and analyzing VEDA datasets through accessible APIs, highlighting VEDA’s versatility beyond its core platform.
