Cloud Computing and Open Science Spur Evolution of Field Campaign Explorer Tool

For Dr. Geoffrey Stano, chief scientist at NASA's Global Hydrometeorology Resource Center Distributed Active Archive Center (GHRC DAAC), the advantage of field campaigns like the Investigation of Microphysics and Precipitation for Atlantic Coast-Threatening Snowstorms (IMPACTS) and the Hurricane and Severe Storm Sentinel (HS3) is that even though they’re only for a very specific time, they bring a lot of capabilities to bear to investigate a particular issue.

He's not wrong. The IMPACTS field campaign, for example, enlisted NASA ER-2 and P-3 research aircraft outfitted with a suite of instruments, including advanced radar, lidar, a microwave radiometer, microphysics probes, and dropsondes. In addition, the data from these instruments were augmented by data from ground-based radar and rawinsondes, multiple NASA and NOAA satellites, and computer simulations. 

Yet, Stano acknowledges that taking advantage of these varied datasets may be a challenge for some users.

"If you're interested in these data but not part of a particular science team, you may not be familiar with the instruments providing all these data. So the question becomes, 'How do I start visualizing and analyzing these datasets?'" he said. "In addition, many of these datasets are three-dimensional datasets, and that adds a layer of complexity when trying to visualize them."

The volume of data field campaigns can generate is another potential issue.

"Although field campaigns aren't producing amounts of data on the order of the Surface Water and Ocean Topography (SWOT) mission or [expected data from the upcoming] NASA-Indian Space Research Organisation (ISRO) Synthetic Aperture Radar (NISAR) mission, we're getting to the point that, if you wanted to process everything, downloading all the data might be difficult for users working with their own computer systems," Stano said.

Given these challenges, data from field campaigns are often underused. To remedy this, the GHRC DAAC developed the Field Campaign Explorer (FCX) data tool to help scientists discover, collect, visualize, and analyze field campaign datasets more rapidly.

The first iteration of FCX was developed in 2014 for the HS3 field campaign. That tool evolved into the Visualization for Integrated Satellite, Airborne, and Ground-based data Exploration (VISAGE) project, which sought to provide visualization and analytic capabilities for diverse coincident datasets with a focus on airborne field campaigns. As GHRC began to move the NASA data it archives and distributes into the cloud, the DAAC developed a new, cloud-based tool that incorporated the lessons learned from the use of VISAGE. Because this new tool was primarily used to visualize data from field campaigns, the FCX acronym was reused. This is the FCX tool currently available on the GHRC DAAC website.

At present, FCX requires users searching for field campaign data to first select a field campaign of interest and then choose an instrument used during that campaign before they can begin visualizing and analyzing data. This will change in the next version of the tool. 

"We want [FCX] to evolve from the focus on individual field campaigns and transform it into a cloud-based three-dimensional visualization tool that lets users select the instrument they're interested in and see datasets from multiple campaigns simultaneously," Stano said.

According to Stano, this shift in focus from campaigns to instruments was encouraged by the DAAC's User Working Group (UWG), suggestions from users, and the realization that field campaign datasets comprise a major component of the GHRC DAAC's holdings.

"In discussions with users and members of the UWG, it was stated that many instruments are deployed across multiple field campaigns," he said. "Instead of having a user search for each campaign in which an instrument was deployed, users expressed an interest in identifying an instrument and then pulling up all of its deployments."

There are some technical challenges to achieving this instrument-focused approach, but Stano believes working to address them is worth it, as the creation of a more expansive and flexible version of the FCX tool will offer significant benefits to the user community. 

"FCX, or any eventual follow-on, has always been envisioned as a cloud-based, open-source system that would enable users to implement FCX without downloading anything and leverage the work of NASA's Earth Science Data and Information System (ESDIS) Project to move [all of the] DAACs' data holdings into the cloud," he said. "We're trying to move away from each DAAC creating their own data visualization tool; we want to make it more of an enterprise-level system."

Further, such a system would support both the ESDIS Project's goal of ensuring that the majority of NASA's Earth science data work in its enterprise data tools and NASA's commitment to open science by contributing to, building on, and using open-source tools and services.

"In addition to giving users the ability to visualize multiple datasets in 3D, subset, and download data, they would have the ability to create their own plug-ins," said Stano. "We could also leverage NASA Openscapes via earthaccess to create standalone visualization data recipes in Jupyter Notebooks and reduce the amount of technical expertise required to work with NASA's Earth science data."

VIDEO PLACEHOLDER

The GHRC team will continue to update FCX and begin to emphasize its capabilities beyond just field campaign datasets. Additionally, as part of the ESDIS focus on enterprise-level capabilities, the FCX team is identifying components of FCX that can be adapted to support the NASA Earth Science Data System (ESDS) Program's Harmony application. Harmony allows users to produce analysis-ready data by subsetting, reprojecting, and converting data to a cloud-optimized format, thereby saving them the time, energy, and cost associated with moving large files. 

"When it's all said and done, GHRC's efforts are providing users with a system that can do coincident, three-dimensional visualization and eventual data analysis," Stano said. "Users won’t have to be experts on all the instruments and figure out how to visualize the data from them."