NASA's Earth Science Data and Information System (ESDIS) Project manages the science systems of NASA's Earth Observing System Data and Information System (EOSDIS), which provides NASA Earth science data to a wide community of users. Among its responsibilities are ensuring that scientists and the public have access to data to facilitate the study of Earth from space and promoting the interdisciplinary use of EOSDIS data products to a broad range of existing and potential user communities.
Two of ESDIS’s most effective approaches for meeting this responsibility are NASA’s Global Imagery Browse Services (GIBS) and NASA Worldview, an application developed by ESDIS that lets users interactively browse a wide variety of global, full-resolution satellite imagery layers and download the underlying data.
Ryan Boller, the Data Visualization Lead for the ESDIS Project, has been involved with Worldview’s development from the start, and throughout his career he’s helped it grow from offering a half-dozen data products to the more than 1,000 it has today. In addition, Boller is also working to bring Worldview into the future by developing processes that will allow users to dynamically generate custom imagery and use artificial intelligence to mine imagery archives.
In the following interview, Boller discusses his career path, his work with the ESDIS Project, Worldview’s origins and growth, and how ongoing advances in technology will lead to new developments and greater user customization.
Let’s begin by talking about your background. What did you study, or what technical skills did you develop, and how did you find your way to NASA?
I’ve always enjoyed taking things apart and sometimes putting them back together to see how they work, so deciding to become an engineer was an easy choice. The tougher decision was figuring out what kind of engineer to be. By the time I got to college, I had done my fair share of tinkering with electronics and software, so I decided to study computer science.
Once in school, I was drawn toward the visual side of the curriculum. This was in an era where the first full-length movies generated entirely by computer graphics were released. The capabilities of desktop computers to render graphics in real time were also just being unlocked. So, I took every computer graphics and vision course I could find, and found fun internships in those areas.
When it was time to graduate, I knew I wanted to stay in these technical fields, but I had to decide where and how to apply my knowledge and skills. Medicine? Aerospace? Entertainment? As I was learning more about aerospace-related careers, the NASA mission was irresistible. Who wouldn’t want to study Earth, the Sun, and the universe? So, I applied in every way I could, and after six months they finally decided to offer me a job.
How did you get into your current position, Data Visualization Lead?
It’s been a long road—over 20 years at this point! I started with a project to reengineer a science data processing system for solar physics missions. I learned a lot from that project but was still keen to pursue the field of data visualization, which combined my interests of computer graphics and science data analysis. Thanks to my supportive management, they found a few wonderful space scientists who were keen to build interactive interfaces to better understand their data. I jumped at these opportunities to build my data visualization skillset and learn about the science along the way.
After several years, I wanted to learn more about and contribute to NASA’s Earth science missions, too. So, I made more requests of my managers and eventually began working with Goddard Space Flight Center’s Scientific Visualization Studio (SVS) to create high-quality animations of Earth’s cryosphere for use in public outreach campaigns. As it turns out, the ESDIS Project was the customer who requested those animations and, six years later, I began working for them. In any case, this was another great experience with great people where I learned more about the art of crafting scientific visualizations for public consumption rather than building interactive interfaces for scientists. It was also at this point where I started to realize that I still really loved data visualization, and I had a lot to learn about the field. So, I went back to school for a master’s degree.
I found a great program and advisor where I was able to study computer science, data visualization, and remote sensing and, as part of my studies, I partnered with a NASA scientist on building a visualization tool to study hurricane formation. After I graduated, I returned to Goddard full-time. I knew I wanted to continue down the path of Earth science data visualization, and after a somewhat coincidental business trip with an ESDIS architect, he recruited me to help with a new project to visualize all EOSDIS data holdings beginning with the near real-time (NRT) data products. That person was Kevin Murphy, who is now the Chief Science Data Officer for NASA’s Science Mission Directorate.
What is your chief task as Data Visualization Lead?
I see myself as a bridge between data scientists and Earth scientists, especially in terms of how data visualization can support the scientific discovery process. So, my challenge is to understand what kind of data visualization technologies are available and match them with the needs of an Earth scientist as part of his or her workflow. I generally find this to be both intimidating and rewarding. It’s intimidating in a sense that, I’m not an Earth scientist and probably won’t be able to fully understand their needs without being one myself. But it’s also very rewarding when I’m able to work with my engineering colleagues to understand the needs of Earth scientists enough to build a system that enables new discoveries that may have been difficult to achieve otherwise.
Tell me about GIBS and Worldview. How did it start? How did we get to where we are now? And what kinds of growing pains were encountered along the way?
As mentioned earlier, I was recruited by Kevin Murphy to help with the goal of eventually visualizing all of the holdings of EOSDIS. The initial idea was to start small and focus on the NRT data by partnering with the Land, Atmosphere Near real-time Capability for EOS (LANCE). They already had great success with the Moderate Resolution Imaging Spectroradiometer (MODIS) Rapid Response system by providing quick turnaround imagery from NASA’s Terra and Aqua satellites. One of its primary limitations, though, was that it wasn’t interactive. Users clicked on one of many squares on a global map and got an image. At the time (2011), the Google Maps model of panning and zooming had been firmly established as a successful approach to browsing map imagery, so we began looking at how to build something similar, although ours would have rapidly updated imagery of Earth from NASA’s satellite missions.
We found excellent partners at NASA’s Jet Propulsion Laboratory (JPL), as they had already been building interactive mapping systems for the Moon, Mars, and Earth. So, the core of our tiled mapping system was and still is based on that technology, which allows us to store and serve vast amounts of NASA satellite imagery efficiently. This portion of the project soon became known as the Global Imagery Browse Services (GIBS). At the same time, a small team at Goddard began building a custom interface—known then as State of the Earth—that let us browse all of that imagery. We changed the interface’s name to Worldview after internal polling and began making some small improvements. Initially, the interface allowed users to browse about a half-dozen products from the past week on a rolling basis. Imagery was available within three hours of observation by the spacecraft and users could pan and zoom the map, as well as change the displayed date. Our initial release was demonstrated at the 2011 Fall Meeting of the American Geophysical Union.
Things really took off from there in terms of using the imagery for NRT applications such as fire and smoke monitoring. We then started to get a lot of helpful user suggestions. Many took the form of, “This is great but can you also visualize ___ products?” and “I want to go further back in time. Can you add historical imagery?”
In fact, one of the main growing pains became how do we scale the system to respond to user requests? Fast forward to today and that initial collaboration between teams at Goddard and JPL is still going strong. In general, Goddard is responsible for managing and operating GIBS while also developing Worldview. JPL develops new technologies that enable the overall project to innovate and scale. And at this point, we’re providing visualizations for more than 1,000 data products, up from the initial half-dozen. But there were many challenges to overcome in the intervening 10-plus years.
What is the most challenging aspect of visualizing data in GIBS/Worldview?
As previously mentioned, the most challenging aspect has been how to scale the system to support the amount of data that users want to visualize. Some of this is “easy”—just buy more storage—and our system administrators and operators can make it look that way, even though it’s not. There’s also the fact that EOSDIS data are very heterogeneous. This heterogeneity is what makes the EOSDIS data collection such a rich archive, but it also makes it difficult to scale since the products can be so different from one another.
Then there’s the challenge of user-friendliness. Even if the system can store and serve a very large number of data products, how do you present users with an interface that is easy to navigate, maybe even fun to use, and doesn’t overwhelm them?
I’m always taken aback by how quickly things change in technology sector. Can you talk about how this change has affected your work? How do you stay current with new developments?
Rapid change in technology can make things fun, interesting, frustrating, and perilous. I’m humbled by and grateful for how much my team has been able to accomplish over this 10-plus year span. I’ve had the privilege of working with so many talented people who are able to keep up with the new opportunities that technology development brings and apply them to our project. We’re also a part of many excellent communities—EOSDIS as a whole, the Interagency Implementation and Advanced Concepts Team (IMPACT), and the science teams associated with them—who collectively keep tabs on where things are headed and work together to devise new approaches. It’s also been helpful to be part of the open source software community where we’re able to leverage others’ advancements and hopefully vice versa.
Where do you see Worldview going in the future?
As data volumes for new missions continue to explode in size, and as users continue to request more products, I think the main answer is there will be “more” in Worldview/GIBS. More products, more visualization types, and more flexibility in how data are visualized.
A currently-in-progress migration to cloud computing is a big part of this plan. The cloud allows us to scale our storage accordingly, while also providing flexibility to dynamically visualize data from cloud-optimized formats. In other words, rather than taking our current approach to pre-generate imagery, users will be able to dynamically generate custom imagery that better meets their needs. And by taking a similar approach, we’ll also be able to dynamically generate custom analyses, such as time-series charts, from cloud-optimized data stores.
Beyond that, we’re looking forward to incorporating a lot of technology advancements in the field such as vector flow visualization, 3D visualization, and using artificial intelligence to mine imagery archives.
If you were talking to a new visualization engineer, what would you recommend that he or she focus on to develop his or her career?
Oof! I’ve been in this field for so long that it’s difficult to think about starting points! For me, it’s been a relatively continuous progression of finding ways to combine my interests of visual design and engineering with science and finding the people and projects that allow me to develop those skills further. To that end, it’s been most rewarding to partner and chat with end users and scientists to try and better understand their needs and propose how new technology might help them better succeed. Even if you don’t understand the science at first, it can be illuminating to just observe their processes and ask questions. Soon enough, you’ll start to recognize where and how your technology background can fill gaps in their processes. There’s always a lot to do, but over time your work can become an invaluable part of their research and discoveries!
What makes your job fun? What is the coolest thing that you have done with Worldview, or what is something you’ve done that you’re particularly proud of?
It’s always rewarding to see what our users do with the system. GIBS/Worldview is designed to be a general-purpose system and people have found a lot of purposes! Beyond the science and NRT applications, it’s been used in education, people have made high fashion clothing out of the imagery, checked if their favorite remote lake is still frozen, etc.
One of my favorite, semi-recurring fun project-related activities is visiting Hayden Planetarium at the American Museum of Natural History in New York. They regularly put on shows which pipe Earth imagery from GIBS into their planetarium software, add some atmospheric effects (we generally take out the atmospheric effects to provide clearer imagery), and project it into their massive dome. Their Director of Astrovisualization narrates the audience through a tour of the universe and our dynamic home planet, showing off phenomena like massive dust storms, hurricanes, and Earth at night. One former astronaut who saw the show commented, “That’s the closest I’ve been to being in space since being in space.”