NASA’s Earth Science Data and Information System (ESDIS) Project manages the science systems of NASA's Earth Observing System Data and Information System (EOSDIS), which provides NASA Earth science data to a wide community of users around the globe. In that capacity, the project is responsible for processing, archiving, and distributing Earth science data products, providing tools to facilitate these efforts, improving services for users, ensuring scientists and the public have access to NASA data, and promoting the interdisciplinary use of EOSDIS data to a range of existing and potential user communities.
Karen Michael has been involved in this mission since 1995, and, as she put it, her career just “evolved over time.” Today, Michael is a System Manager of EOSDIS, a position in which she works to define the responsibilities of flight projects when sending their data to NASA’s Distributed Active Archive Centers (DAACs), and manager of NASA’s Land, Atmosphere Near real-time Capability for EOS (LANCE), a position in which she works to enhance LANCE's reliability, reduce data latency, and make more users aware of the availability of near real-time (NRT) data.
In the following interview, Michael discusses her career path, her work with EOSDIS and LANCE, the common aim of her main roles, and how, after 27 years, she still finds ESDIS to be a project in which she can work and grow.
I'd like to begin by asking about your career path. How did you come to work for ESDIS?
My route to working at NASA was a little unusual. In my first job after graduation, I worked as a chemist in a materials science laboratory. While working there full time, I went back to school part time and got a degree in electrical engineering, and I became interested in robotics. While in the lab, I worked on several NASA contracts, one of which involved making superconductors. I wanted to work for NASA in robotics though, and when the opportunity arose I applied for an engineering position with the agency. I was hired as a civil servant and as there were no positions available in robotics, my first job was designing hardware for a lossless data compression algorithm (i.e., a method of data compression without the loss of information). While working in the hardware lab at NASA, I went back to school at night and got a degree in engineering management.
Sometime later, a coworker who had gone to work for ESDIS announced he was leaving his job and he thought I might be interested in it, so I applied, got it, and began working in integrating and testing for ESDIS for the then upcoming TRMM mission. I really enjoyed working in the ESDIS Project and felt it was a good opportunity for me to grow. I started working with interfaces, documents, and projects and it just evolved over time. I began managing some of the tests for upcoming launches, such as for NASA’s Aqua satellite and the NASA-USGS Landsat 7 satellite. Later, when the system management position became available, I applied for that and got it. I’ve been in ESDIS for about 27 years now and I still find it an excellent project to grow in. The work keeps coming with each new mission and there’s always something new.
You fulfill two major roles: EOSDIS System Manager and LANCE Manager. What does each position entail and what are your main areas of focus?
As system manager, I work with the flight projects, whose data we serve through the DAACs. Basically, I’m the first person in ESDIS to interface with the flight projects and help define what their roles and responsibilities are for providing data to the DAACs.
I also manage NASA’s LANCE, which is our near real-time data system. In most cases, the flight projects are generating standard (or high-level, science-quality) products, which go to the DAACs. The LANCE is a distributed system that produces near real-time products, which are available prior to the standard products and are not archived. In fact, we want users to take the standard products when they’re available. So, the application of LANCE data is much different.
In regard to your role as EOSDIS System Manager, what is the most difficult aspect of coordinating configuration management on a project like ESDIS?
Configuration management (CM) is very important. CM is how we manage changes to requirements, design, and interfaces in order to maintain integrity. Creating and reviewing documents, however, is not something that scientists and engineers generally like to do, so getting them to do it in a timely manner is probably the biggest challenge. In addition, ESDIS is a big project with a long history, so another challenge is just the sheer number of documents we have and the number of accounts we have to manage. Just to give you some idea, I looked into NASA's Configuration Management EOSDIS Tool (COMET) yesterday, which is our content management system, and found that we have close to 200 documents under configuration management. We also have more than 500 users of the system, and looking back to historical documents, we have close to 1,500 that were under CM.
What are the critical steps when coordinating a NASA mission and parts of the EOSDIS data systems, for example the DAACs?
Data from these missions are archived by the DAACs and the selection of which discipline-specific DAAC will archive the data is usually based on the mission’s scientific goals. So, I work with the flight projects to help them understand what their roles and responsibilities are for providing data to the DAACs, and then we get into the interface control document, which defines the protocol and the details of the data exchange. Standardizing the interfaces is really key to keeping the cost and complexity down, so what we try to do is maintain very similar interfaces. We have the DAACs standardized in the way that they ingest and archive and distribute, and in the same manner, the Global Imagery Browse Services (GIBS), which is our imagery browsing system has a standard type of interface. The way that metadata, which are what make data searchable and allow for the efficient management of large data collections, is provided is also very standardized.
Switching gears now to your work as LANCE Manager, what would you say is LANCE's most important objective? Is it making more NRT data available? Reducing latency? System reliability? Or perhaps making more users aware that NRT data are available to them?
I think all those things are important, but if I had to pick one, it would be reducing latency, because that’s the main reason LANCE exists and has become so popular. The data from LANCE has really enabled NASA Worldview and NASA’s Fire Information for Resource Management System (FIRMS), which provides near real-time active fire data to the wildland fire community, to provide imagery in a timely manner to decision-makers all over the world. Worldview and FIRMS are among the top NASA sites visited every day, and for good reason. I think users really like the responsiveness of the low latency data. When you get on Worldview that’s the first thing you see—the imagery that’s just been collected—and the beauty of it is that you don’t need to be a scientist to use Worldview and FIRMS.
Can you explain what makes LANCE data different from the standard products archived at the DAACs?
Standard products are stored at the DAACs and they’re for scientific research and the long-term climate record. LANCE NRT products are used for short-term weather prediction or by people like wildland firefighters and first responders to hazards and disasters who don’t have time to wait for standard products to be processed. It’s the timeliness of the data that they need. But producing products quicker means they’re not as good as the standard products, and so things like the geolocation may be slightly off, because we’re using things like predicted ephemeris (i.e., data that provide the assigned places of a celestial body or human-made satellite at regular intervals) and not waiting for ancillary data that’s also used in standard processing. In a lot of cases, standard processing involves waiting for other sources of data to come in and be available before the processing can start, whereas with LANCE we bypass that step. [Note: Read an in-depth explanation of NRT versus standard products.]
What do you see for the future of NRT data and LANCE?
For a while, near real-time or low latency data wasn’t given much attention at NASA, but it’s starting to be something that future missions are looking at. Maybe it’s not their first objective, but as a second objective. Managing LANCE is something that’s near and dear to my heart. I’ve been with it since its inception and I’ve seen the impact that it’s had. I really hope to see LANCE continue to grow in the future. I think it’s going to be a bit more challenging in that some of the upcoming missions really don’t have NRT requirements. Getting those requests in later is always harder, that’s the lesson we’ve learned from the past, but we’re still seeing it happen because science objectives are what drive missions and standard products are usually the first priority. I have been involved in latency studies for some upcoming missions and they are considering some NRT applications so I’m hoping to see LANCE continue to evolve.
Does NASA keep statistics about how many people accessing NRT data each day?
We do collect those metrics. We collect a lot of metrics about the data we distribute and who we distribute it to. For LANCE there are close to 300 image and data products being distributed at a volume of 40-75 Terabytes per week to users in more than 200 countries. We also track our latency to make sure we’re staying below the three hours or less benchmark for LANCE.
How do user requests come to you? Do they come to you through the LANCE User Working Group or are there other mechanisms for that?
We have two primary mechanisms for that. One is a survey that we conduct about every two years that goes out to LANCE users and the community. It asks these kinds of questions: What data do they use? How do they use it? and What would they like to see provided in the future? We also receive requests through the LANCE User Working Group (UWG), and we take requests from any user who has an application that’s not being filled or who has an idea. They fill out a LANCE enhancement request and we review it and figure out if and how it can be implemented and what it would cost to do it. Then we go back to the ESDIS Project and see if we have the funding. If we don’t, we go to [NASA] Headquarters and request funding. If headquarters approves it, then we go forward. That’s how we’ve been operating in LANCE. It’s a little different from the rest of EOSDIS, because LANCE itself doesn’t really have a budget that allows me to say, this is what we’re going to do this year or next year. It’s based on requests from users and the community.
How frequently do you receive such requests and are they for products that are attainable?
Our UWG meets twice a year, so it could be two times per year. Sometimes we don’t get any requests in the spring and we get three requests in the fall, so maybe on the order of one or two a year.
Typically, it’s been requests for something we have a standard product for, but someone wants it in NRT. Basically, it’s the same standard algorithm, but with modifications required to make it run in NRT. Every once in a while, it might be another source of data that we haven’t collected before. So now, for example, in FIRMS we’re getting direct readout data. It may be the same data that we’re bringing down normally, but in this case we’re getting it even faster, so the request is made for more timely data, such as less than an hour instead of three hours.
So, given the purview of your two roles, is it fair to say they’re both focused on ensuring users have access to the data they need?
Yes, I would say that for sure. We’re serving users. That’s what we’re here to do and I think that’s common to both the standard products and the NRT products available through LANCE. We’re always listening to what users need and trying to improve on what we provide in regard to the formats users want the data to be in, providing the requested imagery, making it easier to access and use NASA Earth science data, and so on.