To achieve global coverage, the researchers developed a new data management procedure that combined data from six receiving stations, eliminated overlapping segments and improved data quality in the process by replacing dropped and bad scan lines with sound data, while reducing the number of data units to be handled. Eidenshink calls the method "orbital stitching."
"It was a megasystem," he said. "We got to the point where we can process one day's data every day." Along the way, the project succeeded in developing processing standards for calibration, atmospheric correction, and geometric registration of the AVHRR data. Collaborating and receiving science guidance was key, Eidenshink said.
"From the beginning, LP DAAC was necessary," Eidenshink said. "DAAC funds were used to acquire the data, Pathfinder funds were used to pay for processing, the DAAC distributed the data and products to the science community. The DAAC science advisory team worked to keep momentum on the project and helped keep linkages between AVHRR and Terra," Eidenshink said.
As the value of the Pathfinder program began to be felt within the scientific community, a 1994 NASA Research Announcement (NRA) re-competed the 1990 product generation awards, and solicited proposals for eight new Pathfinder efforts that would strengthen and extend existing product generation. Among proposals awarded funding were projects to produce 1.25 and 5 km AVHRR grids over the North and South Poles.
Here, the invaluable connections established by Eidenshink's product generation team fed the polar effort. Acting as a clearing house for data from Fairbanks and Tromso, Eidenshink's team staged those data along with data streams from Antarctic stations to NASA's National Snow and Ice Data Center (NSIDC) DAAC in Boulder, perfecting an automated data transfer system in the process.
Ted Scambos, a research scientist with the National Snow and Ice Data Center, and co-PI with James Maslanik on the AVHRR Polar Pathfinders, said "We saw a need for establishing as long-term a baseline as possible indicator of conditions of the snow pack, such as temperature and albedo. Such a baseline had already been established for the passive microwave data sets, with a nice record going back 25 years, but at much coarser resolution. That kind of record doesn't exist in the visible and thermal channels," he said.
The Polar Pathfinders proposed development of a detailed calibrated record of temperature, ice motion, and albedo in the polar regions, extending from the early 1980s.
"We learned a lot going through the process," Scambos said. "Initially we had conservative objectives in terms of how much data we were going to be able to process. Four years ago it seemed a much more daunting task to handle terabytes of data than it does now."
Other improvements included refining the processing algorithm, Scambos said. "We got to know the data better and learned where the weak spots were in data product generation. For instance, telling clouds apart from the ice surface was always a problem spot. It's very difficult to come up with good cloud algorithms over snow and ice. When you've got liquid water clouds over snow and ice, the problem is not severe, but with ice-particle clouds over a dry snowy surface, the distinction becomes difficult. A cold snowy surface and a cirrus cloud are very similar in just about every respect.
"We've made strides on that mostly by looking for day-to-day surface changes. On a daily basis, the true surface doesn't change much, but of course clouds pass by with an attendant big change in texture, temperature, and albedo, so we can eliminate pixels on that basis.