Record Setting Satisfaction
NASA’s EOSDIS received an all-time high overall score on its 2018 American Customer Satisfaction Index (ACSI) survey.
Josh Blumenfeld, EOSDIS Science Writer
For the 15th consecutive year, NASA’s Earth Observing System Data and Information System (EOSDIS) received high scores for customer satisfaction in the annual American Customer Satisfaction Index (ACSI) survey. In addition, all EOSDIS Distributed Active Archive Centers (DAACs) received strong individual evaluations.
EOSDIS received an aggregate Customer Satisfaction Index (CSI) of 79 out of 100, which is the highest CSI ever received by EOSDIS and an indication of strong, consistent performance (and one point higher than last year’s score). The EOSDIS CSI score is also more than 10 points higher than the aggregate federal government 2018 CSI. In fact, EOSDIS has maintained an average aggregate CSI of 77 over the past 12 years and has received a CSI at or above 74 every year the EOSDIS ACSI survey has been conducted.
These consistently strong scores represent overall satisfaction with EOSDIS and DAAC products and services by the global EOSDIS data user community. More importantly, the annual ACSI survey provides a roadmap for NASA's Earth Science Data and Information System (ESDIS) Project (which manages EOSDIS science operations, including data archiving and distribution) and the DAACs on ways these products and services can further be improved or enhanced along with insight into emerging data user needs.
Before looking at some of the high-end results of the 2018 survey, let’s first look at the survey background and how it is conducted.
The ACSI survey is administered by the CFI Group, an independent organization that is under contract to the federal government to assess user satisfaction with products and services. Along with the EOSDIS, the CFI Group conducts annual surveys for federal entities that include the National Weather Service, the General Services Administration, and the Department of Education.
The ACSI model is a set of causal equations linking customer expectations, perceived quality, and perceived value to customer satisfaction, which is reflected mathematically in the CSI score. Satisfaction, in turn, is further linked to a customer’s likelihood to recommend products and services and their willingness to use products and services in the future. Because the CSI score is based on aggregate weighted scores in several factors that are calculated using the proprietary ACSI methodology, a score in the upper-70s can be considered an indication of “strong” performance.
One expected result of high customer satisfaction with services is user trust and loyalty. This is accounted for in the ACSI algorithm and reflected by a number indicating the likelihood of a respondent to recommend the evaluated products and services to others coupled with the likelihood of a respondent to use the services in the future. Respondents’ likelihood to recommend EOSDIS products and services (88, one point higher than in 2017) and likelihood to use EOSDIS services in the future (89, the same record-high score as in 2017) remain very strong, and are consistent with previous EOSDIS ACSI surveys.
The 2018 survey was conducted online between October 1 and October 23. The CFI Group emailed roughly 285,698 survey invitations to individuals who used EOSDIS data and/or products in the past year and received 2,778 completed surveys—a response rate close to 1%. While this response rate is lower than previous surveys, it is still a large enough sample size for calculating a statistically valid CSI score, according to the CFI Group.
Outside of three survey questions asking respondents to rate their overall expectations, perceived quality, and perceived value of EOSDIS services and products on a 1 to 10 scale (which are required by the CFI Group and used to calculate the CSI score), the remaining questions on the EOSDIS survey are developed collaboratively by the ESDIS Project, the DAACs, and the CFI Group. Respondents to the survey are asked to evaluate their experience with the specific DAAC or DAACs from which they receive data and products. In addition, the survey allows respondents to provide open-ended comments. These are some of the most valuable areas of the survey since they allow respondents to candidly express their specific likes, dislikes, satisfactions, and suggested improvements.
The CFI Group breaks EOSDIS services into six “Satisfaction Drivers” representing specific areas of performance affecting overall EOSDIS customer satisfaction. Each satisfaction driver is further evaluated by its weighted Impact on Satisfaction, which ranges from 0.0 to 1.5 and indicates the leverage an individual driver has on customer satisfaction and the area or areas in which improvements matter the most to survey respondents.
The driver with the greatest impact on EOSDIS customer satisfaction continues to be Product Quality, which has an impact value of 1.1. The Product Quality score of 85 (received in both 2018 and 2017) is the highest Product Quality score received since the EOSDIS survey was first conducted in 2004. As noted by the CFI Group, maintaining a high Product Quality score is essential to maintaining a high CSI score due to the high impact of Product Quality on overall satisfaction.
Product Documentation also has an impact value of 1.1, indicating this driver has a high impact on the overall CSI score. However, the combination of the high impact (1.1) and lower score (81) in this driver makes Product Documentation a priority area for improvement, according to the CFI Group. While 73% of survey respondents looked for or obtained documentation, they indicated difficulty finding documentation dealing with general data conversion as well as documents dealing with specific DAACs. Improvements in documentation clarity and in documentation to aid in data use are two recommendations by the CFI Group to help raise the score in this area.
Overall, EOSDIS Satisfaction Driver Scores remained remarkably consistent from 2017, with no change in the scores for Product Quality (85), Product Selection & Order (83), and Product Delivery (84). The Product Documentation score increased one point to 81, while Customer Support and Product Search both decreased one point from 2017.
Customer Support (86) remains the highest scoring satisfaction driver in the EOSDIS satisfaction model. Respondents noted that customer support personnel are professional, knowledgeable, quick, and helpful.
Now, let’s dig a bit more deeply into some of the specific findings of the 2018 ACSI survey. In the following summary tables, total percentages may not equal 100% due to survey questions allowing for multiple responses; all non-percentage values are out of 100.
Survey respondents self-identifying themselves as university students (39%)—which includes both undergraduate (10%) and graduate (29%) students—and Earth science researchers (34%) were the most common EOSDIS data users, followed by university professors (16%) and the general public (13%). These percentages are consistent with previous surveys. The disciplines for which EOSDIS data are used also are consistent with previous years. While Land decreased from 67% in 2017 to 61%, the other disciplines noted by survey respondents remained either unchanged or within 1% of 2017 values. Note that the disciplines of Hydrology, Natural Hazards, and Emergency Planning/Management were broken out separately in the 2018 survey, but were not included in previous year’s surveys.
ACSI survey respondents are asked to evaluate their experience with the specific DAAC or DAACs from which they receive data and products. For links to the individual DAACs mentioned below, please see the DAAC page on the Earthdata website.
NASA's Land Processes DAAC (LP DAAC) continues to be the most frequent DAAC evaluated (30%). Since LP DAAC archives and distributes NASA Earth observing data related to land-oriented disciplines (including land cover, topography, and vegetation indices), the high frequency of LP DAAC evaluation is in line with the high level of use of EOSDIS data in land-oriented disciplines. The 2018 LP DAAC evaluation frequency, though, is 8% lower than in 2017.
CSI scores were computed for each DAAC based on individual DAAC survey responses. All DAACs achieved CSIs between 73 and 85, which is higher than the 2017 survey and considered strong scores by the CFI Group. Among DAACs with at least 50 evaluations in the 2018 survey, NASA's Alaska Satellite Facility DAAC (ASF DAAC) and Physical Oceanography DAAC (PO.DAAC) both saw two-point CSI increases. NASA's Atmospheric Science Data Center (ASDC) saw the largest change in CSI, with a three-point drop from their 2017 score (77 to 74).
Note that while NASA's Ocean Biology DAAC (OB.DAAC) saw the largest change in CSI score between 2017 and 2018 (+9 points), it also had the fewest evaluations in the 2018 survey (35). Two other DAACs also had fewer than 50 evaluations in 2018: NASA's Oak Ridge National Laboratory DAAC (ORNL DAAC) 45 evaluations; and NASA's Crustal Dynamics Data Information System (CDDIS), 37 evaluations.
Summaries of all EOSDIS ACSI reports are available on the ACSI Reports page in the Performance section of the Earthdata website, and full reports are available upon request. Thanks to your participation, the evaluations and comments from the 2018 EOSDIS ACSI survey are being incorporated into enhancements you will see throughout 2019 to further increase the usefulness and quality of EOSDIS data and services. Be sure to check your email this fall for an invitation to participate in the 2019 survey—your opinion matters and will help make NASA Earth observing data and services even better.
Last Updated: Aug 6, 2019 at 2:26 PM EDT