Home › Review comment 8
Printer-friendly version Add This

Review comment 8

Review of netCDF version 3 implementation and operational suitability

NASA's Earth Science Data Systems Standards Process Group (SPG) is considering the network Common Data Form version 3 (netCDF classic), for adoption as a community standard. You are invited to review this Requests For Comment (RFC) in the context of your implementation experience with this data format specification and its suitability for operational use. Only answer questions applicable to your experience. Please send completed review to: spg-rfc-011@lists.nasa.gov.

Implementation Experience questions:

  1. (Your background) Describe in a sentence or two your overall implementation experience related to the proposed specification. (e.g., specification implementer, tools developer, data provider, scientific analyst, science user, etc.) Have you directly implemented the netCDF classic format specification or modified a netCDF classic library using the specification? Did you use pre-existing software, and if so, what did you use?

    I supervise programmers that work on software to convert HDF-EOS to NetCDF/CF-1 format. We used the netCDF library without modification.

  2. (Completeness) Does the specification provide all the detail you need to implement it in software? (e.g., to read or write a data file; to implement or modify the library, a profile or extension; or develop a tool such as a format translator) If not, describe what is missing in the specification.

    N/A.

  3. (Accuracy) Do any parts of the specification contain inaccuracies, or internal inconsistencies? If so, please provide details.

    N/A

  4. (Clarity) Is any part of the specification ambiguous, or poorly explained? If so, please provide details.

    N/A

  5. (Balance) Does the standard describe the right set of concepts and data types, and enable the appropriate data operations for its intended users? Is this set of concepts and data types an overly broad set (requiring excessive complexity) or narrowly simplistic set?

    Yes, the standard does describe the right concepts and data types for our intended users.

  6. (Usefulness) How well does this specification meet your information sharing needs? (e.g., does it work well with the data types and data manipulations in your application? Does it properly represent your datasets? What are the pros and cons of this data format?)

    Level 3 gridded data are fairly straightforward to represent in netCDF. Although difficult, we eventually worked out how to represent HDF-EOS swath data in netCDF/CF1 format. This made converted AIRS L2 swath data directly usable in certain tools that are netCDF/CF1 aware, such as IDV and McIDAS-V.

  7. (Implementation) What implementation challenges does the proposed standard present? (e.g., does it require advanced processing power, large amounts of memory, complex configuration, etc.? Does it scale to a production environment?)

    Unfortunately, not all netCDF analysis tools are fully capable of understanding the netCDF/CF1 constructs for swath data.

  8. (Flexibility) In what software environment(s) have you used netCDF classic (e.g., Solaris, Linux, Windows, Mac OS X)? Have you implemented, tested or deployed netCDF classic or packages other than those provided by the original netCDF classic and developers?

    Linux

Operational Suitability questions:

  1. Do you currently use or plan to use netCDF classic in a production setting? What types of applications do you use with netCDF classic? Is netCDF classic applicable to your applications (e.g., Does it work well with the data types and data manipulations in your application?)

    We currently provide netCDF conversion operationally both as an on-the-fly conversion and as a download format in Giovanni.

  2. Why do you choose to use netCDF classic over other data formats for your applications?

    netCDF is better understood by a number of applications when combined with CF-1 than are other formats, such as HDF-EOS.

  3. Have you or your users encountered any difficulty when using some of the data access or visualization tools (e.g., IDL, GrADS, etc.) on netCDF classic data files? If you have, please provide a brief description of your experience.

    So long as CF-1 conventions are followed, Level 3 (gridded) data are handled well by many visualization tools. Even Level 2 (swath) data can be handled by select tools (IDV and McIDAS-V), making the format somewhat unique.

  4. Does the netCDF file format meet your requirements for storing and accessing data? (e.g., Can it handle the data types in your applications?)

    N/A

  5. What operational challenges or limitations does netCDF classic present? (e.g., Does it take a long time to learn how to use it? Does it require advanced processing power, large amounts of memory, complex configuration, etc)

    None.

  6. What benefits does netCDF classic present? Do the benefits of netCDF classic outweigh the challenges? (e.g., Does it offer the flexibility you want to package the data types in your applications? Does it facilitate interdisciplinary studies?)

    Note that most of the benefits confer only when CF-1 is also added. Although in theory, CF-1 can apply to other formats, in reality it is implemented fully only in netCDF. However, this allows data to be easily imported and viewed into a number of very powerful analysis and visualization tools.

  7. How much data do/will you provide or archive in netCDF classic? (number of distinct data products or data sets, total data volume, number of files.)

    ~7 data products, amounting to tens of terabytes

  8. How many users do you have or expect to have for data in netCDF classic, and what is your expected user community?

    Eventually, we expect hundreds of users in both the IDV / McIDAS-V and Giovanni user communities.

  9. (User comments) Any additional comments, observations or criticisms of netCDF classic and the RFC can be provided here.