Review Comment 9

Review of netCDF version 3 implementation and operational suitability
NASA's Earth Science Data Systems Standards Process Group (SPG) is considering the network Common Data Form version 3 (netCDF classic), for adoption as a community standard. You are invited to review this Requests For Comment (RFC) in the context of your implementation experience with this data format specification and its suitability for operational use. Only answer questions applicable to your experience. Please send completed review to:
spg-rfc-011@lists.nasa.gov.

Implementation Experience questions:

1.(Your background) Describe in a sentence or two your overall implementation experience related to the proposed specification. (e.g., specification implementer, tools developer, data provider, scientific analyst, science user, etc.) Have you directly implemented the netCDF classic format specification or modified a netCDF classic library using the specification? Did you use pre-existing software, and if so, what did you use?

**I am a tools developer, scientific programmer, and end-user of netCDF3, with experience of netCDF2/3/4 since 1993.**

2.(Completeness) Does the specification provide all the detail you need to implement it in software? (e.g., to read or write a data file; to implement or modify the library, a profile or extension; or develop a tool such as a format translator) If not, describe what is missing in the specification.

**Yes. The standard is complete AFAIK. The Argonne parallel netCDF group independently re-implemented the specification to produce standard-compliant files.**

3.(Accuracy) Do any parts of the specification contain inaccuracies, or internal inconsistencies? If so, please provide details.

**Not that I know of.**

4.(Clarity) Is any part of the specification ambiguous, or poorly explained? If so, please provide details.

**I've never read the binary format standard that carefully.**

5.(Balance) Does the standard describe the right set of concepts and data types, and enable the appropriate data operations for its intended users? Is this set of concepts and data types an overly broad set (requiring excessive complexity) or narrowly simplistic set?

**NetCDF3 classic format is over 15 years old. The main limitations of this format are lack of support for 64byte integers, packing, and arrays that exceed 2 GB. NetCDF4 fixes most of these limitations.**

6.(Usefulness) How well does this specification meet your information sharing needs? (e.g., does it work well with the data types and data manipulations in your application? Does it properly represent your datasets? What are the pros and cons of this data format?)

**We (my group and the climate modeling community in general) use netCDF3 because it is the right balance of simple interface and power required to store our gridded data. In comparison to HDF4 or HDF5, netCDF3 is much easier to use, and thus much more useful. Sharing files is transparent. In fact, DAP makes sharing netCDF files over the network easy.**

7.(Implementation) What implementation challenges does the proposed standard present? (e.g., does it require advanced processing power, large amounts of memory, complex configuration, etc.? Does it scale to a production environment?)

**No challenges that I can see.**


8.(Flexibility) In what software environment(s) have you used netCDF classic (e.g., Solaris, Linux, Windows, Mac OS X)? Have you implemented, tested or deployed netCDF classic or packages other than those provided by the original netCDF classic and developers?

**I have used and tested netCDF3 tools on all current OS's.**

Operational Suitability questions:

9.Do you currently use or plan to use netCDF classic in a production setting? What types of applications do you use with netCDF classic? Is netCDF classic applicable to your applications (e.g., Does it work well with the data types and data manipulations in your application?)

**I do not work in a 'production setting'. However, many users employ my netCDF tools (NCO) in production settings, e.g.,**

10.Why do you choose to use netCDF classic over other data formats for your applications?

**Simplicity and power of the API. It's all you need for gridded data.**

11.Have you or your users encountered any difficulty when using some of the data access or visualization tools (e.g., IDL, GrADS, etc.) on netCDF classic data files? If you have, please provide a brief description of your experience.

**The problems I have had are always caused by the application, not the netCDF3 format, as far as I can remember.**

12.Does the netCDF file format meet your requirements for storing and accessing data? (e.g., Can it handle the data types in your applications?)

**Yes. And, not to put too fine a point on it, but I've tried writing applications in the other widely used self-describing data format for geosciences, HDF, and I gave up because the API was too complex. That's why I have already updated NCO to support netCDF4, so I can have (some of) the power of the HDF back-end without the headaches of the HDF API.**

13.What operational challenges or limitations does netCDF classic present? (e.g., Does it take a long time to learn how to use it? Does it require advanced processing power, large amounts of memory, complex configuration, etc)

**Because it lacks built-in packing support, file sizes can get big for sparse datasets, but there are packing conventions that ameliorate this.**

14.What benefits does netCDF classic present? Do the benefits of netCDF classic outweigh the challenges? (e.g., Does it offer the flexibility you want to package the data types in your applications? Does it facilitate interdisciplinary studies?)

**Simple API. Wide support. Free as in beer and speech. The benefits of this combination are not surpassed by any other geoscience data format.**

15.How much data do/will you provide or archive in netCDF classic? (number of distinct data products or data sets, total data volume, number of files.)

**10 TB.**

16.How many users do you have or expect to have for data in netCDF classic, and what is your expected user community?

**I have a dozen or so colleagues with whom I exchange data in netCDF3 format. As a software developer (NCO), I have hundreds to thousands of users who apply my tools to netCDF3 files.**

17.(User comments) Any additional comments, observations or criticisms of netCDF classic and the RFC can be provided here