Summary
This document nominates the netCDF-4/Hierarchical Data Format Version 5 (HDF5) File Format for adoption as a NASA Earth Science Data Systems (ESDS) community standard. It specifies the netCDF-4/HDF5 file format independent of the netCDF I/O libraries designed to read and write netCDF-4/HDF5 data. The netCDF-4/HDF5 file format enables the expansion of the netCDF model, libraries, and machine-independent data format for geoscience data. Together the netCDF interfaces, libraries, and formats support the creation, access, and sharing of scientific data.
With suitable community conventions, the netCDF-4/HDF5 data format can help improve the capability to read and share important scientific data among data providers, data users, and data services.
Status
The netCDF-4/HDF5 File Format is an approved standard recommended for use in NASA's Earth Science Data Systems in March 2011.
|
NASA Earth Science Community Recommendations for Use
Strengths
netCDF-4 is straightforward to use relative to HDF5, with a lower learning curve. The many tools available enhance ease of use. Use of the HDF5 storage layer in netCDF-4 software provides features for improved performance, such as compression, parallel I/O, relaxed size limits, and the performance benefits of chunking and endianness control.
Weaknesses
Installation and set-up could be improved. HDF5 users point out that more manual intervention is required for installing netCDF-4/HDF5 than for HDF5 alone. NetCDF users point out that for netCDF-4, multiple software libraries must be installed (netCDF, HDF5, possibly other supporting libraries), rather than the one software library required for netCDF-3.
Applicability
netCDF-4 handles many data types and structures needed for Earth science. Those already using HDF tools can access netCDF-4 data using the HDF-5 API rather than netCDF. Those who have not been using HDF tools welcome access to much of the power of HDF via the simpler netCDF API. Community reviewers of the RFC cite many terabytes of data in netCDF-4, with thousands of users.
Limitations
One reviewer noted some internal inconsistencies between the netCDF-4 specification and the DAP library implementation. A more significant issue is that support for Windows users lags significantly behind Linux. This appears to be a problem for both the HDF5 and netCDF-4 software libraries.