HDF5 - Hierarchical Data Format

HDF5 is a data model, library, and file format for storing and managing data.



External Links:
HDF5 support page
HDF5: Examples from Learning The Basics

Example C code to test HDF5

Here is an example C code that uses the HDF5 library from the HDF5 support website.

To avoid copy/paste errors when using, please copy this from the apps directory:

/usr/local/apps/examples/code/hdf5

Linking HDF5 with Intel compiler

To put the HDF5 libraries in the path for Intel, use

module load hdf5/1.10.2-intel2017  


To link to HDF5, use -lhdf5 to include the library and use -I/usr/local/apps/hdf-centos7/hdf5/v1.10.2/include to include the header files.

To test linking HDF5 with the Intel compiler, compile the example code above, run it, and check that it creates a properly formatted HDF5 file. (Note that normally code should never be run on a login node; this small test is an exception.)

[unityID@login]$ cp -r /usr/local/apps/examples/code/hdf5 .
[unityID@login]$ cd hdf5
[unityID@login]$ module purge
[unityID@login]$ module load hdf5/1.10.2-intel2017
[unityID@login]$ mpicc -o test_hdf5_intel h5_crtdat.c -lhdf5 -I/usr/local/apps/hdf-centos7/hdf5/v1.10.2/include
[unityID@login]$ ./test_hdf5_intel 
[unityID@login]$ file dset.h5 
dset.h5: Hierarchical Data Format (version 5) data

Here, the MPI compile wrapper (mpicc) is necessary because the HDF5 includes an MPI header file.

Linking HDF5 with GNU compiler

To put the HDF libraries in the path for GNU, use

module load hdf5/1.10.5-gcc4.8.5 


To link to HDF5, use -lhdf5 to include the library and use -I/usr/local/apps/hdf-centos7/hdf5/v1.10.5-gcc4.8.5/include to include the header files.

To test linking HDF5 with the GNU compiler, compile the example code above, run it, and check that it creates a properly formatted HDF5 file. (Note that normally code should never be run on a login node; this small test is an exception.)

[unityID@login]$ cp -r /usr/local/apps/examples/code/hdf5 .
[unityID@login]$ cd hdf5
[unityID@login]$ module purge
[unityID@login]$ module load hdf5/1.10.5-gcc4.8.5
[unityID@login]$ gcc -o test_hdf5 h5_crtdat.c -lhdf5 -I/usr/local/apps/hdf-centos7/hdf5/v1.10.5-gcc4.8.5/include
[unityID@login]$ ./test_hdf5 
[unityID@login]$ file dset.h5 
dset.h5: Hierarchical Data Format (version 5) data

Here, HDF5 was also compiled with MPI, but the MPI compile wrapper (mpicc) is not necessary. The code will correctly compile with mpicc as well.

Notes

HDF5 was compiled for C only, as NetCDF requires only the HDF5 C libraries and HDF5 is primarily installed as a NetCDF prerequisite. If an application requires the HDF5 FORTRAN libraries/modules to be compiled, contact HPC staff. Staff will either update the code to include the FORTRAN libraries or assist with installing a user maintained version of HDF5 depending on other application requirements.

HDF5 modules include the MPI libraries (IntelMPI for Intel, and OpenMPI for GNU). After loading the HDF5 module, type module list for the list of other included modules. To use HDF5 with another code, the MPI libraries must be the same.

HDF5 was compiled with the following compiler flags. Intel was compiled similarly but with a different prefix.

HDF5:
                ./configure --prefix=/usr/local/apps/hdf-centos7/hdf5/v1.10.5-gcc4.8.5 --enable-hl --enable-parallel

If an application requires the HDF5 libraries/modules to be compiled with different flags, the user must install a custom version under a space for user maintained software.

Last modified: March 23 2022 13:14:02.