Compiling Code

List of compilers

Three sets of compilers are available on Hazel: GNU, Intel, and Nvidia (formerly Portland Group). Each set includes Fortran, C, and C++ compilers.

• Best practices and troubleshooting tips for compiling

GNU compilers

The GNU compilers are provided as part of the Linux distribution used on the Hazel cluster.

To use the GNU implementation of Open MPI, load the module module load openmpi-gcc.

To compile with GNU:

Serial compilation

• C - gcc
• C++ - g++
• Fortran - gfortran

Parallel compilation using MPI

• C - mpicc
• C++ - mpicxx, also mpiCC and mpic++
• Fortran - mpifort, also mpif77 and mpif90

Parallel compilation using OpenMP

• When compiling, use the flag -fopenmp.

Intel compilers

To use the Intel compilers, load the module module load PrgEnv-intel.

To compile with Intel:

Serial compilation

• C - icc
• C++ - icpc
• Fortran - ifort

Parallel compilation using MPI

• C - mpiicc, also mpicc
• C++ - mpiicpc, also mpicxx
• Fortran - mpiifort, also mpif77 and mpif90

Parallel compilation using OpenMP

• When compiling, use the flag -qopenmp.

Nvidia (Portland Group compilers)

To use the PGI compilers, load the module module load PrgEnv-pgi.

To compile with PGI:

Serial compilation

• C - pgcc
• C++ - pgc++
• Fortran - pgfortran also pgf77 and pgf90

Parallel compilation using OpenMP

• When compiling, use the flag -mp.

Parallel compilation using OpenACC

• When compiling, use the flag -acc or ta=[target].

Compiler tips and recommendations

Warnings

Do not cross compile. Use the same compiler for all parts of a program. Do not compile a library dependency with PGI and then compile the executable with Intel.

Use the same environment variables used for compilation when running the code. If module load intel was used to compile the code, use module load intel to set the environment in a submit script before launching the executable.

By default, the PGI compiler will perform optimizations based on the node it was compiled on. If a PGI program is compiled on a login node it may not run on an older node, resulting in an error of illegal instruction. To compile without optimizing for architecture, please refer to the latest PGI compiler documentation.

For Intel, the compile flag -xHost is used to optimize based on the compilation node. For programs that come packaged with a default Makefile, check compiler options before using. A program compiled with -xHost on one node may result in an instruction set error on another node. To properly use -xHost, the compilation may be performed directly on the target node type by using an interactive session (bsub -Is, see LSF for Interactive jobs).

See Advanced Compiler Options for instructions on creating a single binary that is optimized for several different architectures.

To specify an instruction set architecture when submitting a batch job, use the LSF resource specification.

Choosing a compiler

To load a version of a compiler that is not the default, view the available compiler versions with module avail and then load the full pathname of the module, e.g., module load PrgEnv-intel/2016.0.109.

The Intel compiler and Intel MPI often give the best performance for scientific computing programs. For performance, use Intel.

Some difficult to compile codes depend on the GNU compiler. For ease of compiling, use GNU.

When doing GPU programming with OpenACC, use the PGI compiler.

When working with collaborators or users in the general public, it may be useful to ensure the code compiles with GNU so that end users are not required to purchase a compiler license. GNU is free and open source, and PGI has a community edition that may be free for general public users. Compiling code with different compilers is also a way to expose bugs.

Setting environment variables

Click here for Hazel specific information on setting the compute environment.

See the R documentation for installing R packages for examples of setting the compute environment to install software. The procedure of setting paths and environment variables is not specific to R.

To compile a research application or third party software

If the application will not fit in a home directory, request a space for user maintained software.

Read the README, FAQ, and tutorials provided by the software. Software may not be installed using automatic installers that require sudo permissions, so users must spend time understanding the software requirements such as compiler types and library dependencies.

Automatic package installers require sudo because they write to system directories. Often, the only necessary modification to an install script is for the user is to specify an alternate installation directory prefix.

A successful compilation often requires an understanding of the compute environment on Hazel. Sometimes the compile requirements are a matter of locating the correct library or setting the proper path on Hazel, but other times a user may have to install their own custom compiled libraries.

Here is some general guidance for installing software.