Using MPI within a Conda package

Conda is package management system that installs applications and all of their dependencies. That means that if an installed package requires MPI, Conda will install its own MPI. User installed MPI will not work properly with LSF, therefore a user must use the system MPI.

Below are examples of using system MPI for R or Python libraries requiring MPI.

Rmpi within a Conda environment

Install all of the necessary Conda packages, preferably with a YAML file, as per the Conda documentation, but do not include Rmpi. For existing Conda environments, MPI may be uninstalled. Type conda list to find which packages contain MPI, and uninstall. For example, in a Conda environment with Rmpi installed, conda remove mpi will remove the mpi, openmpi, and r-rmpi packages. It will uninstall any packages that depend on the mpi package.

Installing Rmpi

1) Activate the Conda environment

First, activate the Conda environment, using

conda activate [/path/to/env]

2) Set the environment to use system MPI

module load openmpi-gcc/openmpi1.8.4-gcc4.8.2    #This version of MPI is necessary for a successful compile
export RMPI_INCLUDE = /usr/local/apps/openmpi/1.8.4-gcc4.8.2/include
export RMPI_LIBPATH = /usr/local/apps/openmpi/1.8.4-gcc4.8.2/lib

3) Open R, and use install.packages

R
install.packages("Rmpi",repos="https://mirrors.nics.utk.edu/cran/",configure.args="--with-Rmpi-libpath=$RMPI_LIBPATH --with-Rmpi-include=$RMPI_INCLUDE --with-Rmpi-type=OPENMPI")

4) Set the environment in a batch script

In the batch script, activate the Conda environment, and set the MPI environment by loading the module.

module load openmpi-gcc/openmpi1.8.4-gcc4.8.2
conda activate [/path/to/env] 
mpirun -n 1 Rscript ./my-Rscript.R
conda deactivate

5) Before using Rmpi within the Conda environment

Confirm that the MPI is functioning properly with LSF by running the sample Rmpi script provided in the R documentation. Look at the output and confirm that R is using multiple nodes.

Python mpi4py within a Conda environment

Install all of the necessary Conda packages, preferably with a YAML file, as per the Conda documentation, but do not include mpi4pi. For existing Conda environments, MPI may be uninstalled. Type conda list to find which packages contain MPI, and uninstall. For example, in a Conda environment with mpi4py installed, conda remove mpi will remove the mpi, openmpi, and mpi4py packages. It will uninstall any packages that depend on the mpi package.

Installing mpi4py

1) Activate the Conda environment

First, activate the Conda environment, using

conda activate [/path/to/env]

2) Set the environment to use system MPI.

The mpi4py installs were tested with the Intel 2017 compiler. This is recommended, as it also sets the environment to use the Intel MKL libraries.

module load PrgEnv-intel

3) Use pip install

Using pip install will compile mpi4py, rather than downloading it as a prepackaged binary, and that will force it to use the system MPI.

pip install mpi4py

4) Set the environment in a batch script

In the batch script, activate the Conda environment, and set the MPI environment by loading the module.

module load PrgEnv-intel 
conda activate [/path/to/env] 
mpirun ./my-mpi4py-script.R
conda deactivate

5) Before using mpi4py within the Conda environment

Confirm that the MPI is functioning properly with LSF by running the sample Python script provided in the Python documentation. Look at the output and confirm that R is using multiple nodes.

Still having problems?

• Please contact HPC Staff for further assistance. When opening the ticket, please include information such as the packages installed, method of installation, environment variables that were set, the batch script used, and the resulting LSF output and error fies.
• For more complex applications using MPI, or other applications designed for high-performance, a Conda environment is usually not the preferred installation method. Check the application for instructions on compiling from source. Here is some guidance on installing software.

Last modified: April 02 2024 13:40:32.