Submitting Parallel Jobs

Please examine the runtime behavior of your applications.

Running with the incorrect LSF specifications can result in violating the Acceptable Use Policy and you may be asked to terminate your jobs.

Read the documentation to determine the expected behavior of an application, then confirm the behavior with a short test.

Here is a description of how to find information about expected behavior and how to test the behavior on an interactive compute node.

In general:

• If a job is serial, choose a single core.
• For memory intensive applications, the maximum memory required should be specified in the LSF script.
• For parallel jobs, the number of cores requested should equal the number of threads in the program, that is, if it can run on multiple cores, choose -n to match. An exception is for hybrid MPI-OpenMP jobs where -n is the number of tasks and the number of threads per task is managed by using ptile and -x, as in this example batch script.
• If a job is not a distributed memory job (normally MPI based), use #BSUB -R "span[hosts=1]".
• You must verify the threading behavior of the application. Some newer applications automatically run with the number of threads equal to the number of hardware cores.
• When writing your own code, note that in external libraries, some subroutines and functions use multithreaded math libraries. The number of cores requested should accommodate this.

See this documentation for more information on queues and core counts.

For further guidance:

• See Step 6 from the Quick Start Tutorial which has detailed examples and explanations for how to examine, test, and submit shared and distributed memory jobs.