Note:
The video version of this segment of the tutorial is here.
Running from the scratch directory
You should keep source code and scripts in your home directory, but you should run your jobs from the scratch directory. Code and scripts need to be in permanent storage. Output files can be large, and they should be generated in the scratch directory. Since the scratch directory is purged every 30 days, you must move the outputs that you want to keep to a local computer or to mass storage. You will ask for mass storage in Step 7.
Exercise 4.0: Always run from the scratch directory
The directory name, /share, is an alias to another directory, and it is not the same for every user. Usually you would not need to know this, but printing the working directory from /share may be confusing. Here is what the full path may look like for a typical user:
[unityID@login ~]$ cd /share/groupname/unityID/
[unityID@login unityID]$ pwd
/gpfs_common/share01/groupname/unityID
Copy the current working version of the guide directory from your home directory to your scratch space on /share.
Hint: Add the argument '-r' (for recursive) to the usual copy command to copy an entire directory.
What is the full path of your scratch directory?
Finding software specific information
Click to open a new tab to the HPC home page. Look for Software Packages under Resources. In this list, find the current version of R and the Load Environment. Look at the column How to use... for examples.
Using the software documentation
Use the R documentation to complete the following exercises.
Exercise 4.1: Submit a R job to a compute node using LSF
All of the required instructions are in the documentation on using R.
Navigate the file system so that the current working directory is the guide directory that you have just copied to /share.
Create a batch submission script submit.sh that runs the R script weather.R on one node with a 10 minute time limit, then submit the job to LSF.
Hint: In submit.sh, type the same text as contained in the example batch script in the R documentation. Change the wall clock time, and change the name of the script from my_program.R to weather.R. Optional: Change the names of the output and error files, and add a BSUB argument that sets the jobname. This example is not memory intensive, so you may delete the line for requesting an exclusive node.
Type bjobs. When you first submit a job to LSF, the job will be listed with a PEND (pending) status. Hit the [up] arrow key to repeat bjobs to check the status. After the job begins, the status will be changed to RUN.
The R code should produce a PDF file called weather.pdf containing temperature. After the job is finished, display the contents of your directory to confirm that weather.pdf has been created, and type file weather.pdf to check that it is a valid PDF file. If not, check the LSF output files generated from your run. Modify and resubmit the job if necessary.
The LSF error file should be empty (no errors), and the LSF output file should contain a list of temperatures from the program and the LSF data related to the run (e.g., CPU time, memory, and LSF run parameters).
Copy the output file weather.pdf to your local computer.
Exercise 4.2: Always save code and scripts to a directory with backups
Copy the submit.sh script to your guide directory on /home.
Keep the browser tab with the R instructions open for Step 5.
Go to Step 5