1. OIT HPC
  2. Get Started
  3. Links for Learning

Links for Learning HPC

See the following links for learning HPC concepts and skills, from beginner to advanced.

Linux

First set up your environment

    If you have an account on Hazel, log in and type the commands into the terminal. If not, set up your environment using Learn Linux without Linux.

Learning path by use case
  • Just the bare minimum

      Go to Essential Linux, try the 8 basic commands, and create a text file using nano.

  • Some background first please!

      Go through the minimal set of the LinkedIn Learning video tutorial. Most of the video content is explaining basic concepts, such as why we would want a command line instead of a GUI, how directories and files are structured, and how commands are formed. It explains Linux in terms of what you already know from using Windows or a Mac. After that, it goes over clearly and in depth - with great examples - all of the 'bare minimum' commands.

  • Background, essentials, and a bit more

      Go through the entire LinkedIn Learning video course. If some sections seem boring to you, still go through them but at higher playback speed so you don't miss any background from skipped sections.

  • Less background, more commands, faster pace

      Going through the text based Full Linux Tutorial is the most efficient way to learn the most commands as quickly as possible.

  • Going to be around for a while?

      Go through the Software Carpentry Tutorial. They start from zero, but go into further depth and introduce shell scripting. After that, do the Full Linux Tutorial. You will skim through the first few sections, pick up some new concepts in the next few, and learn some extras in the last couple sections.

      Also, take advantage of the NC State subscription to LinkedIn Learning. There are several advanced courses on Linux.

Linux/Unix commands and scripting

Text based tutorials
  • Learn Linux without Linux gives instructions for setting up a Linux environment without access to Hazel or another Linux machine.
  • Essential Linux (from the Quick Start Tutorial) lists the minimal set of Linux to get started: 8 commands and the use of a text editor. It links to tutorial segments and a video covering those commands.
  • Full Linux tutorial is a tutorial developed externally but modified specifically for the Hazel environment. Completing the Introduction through Tutorial Two will cover the essential commands completely. Since the tutorial was created to mimic the Henry2 environment, where the default shell was tcsh. This will only cause issues in bash (default for emulators) when completing the later chapters on setting the environment.
  • Software Carpentry tutorial on the Unix Shell. The tutorial explains the basics of the Unix shell (Linux is a Unix-like operating system), including the difference between graphical user interface (GUI) and command line interface, using the prompt, navigating the file system, basic Unix/Linux commands, and the use of a text editor. Setup through Section 3 will cover all basic essentials necessary for interacting with Hazel, with the exception of how to actually look at a file (see cat and less). Note in Section 3, Hazel users do not have access to the touch command. Authors estimate that time to completion of the first three sections is 95 minutes, not including setting the environment. The remaining sections give an excellent introduction to more advanced file navigation along with basic shell scripting.
    • Video tutorial
  • LinkedIn Learning video tutorial on Linux. Going through the full Linux tutorial on LinkedIn Learning is well worth the time and consists of less than 2.5 hours total of video instruction and setup. For a quicker start, this Learning Path is a subset of content containing just the essentials needed for basic interaction with Hazel, consisting of about 40 minutes of video instruction plus setting the environment.
  • See the video Essential Linux Commands - Fluency Drill for practice using the basic set of commands. It also explains file and directory permissions, and how to change them.

    • Command line text editors

  • Vi/Vim: See the full LinkedIn Learning video tutorial on Vim or a short section on Vim from LinkedIn Learning Linux tutorial (skip the sudo apt install part).
  • nano: See the full LinkedIn Learning video tutorial on nano or a short section on nano from LinkedIn Learning Linux tutorial.

    • High Performance Computing

    For Hazel specific instruction, see the web-based Quick Start Tutorial or the HPC video tutorials.

    Basic concepts

  • Introduction to High-Performance Computing. This Software Carpentry tutorial is customized for the computing environment at the University of Leeds, but much of it is general and applies to the Hazel cluster. In particular, the setup and lessons 1 and 2 are completely general. In the sections about connecting to the system and transferring files, the only difference is the computer address. On Hazel, that address is login.hpc.ncsu.edu. The section on scheduling is very specific to their scheduler PBS (Hazel uses LSF). Instead of reading that section, see the documentation on running jobs on Hazel. The section on Software is perfectly applicable, but the exact output from the example commands will understandably be different. (Also, please do not load modules by default by modifying login files as mentioned in the tutorial.) See setting the compute environment for additional detail on the compute environment and loading software modules.
  • HPC Glossary from NICS. Here is a glossary of common HPC terms collected by the National Institute for Computational Sciences.
  • Also see the links below under Parallel processing.

    • General hardware

  • Understanding performance and obtaining hardware information. San Diego Supercomputer Center (SDSC), December 2018.

    This webinar discusses how to find CPU specs, memory quantity, cache configuration, file systems, OS version, GPU properties, and instruction sets (ISA). It also discusses why one should understand them, and it includes a short discussion of profiling, monitoring, and optimizing. The target audience is developers - users who write parallel programs but are not necessarily familiar with all of the hardware and performance terms. It is also helpful for users who are familiar with general hardware information but do not know how to find that information on Hazel.

    Parallel processing

  • Parallel Jobs Hazel: Basic HPC Workshop video series, March 2020.
    This video is the fourth installation of the Basic HPC Workshop. It covers concepts and definitions of parallel computing, how to find hardware specs of nodes on the cluster, and it shows examples of how to run both shared and distributed memory jobs.
  • Intro to Parallel Computing - MPI - 1. XSEDE Monthly Workshop Series: MPI Workshop, April 2019.
    This introductory video explains why parallel programming is necessary, showcases typical exascale computing problems, and gives a demonstration of the basic theory of parallelizing a calculation. It also covers key terminology including the definition of cores, processors, nodes, and PEs (processing elements), the difference between CPU and GPU in terms of proper use cases, and discusses performance and the limitations of parallelization. The video explains which parallel programming platforms are used for each type of parallelism (MPI, OpenMP, OpenACC, CUDA, etc.).
  • Introduction to Parallel Computing. Lawrence Livermore National Laboratory (LLNL).
    This is the material used for the first 2.5 hours of the HPC Training Workshops given by LLNL for their HPC users. Skimming over it will give a great overview of HPC concepts, and reading it carefully will give a comprehensive overview while defining most or all of the HPC terms and definitions a user is likely to ever come across. For HPC users wanting to begin learning and developing parallel applications, go through this tutorial. For an HPC user who only needs to understand parallel architecture enough to run code (i.e. submit HPC jobs) efficiently but does not plan to develop code, check out the diagrams in the Parallel Memory Architectures and Parallel Programming Models sections.
  • The XSEDE HPC Summer Boot Camp was hosted through the NC State University Libraries in June 2019. See to go through the Boot Camp exercises on Hazel.