The following is a list of the storage options available on Hazel. For more details, see the video: Hazel: Navigating the different types of HPC storage
To move data to and from the cluster, see File Transfer.
The home directory is intended for scripts, small application source code, and executables. There is not enough space in the home directory for large data files.
Each account has 1 GB of quota in /home/user_name.
This space is backed up daily. One copy of each file is retained in the backup. Deleted files are retained for about 7 days. Files not associated with a valid Unity ID will be archived.
To check how much space is available in the home directory, type
quota -s.
Scratch space is intended for the storage requirements for running jobs. Applications should use scratch space during job execution, i.e., jobs should be submitted from /share.
Scratch space is not backed up and files that have not been accessed for 30 days are automatically deleted.
Each project has 10 TB of quota in /share/group_name. Users should create a subdirectory under this location for their use:
cd /share/group_name mkdir user_nameTo find the group_name, see the first listed output from the
groups command. This is the default group.
To check how much space is being used in a directory, type du -h -d 0. Depending on the amount of files, this may take a long time.
To check overall /share quota usage use command mmlsquota -g group_name gpfsHPCcommon2
This space is not backed up. Files that have not been accessed in 30 days are automatically deleted.
Each NC State researcher may obtain an allocation of research storage to use during their employment at NC State University. Also, PIs may obtain research storage allocation for awarded grants.
HPC provides space for user installed software. The space is backed up daily. If a /usr/local/usrapps/group_name directory does not exist, see the requirements for requesting the space on the HPC software page.
Directories in /usr/local/usrapps may not be used for data or as a working space from which to execute jobs. A compute node cannot write to /usr/local/usrapps. Globus and HPC-VCL cannot write to this space either.
Since a class lasts longer than 30 days, it may be necessary to store large inputs and examples in case of purge; in that case, the instructor may request a mass storage space in /gpfs_archive.
For more details and an example use case on using mass storage for a class, see Hazel for Instructional use: access, storage, software, and training.
There is a 1 TB group quota on this space. Mass storage is accessible only from the login nodes. It is not available from compute nodes and cannot be used as an alternative to scratch space for running jobs.
The mass storage is a hierarchically managed file system consisting of a disk cache and a tape library. In order to speed archival and retrieval, a tar archive of a directory, including all files and subdirectories, should be used to store data. Storing many individual files may be distributed across numerous tapes and take a long time to retrieve in the event that the recovery of an entire directory is required.
/home/user_name
and
/gpfs_archive/group_name/user_name.
The NC State University Libraries offers help during all phases of the research data lifecycle, including preparation of data management plans (DMPs) for grant proposals, consulting on best practices for storage, organization, and preservation, and helping to optimize sharing and discovery of data.