Basic user storage

The main types of storage available to the general user are the home directory and the scratch directory. Many users will only use these spaces.

    Home directory

    The home directory is intended for scripts, small application source code, and executables. There is not enough space in the home directory for large data files.

    Each account has 1 GB of quota in /home/user_name.

    This space is backed up daily. One copy of each file is retained in the backup. Deleted files are retained for about 7 days. Files not associated with a valid Unity ID will be archived.

    To check how much space is available in the home directory, type quota -s.

    Scratch directory

    Scratch space is intended for the storage requirements for running jobs. Applications should use scratch space during job execution, i.e., jobs should be submitted from /share.

    Each project has 10 TB of quota in /share/group_name. Users should create a subdirectory under this location for their use:

    cd /share/group_name
    mkdir user_name
    
    To find the group_name, see the first listed output from the groups command. This is the default group.

    To check how much space is being used in a directory, type du -h -d 0. Depending on the amount of files, this may take a long time.

    This space is not backed up. Files that have not been accessed in 30 days are automatically deleted.

Supplemental group storage

Supplemental storage is available for data and applications.

    Directory for user maintained software

    HPC provides space for user installed software. The space is backed up daily. If a /usr/local/usrapps/group_name directory does not exist, see the requirements for requesting the space on the HPC software page.

    Directories in /usr/local/usrapps may not be used for data or as a working space from which to execute jobs. A compute node cannot write to /usr/local/usrapps. Globus and HPC-VCL cannot write to this space either.

    Instructional use mass storage directory

    Since a class lasts longer than 30 days, it may be necessary to store large inputs and examples in case of purge; in that case, the instructor may request a mass storage space in /gpfs_archive.

    For more details and an example use case on using mass storage for a class, see Henry2 for Instructional use: access, storage, software, and training.

    There is a 1 TB group quota on this space. Mass storage is accessible only from the login nodes. It is not available from compute nodes and cannot be used as an alternative to scratch space for running jobs.

    The mass storage is a hierarchically managed file system consisting of a disk cache and a tape library. In order to speed archival and retrieval, a tar archive of a directory, including all files and subdirectories, should be used to store data. Storing many individual files may be distributed across numerous tapes and take a long time to retrieve in the event that the recovery of an entire directory is required.

Archival Policy

Files not associated with a valid Unity ID will be archived to tape and then deleted after 10 years.
  • The validation of Unity IDs for all HPC users is performed once a year in December.
  • If a Unity ID is no longer valid, i.e., the user has left the university, the archiving will be performed on their directories /home/user_name and /gpfs_archive/group_name/user_name.
  • If the invalid Unity ID belongs to a PI, the archiving will be performed on their directories only if there are no active accounts remaining under their project(s).

Additional Resources

Consulting and data management plans are available.

Libraries: Storage and Data Management Resources

The NC State University Libraries offers help during all phases of the research data lifecycle, including preparation of data management plans (DMPs) for grant proposals, consulting on best practices for storage, organization, and preservation, and helping to optimize sharing and discovery of data.

Copyright © 2022 · Office of Information Technology · NC State University · Raleigh, NC 27695 · Accessibility · Privacy · University Policies