1. OIT HPC
  2. Running Jobs
  3. QOS Availability

QOS Resource Availability

How to find out what resources a job under a particular QOS could actually claim right now — free hardware is only half the answer, because most QOS also carry a shared group budget.

On This Page

Which QOS Can You Use?

Start with sqos. It lists every QOS you can submit with, the partitions each is valid on, and the limits that apply — wall time, per-user and per-job caps, and the QOS's shared GrpTRES group budget: 

$ sqos
QOS     Partitions        MaxWall      MaxTRES/User     MaxTRES/Job  GrpTRES
normal  compute           4-00:00:00   cpu=512          -            -
long    compute           10-00:00:00  cpu=512          -            -
gpu     gpu               4-00:00:00   -                -            -
short   compute_partners  02:00:00     -                -            -

sqos                    # QOS available to you
sqos -v                 # also show priority and flags, plus the source associations
sqos alice              # QOS available to another user

sqos answers the entitlement question only — what you are allowed to submit, and the ceilings that will apply. It reads your associations and the QOS definitions, never the live queue, so it says nothing about how much of a budget is currently spent. For that, read on.

See What QOS and accounts can I use? for the full sqos reference (including -m, which inverts the question to who holds a QOS), and Partitions and Resources for the partition and QOS tables.

Available vs. Allocated: the Two Scopes

Whether your job can start depends on two independent things, and each has its own columns in si:

  • The hardware — is anything physically free in the partition? Plain si shows this per partition and CPU architecture (--gpus per GPU model, --memory in GiB): Avail is idle, Alloc is allocated by every QOS put together, and Total is installed capacity.
  • The QOS budget — most QOS carry a cluster-wide GrpTRES group limit shared by everyone who holds the QOS. Once that budget is spent, the next job pends even if the partition is half empty.

si --qos <QOS> combines the two. It adds a GrpTRES column (the QOS's group cap) and a Used column (what that QOS's running jobs currently hold against it), restricts the table to the partitions that allow the QOS, and lowers Avail to the smaller of what is physically free and what remains of the budget — so the number you see is what a job under that QOS could actually take right now. Add --partition to narrow it to one partition: 

si --qos long                                 # what a 'long' job could claim, everywhere it can run
si --partition compute --qos long             # the same, restricted to one partition
si --gpus --qos gpu                           # GPUs a job under QOS 'gpu' could claim, per model
si --partition gpu_partners --gpus --qos p_mygroup   # a partner group's GPU allocation

The same table shows both ways Avail can reach zero, and telling them apart is the point of the view: 

$ si --gpus --qos gpu
Partition  GPU       GrpTRES  Used  Avail  Alloc  Total
gpu        GTX_1080        -     -      1      1      2
gpu        RTX_2080        4     1      3      1      4
gpu        A30             4     4      0      0      4
gpu        H100            8     0      0      8      8
  • A30: Avail 0 because the QOS budget is spent (Used 4 of GrpTRES 4) — the hardware itself is idle (Alloc 0), but no further gpu-QOS job can have it until one finishes.
  • H100: Avail 0 because the hardware is busy (Alloc 8 of Total 8) — the QOS has budget left (Used 0), but other QOS's jobs hold every device.

Note the two scopes on each row: GrpTRES and Used describe the QOS cluster-wide (a shared cap repeats on every row), while Alloc and Total describe that row's partition and architecture — which is why Used can exceed Alloc, and why Alloc counts every QOS's jobs, not just the one you asked about.

For a partner QOS, only the hardware the group purchased appears — the typed per-architecture / per-model caps are what govern placement: 

$ si --partition compute_partners --qos p_ssp
Partition         Architecture  GrpTRES  Used  Avail  Alloc  Total
compute_partners  Icelake-8358      256   192     64    192    576

Under the table, si --qos prints the QOS's total running usage and a timestamp: 

QOS p_ssp running usage: cpu=192  mem=1500G  node=2
(QOS usage as of 2026-10-28 09:40:12, 4 min ago)
How fresh are these numbers? The physical columns (Avail before capping, Alloc, Total) are read live. The Used figure and the budget cap on Avail come from a usage snapshot refreshed about every 10 minutes — the "as of" line shows its age, and si warns if the snapshot goes stale. Per-user limits (MaxTRES/User) are not applied to Avail; check yours with sqos.

If you name a partition the QOS is not allowed on, si says so (si: QOS long is not permitted on partition gpu) rather than printing an empty table — a quick way to confirm which partition a QOS belongs to.

Who Is Using It, and What Is Queued Behind It

si --qos gives the aggregate; sq gives the jobs behind it: 

sq --run -q long --tres     # running jobs under the QOS, with the resources each holds
sq --pend -q long           # jobs queued under the QOS, with priority and reason
sq --pend -q long --start   # ...plus the scheduler's estimated start times
sq --run -q p_mygroup       # partner members: every member's jobs under the group QOS

With --tres, the TOTAL line sums to the same figure as si --qos's Used column. Job queries are restricted to your own jobs, so an ordinary user's list covers only their share of the QOS — a short list does not mean the QOS is idle, and si --qos remains the cluster-wide number. Members of a partner QOS are the exception: sq -q p_<group> shows every member's jobs under that QOS.

Pending with reason QOSGrpCpuLimit or QOSGrpGRES? The blocker is the QOS's group budget, not free hardware. si --qos <QOS> shows how much of the budget is spent and sq --run -q <QOS> what is holding it; the job starts when enough of the budget frees up. See Who is using a QOS right now? for a worked example.

Native Slurm Commands

The wrappers above assemble their answers from standard Slurm commands, which work on the login nodes too: 

sinfo -p compute -o "%P %t %D %C"         # node states and A/I/O/T core counts for a partition
scontrol show partition compute_partners  # a partition's AllowQOS list and limits
sacctmgr show qos long format=Name,Priority,MaxWall,GrpTRES,MaxTRESPU%40,MaxTRES%40
                                          # a QOS's defined limits
squeue -q long                            # jobs under a QOS (your own jobs only)

One reading to know: in sinfo's %C column the four numbers are Allocated/Idle/Other/Total cores.

Job-level queries (squeue, sacct) are restricted to your own jobs on this cluster, which is why si --qos publishes the aggregate usage figures instead.