Ticket 22458 - What is max value allowed for Priority in a QOS?
Summary: What is max value allowed for Priority in a QOS?
Status: RESOLVED INFOGIVEN
Alias: None
Product: Slurm
Classification: Unclassified
Component: Other (show other tickets)
Version: 24.05.3
Hardware: Linux Linux
: 4 - Minor Issue
Assignee: mckay@schedmd.com
QA Contact:
URL:
Depends on:
Blocks:
 
Reported: 2025-03-28 11:54 MDT by David Chin
Modified: 2025-03-31 10:47 MDT (History)
0 users

See Also:
Site: Perelman School of Medicine University of Pennsylvania
Alineos Sites: ---
Atos/Eviden Sites: ---
Confidential Site: ---
Coreweave sites: ---
Cray Sites: ---
DS9 clusters: ---
HPCnow Sites: ---
HPE Sites: ---
IBM Sites: ---
NOAA SIte: ---
NoveTech Sites: ---
Nvidia HWinf-CS Sites: ---
OCF Sites: ---
Recursion Pharma Sites: ---
SFW Sites: ---
SNIC sites: ---
Tzag Elita Sites: ---
Linux Distro: ---
Machine Name:
CLE Version:
Version Fixed:
Target Release: ---
DevPrio: ---
Emory-Cloud Sites: ---

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this ticket.
Description David Chin 2025-03-28 11:54:26 MDT 
In the cmdline:

    sacctmgr modify qos highprio-gpu set priority=XXX

what is the max value allowed for "priority"?

We have several user jobs which need to go to the top of the pending list. Already tried "scontrol top JOBID" for each of those jobs, but did not seem to have an effect.

So, trying to add a QOS, which could also be used for other users in future.

QOS is defined as:

❯ sacctmgr show qos format=name%15,priority,flags,maxwall,maxtresperuser%80,MaxJobsPU where name=highprio-gpu
           Name   Priority                Flags     MaxWall                                                                        MaxTRESPU MaxJobsPU 
--------------- ---------- -------------------- ----------- -------------------------------------------------------------------------------- --------- 
   highprio-gpu     999999          OverPartQOS                           cpu=256,gres/gpu:a100=12,gres/gpu:a40=2,gres/gpu:l40=1,gres/gpu=12
Comment 2 mckay@schedmd.com 2025-03-28 17:18:33 MDT 
The short answer is that job priority is a 32-bit unsigned integer, so essentially 4294967293

Setting the qos is a way to affect the priority, but its use comes with some quirks, namely:

* the 'priority/multifactor' plugin must be in use for the qos to be honored
(https://slurm.schedmd.com/priority_multifactor.html#general)

* 'scontrol top', for users, requires 'enable_user_top' to be set (https://slurm.schedmd.com/scontrol.html#OPT_top and https://slurm.schedmd.com/slurm.conf.html#OPT_enable_user_top)


As an admin, you have a couple of other knobs you can turn for manipulating the priority:

* renicing the job: https://slurm.schedmd.com/scontrol.html#OPT_Nice

* setting the priority directly: https://slurm.schedmd.com/scontrol.html#OPT_Priority

For example:
> scontrol update jobid=9866 priority=70000

Let me know if you any other questions.
Comment 3 David Chin 2025-03-31 09:57:45 MDT 
That answers my question. We can close this out.

Thanks much!
Comment 4 mckay@schedmd.com 2025-03-31 10:47:36 MDT 
Glad to be of help.