Ticket 22458

Summary: What is max value allowed for Priority in a QOS?
Product: Slurm Reporter: David Chin <david.chin>
Component: OtherAssignee: mckay <mckay>
Status: RESOLVED INFOGIVEN QA Contact:
Severity: 4 - Minor Issue    
Priority: ---    
Version: 24.05.3   
Hardware: Linux   
OS: Linux   
Site: Perelman School of Medicine University of Pennsylvania Slinky Site: ---
Alineos Sites: --- Atos/Eviden Sites: ---
Confidential Site: --- Coreweave sites: ---
Cray Sites: --- DS9 clusters: ---
Google sites: --- HPCnow Sites: ---
HPE Sites: --- IBM Sites: ---
NOAA SIte: --- NoveTech Sites: ---
Nvidia HWinf-CS Sites: --- OCF Sites: ---
Recursion Pharma Sites: --- SFW Sites: ---
SNIC sites: --- Tzag Elita Sites: ---
Linux Distro: --- Machine Name:
CLE Version: Version Fixed:
Target Release: --- DevPrio: ---
Emory-Cloud Sites: ---

Description David Chin 2025-03-28 11:54:26 MDT 
In the cmdline:

    sacctmgr modify qos highprio-gpu set priority=XXX

what is the max value allowed for "priority"?

We have several user jobs which need to go to the top of the pending list. Already tried "scontrol top JOBID" for each of those jobs, but did not seem to have an effect.

So, trying to add a QOS, which could also be used for other users in future.

QOS is defined as:

❯ sacctmgr show qos format=name%15,priority,flags,maxwall,maxtresperuser%80,MaxJobsPU where name=highprio-gpu
           Name   Priority                Flags     MaxWall                                                                        MaxTRESPU MaxJobsPU 
--------------- ---------- -------------------- ----------- -------------------------------------------------------------------------------- --------- 
   highprio-gpu     999999          OverPartQOS                           cpu=256,gres/gpu:a100=12,gres/gpu:a40=2,gres/gpu:l40=1,gres/gpu=12
Comment 2 mckay@schedmd.com 2025-03-28 17:18:33 MDT 
The short answer is that job priority is a 32-bit unsigned integer, so essentially 4294967293

Setting the qos is a way to affect the priority, but its use comes with some quirks, namely:

* the 'priority/multifactor' plugin must be in use for the qos to be honored
(https://slurm.schedmd.com/priority_multifactor.html#general)

* 'scontrol top', for users, requires 'enable_user_top' to be set (https://slurm.schedmd.com/scontrol.html#OPT_top and https://slurm.schedmd.com/slurm.conf.html#OPT_enable_user_top)


As an admin, you have a couple of other knobs you can turn for manipulating the priority:

* renicing the job: https://slurm.schedmd.com/scontrol.html#OPT_Nice

* setting the priority directly: https://slurm.schedmd.com/scontrol.html#OPT_Priority

For example:
> scontrol update jobid=9866 priority=70000

Let me know if you any other questions.
Comment 3 David Chin 2025-03-31 09:57:45 MDT 
That answers my question. We can close this out.

Thanks much!
Comment 4 mckay@schedmd.com 2025-03-31 10:47:36 MDT 
Glad to be of help.