Ticket 22459 - Max Running Job Array Steps
Summary: Max Running Job Array Steps
Status: OPEN
Alias: None
Product: Slurm
Classification: Unclassified
Component: Configuration (show other tickets)
Version: 23.11.10
Hardware: Linux Linux
: 4 - Minor Issue
Assignee: Michael Norris
QA Contact:
URL:
Depends on:
Blocks:
 
Reported: 2025-03-28 12:34 MDT by douglas.kirby
Modified: 2025-03-28 13:23 MDT (History)
3 users (show)

See Also:
Site: University of Connecticut
Alineos Sites: ---
Atos/Eviden Sites: ---
Confidential Site: ---
Coreweave sites: ---
Cray Sites: ---
DS9 clusters: ---
HPCnow Sites: ---
HPE Sites: ---
IBM Sites: ---
NOAA SIte: ---
NoveTech Sites: ---
Nvidia HWinf-CS Sites: ---
OCF Sites: ---
Recursion Pharma Sites: ---
SFW Sites: ---
SNIC sites: ---
Tzag Elita Sites: ---
Linux Distro: ---
Machine Name:
CLE Version:
Version Fixed:
Target Release: ---
DevPrio: ---
Emory-Cloud Sites: ---


Attachments

Note You need to log in before you can comment on or make changes to this ticket.
Description douglas.kirby 2025-03-28 12:34:44 MDT
Is there a config option to limit the number of array steps that are run concurrently per user?

I saw in ticket#2994 there's a user side option to limit the number of running tasks e.x. --array=0-15%4, and we can add it after the fact to a users job, but a server limit would be preferable.
Comment 1 Michael Norris 2025-03-28 13:23:04 MDT
(In reply to douglas.kirby from comment #0)
> Is there a config option to limit the number of array steps that are run
> concurrently per user?
> 
> I saw in ticket#2994 there's a user side option to limit the number of
> running tasks e.x. --array=0-15%4, and we can add it after the fact to a
> users job, but a server limit would be preferable.

Douglas,

There are a few things you can try. One paramater you could try to set up is MaxJobs
https://slurm.schedmd.com/sacctmgr.html#OPT_MaxJobs
https://slurm.schedmd.com/sacctmgr.html#OPT_MaxJobs_1


There are several options similar to that you may want to tinker with in order to get what you are looking for. 

However if you want to specifically limit array size you may want to look at building out a job_submit.lua plugin
https://slurm.schedmd.com/job_submit_plugins.html

You should be able to use this to read in the array size and either modify it down to your desired minimum or reject the job. Let me know if you have any questions about this

Michael Norris