2994 – Max number of instances of 1 job array.

Ticket 2994 - Max number of instances of 1 job array.

Summary: Max number of instances of 1 job array.

Status:	RESOLVED CANNOTREPRODUCE

Alias:	None

Product:	Slurm
Classification:	Unclassified
Component:	Scheduling (show other tickets)
Version:	14.11.10
Hardware:	Linux Linux

Severity:	4 - Minor Issue
Assignee:	Moe Jette
QA Contact:

URL:

Depends on:
Blocks:

Reported:	2016-08-12 07:54 MDT by paull
Modified:	2016-08-25 11:16 MDT (History)
CC List:	1 user (show)

See Also:
Site:	DownUnder GeoSolutions
Alineos Sites:	---
Atos/Eviden Sites:	---
Confidential Site:	---
Coreweave sites:	---
Cray Sites:	---
DS9 clusters:	---
HPCnow Sites:	---
HPE Sites:	---
IBM Sites:	---
NOAA SIte:	---
NoveTech Sites:	---
Nvidia HWinf-CS Sites:	---
OCF Sites:	---
Recursion Pharma Sites:	---
SFW Sites:	---
SNIC sites:	---
Tzag Elita Sites:	---
Linux Distro:	---
Machine Name:
CLE Version:
Version Fixed:
Target Release:	---
DevPrio:	---
Emory-Cloud Sites:	---

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this ticket.

Description paull 2016-08-12 07:54:51 MDT

Hi,

I would like to know if there was an option to limit the amount of nodes for a job array used at one time?

Ex: Job 1111_[1-100], max nodes used at 1 time = 10. Therefore, at max, while this array is in queue, only 10/100 jobs are running.

Submit option or update to job after submission? I would like the user to be able to control this.

Thanks,

Paul

Comment 3 Moe Jette 2016-08-12 08:24:58 MDT

Would a mechanism to limit the number of tasks from a job array that can be running at any time be sufficient? Relevant documentation below:

"A maximum number of simultaneously running tasks from the job array may be specified using a "%" separator. For example "--array=0-15%4" will limit the number of simultaneously running tasks from this job array to 4."

This count can be altered on job already submitted using the scontrol command. for example: scontrol update jobid=1234 ArrayTaskThrottle=4
ArrayTaskThrottle=<count>
    Speciify the maximum number of tasks in a job array that can execute at the same time. Set the count to zero in order to eliminate any limit. The task throttle count for a job array is reported as part of its ArrayTaskId field, preceded with a percent sign. For example "ArrayTaskId=1-10%2" indicates the maximum number of running tasks is limited to 2.

Comment 5 paull 2016-08-12 10:40:48 MDT

Hi Moe,

Thanks for your response. This may be exactly what the doctor ordered, but I will follow up after getting the green light from our team. 

Thanks,

Paul

Also, would it be possible to get some insight on ticket #2965?

Comment 6 Moe Jette 2016-08-15 08:23:05 MDT

(In reply to paull from comment #5)
> Also, would it be possible to get some insight on ticket #2965?

The last comment there was from Tim asking for some logs...

Comment 7 paull 2016-08-15 10:03:39 MDT

Moe,

Thanks for your help. I don't have an update as of yet on if this is what they team needs or not. As soon as I do I will update this ticket.

Thanks!

Paul

Comment 8 paull 2016-08-25 11:16:05 MDT

Moe,

Since we will be updating soon, I don't see a need for this to stay open. The solution you provided has been helpful.

Thanks!

Paul