Ticket 4688

Summary:	Task-Specific Dependency for Chained Job Arrays?
Product:	Slurm	Reporter:	Lyn <schedulerqueen>
Component:	User Commands	Assignee:	Jacob Jenson <jacob>
Status:	RESOLVED INVALID	QA Contact:
Severity:	6 - No support contract
Priority:	---
Version:	17.02.0
Hardware:	Linux
OS:	Linux
Site:	-Other-	Alineos Sites:	---
Atos/Eviden Sites:	---	Confidential Site:	---
Coreweave sites:	---	Cray Sites:	---
DS9 clusters:	---	HPCnow Sites:	---
HPE Sites:	---	IBM Sites:	---
NOAA SIte:	---	OCF Sites:	---
Recursion Pharma Sites:	---	SFW Sites:	---
SNIC sites:	---	Linux Distro:	---
Machine Name:	AWS	CLE Version:
Version Fixed:		Target Release:	---
DevPrio:	---	Emory-Cloud Sites:	---

Description Lyn 2018-01-25 16:08:00 MST

Hi SchedMD Crew,

Happy 2018.

I have a use case (at an unsupported site) for something like the following:

sbatch --array=1-10000 array_job1.sh
Submitted batch job 72893
sbatch --array=1-10000 --depend=afterok:72893_%i array_job2.sh

So (hopefully obviously), what I'm wanting to do is have each array task N from array_job2 depend on the successful completion of array_job1 task N.

Feels like I've seen this request from others, but my search didn't yield anything useful. My read of the current job_array.html guide's dependencies section appears not to support that scenario. Am I reading it correctly? If yes, is there any plan for such an enhancement?

Thanks much for any feedback.

Best,
Lyn

Comment 1 Jacob Jenson 2018-01-25 16:24:01 MST

Hi Lyn, 

Slurm has this functionality. If this site wants to purchase a support contract then we can route this ticket to the support team and they can help you achieve the job dependency listed below. Otherwise all we can do is refer you to the documentation. 

Jacob

Comment 2 Lyn 2018-01-25 17:05:29 MST

Thanks for the quick reply, Jacob.

Just to clarify, the only job_array.html documented examples showing
dependency on a specific job-array task are cases where the dependent job
being submitted is a normal, non-array job, and the task IDs are
hard-coded, so they are not the same use case. (I've also looked at the
sbatch man page, in case the example was placed there instead, and found
nothing additional.) Given your feedback though, I've tried what would be
the natural, expected Slurm syntax of 'sbatch --array=1-10000
--depend=after:72893_%a array_job2.sh', and I get a job dependency error.
So perhaps there's a documentation gap here, if the functionality does
exist.

Again, thanks for your quick reply (and I understand that you'll be closing
this item).

All the best,
Lyn

On Thu, Jan 25, 2018 at 1:24 PM, <bugs@schedmd.com> wrote:

> *Comment # 1 <https://bugs.schedmd.com/show_bug.cgi?id=4688#c1> on bug
> 4688 <https://bugs.schedmd.com/show_bug.cgi?id=4688> from Jacob Jenson
> <jacob@schedmd.com> *
>
> Hi Lyn,
>
> Slurm has this functionality. If this site wants to purchase a support contract
> then we can route this ticket to the support team and they can help you achieve
> the job dependency listed below. Otherwise all we can do is refer you to the
> documentation.
>
> Jacob
>
> ------------------------------
> You are receiving this mail because:
>
>    - You reported the bug.
>
>