Ticket 4688 - Task-Specific Dependency for Chained Job Arrays?
Summary: Task-Specific Dependency for Chained Job Arrays?
Status: RESOLVED INVALID
Alias: None
Product: Slurm
Classification: Unclassified
Component: User Commands (show other tickets)
Version: 17.02.0
Hardware: Linux Linux
: 6 - No support contract
Assignee: Jacob Jenson
QA Contact:
URL:
Depends on:
Blocks:
 
Reported: 2018-01-25 16:08 MST by Lyn
Modified: 2018-01-25 17:05 MST (History)
0 users

See Also:
Site: -Other-
Alineos Sites: ---
Atos/Eviden Sites: ---
Confidential Site: ---
Coreweave sites: ---
Cray Sites: ---
DS9 clusters: ---
HPCnow Sites: ---
HPE Sites: ---
IBM Sites: ---
NOAA SIte: ---
OCF Sites: ---
Recursion Pharma Sites: ---
SFW Sites: ---
SNIC sites: ---
Linux Distro: ---
Machine Name: AWS
CLE Version:
Version Fixed:
Target Release: ---
DevPrio: ---
Emory-Cloud Sites: ---


Attachments

Note You need to log in before you can comment on or make changes to this ticket.
Description Lyn 2018-01-25 16:08:00 MST
Hi SchedMD Crew,

Happy 2018.

I have a use case (at an unsupported site) for something like the following:

sbatch --array=1-10000 array_job1.sh
Submitted batch job 72893
sbatch --array=1-10000 --depend=afterok:72893_%i array_job2.sh

So (hopefully obviously), what I'm wanting to do is have each array task N from array_job2 depend on the successful completion of array_job1 task N.

Feels like I've seen this request from others, but my search didn't yield anything useful. My read of the current job_array.html guide's dependencies section appears not to support that scenario. Am I reading it correctly? If yes, is there any plan for such an enhancement?

Thanks much for any feedback.

Best,
Lyn
Comment 1 Jacob Jenson 2018-01-25 16:24:01 MST
Hi Lyn, 

Slurm has this functionality. If this site wants to purchase a support contract then we can route this ticket to the support team and they can help you achieve the job dependency listed below. Otherwise all we can do is refer you to the documentation. 

Jacob
Comment 2 Lyn 2018-01-25 17:05:29 MST
Thanks for the quick reply, Jacob.

Just to clarify, the only job_array.html documented examples showing
dependency on a specific job-array task are cases where the dependent job
being submitted is a normal, non-array job, and the task IDs are
hard-coded, so they are not the same use case. (I've also looked at the
sbatch man page, in case the example was placed there instead, and found
nothing additional.) Given your feedback though, I've tried what would be
the natural, expected Slurm syntax of 'sbatch --array=1-10000
--depend=after:72893_%a array_job2.sh', and I get a job dependency error.
So perhaps there's a documentation gap here, if the functionality does
exist.

Again, thanks for your quick reply (and I understand that you'll be closing
this item).

All the best,
Lyn

On Thu, Jan 25, 2018 at 1:24 PM, <bugs@schedmd.com> wrote:

> *Comment # 1 <https://bugs.schedmd.com/show_bug.cgi?id=4688#c1> on bug
> 4688 <https://bugs.schedmd.com/show_bug.cgi?id=4688> from Jacob Jenson
> <jacob@schedmd.com> *
>
> Hi Lyn,
>
> Slurm has this functionality. If this site wants to purchase a support contract
> then we can route this ticket to the support team and they can help you achieve
> the job dependency listed below. Otherwise all we can do is refer you to the
> documentation.
>
> Jacob
>
> ------------------------------
> You are receiving this mail because:
>
>    - You reported the bug.
>
>