running your 666bbe05fe2 + our usual modifications Maybe because array tasks are in a mixture of states? # squeue -u davef PARTITION PRIORITY NAME USER ST TIME NODES NODELIST(REASON JOBID teambond 400 dp_hal_srme_match davef CG 0:00 64 clus1149 336247_21 teambond 400 dp_hal_srme_match davef PD 0:00 1 (Priority) 336247_[13,15-17,19-20,22-816] teambond 0 dp_hal_srme_match davef SE 0:00 1 (JobHeldUser) 336247_1 teambond 0 dp_hal_srme_match davef SE 0:00 1 (JobHeldUser) 336247_2 teambond 0 dp_hal_srme_match davef SE 0:00 1 (JobHeldUser) 336247_3 teambond 0 dp_hal_srme_match davef SE 0:00 1 (JobHeldUser) 336247_4 teambond 0 dp_hal_srme_match davef SE 0:00 1 (JobHeldUser) 336247_5 teambond 0 dp_hal_srme_match davef SE 0:00 1 (JobHeldUser) 336247_6 teambond 0 dp_hal_srme_match davef SE 0:00 1 (JobHeldUser) 336247_7 teambond 0 dp_hal_srme_match davef SE 0:00 1 (JobHeldUser) 336247_8 teambond 0 dp_hal_srme_match davef SE 0:00 1 (JobHeldUser) 336247_9 teambond 0 dp_hal_srme_match davef SE 0:00 1 (JobHeldUser) 336247_10 teambond 0 dp_hal_srme_match davef SE 0:00 1 (JobHeldUser) 336247_11 teambond 0 dp_hal_srme_match davef SE 0:00 1 (JobHeldUser) 336247_12 teambond 0 dp_hal_srme_match davef SE 0:00 1 (JobHeldUser) 336247_14 teambond 0 dp_hal_srme_match davef SE 0:00 1 (JobHeldUser) 336247_18 # scontrol release job=336247 # squeue -u davef PARTITION PRIORITY NAME USER ST TIME NODES NODELIST(REASON JOBID teambond 400 dp_hal_srme_match davef CG 0:00 64 clus1149 336247_21 teambond 400 dp_hal_srme_match davef PD 0:00 1 (Priority) 336247_[13,15-17,19-20,22-816] teambond 0 dp_hal_srme_match davef SE 0:00 1 (JobHeldUser) 336247_1 teambond 0 dp_hal_srme_match davef SE 0:00 1 (JobHeldUser) 336247_2 teambond 0 dp_hal_srme_match davef SE 0:00 1 (JobHeldUser) 336247_3 teambond 0 dp_hal_srme_match davef SE 0:00 1 (JobHeldUser) 336247_4 teambond 0 dp_hal_srme_match davef SE 0:00 1 (JobHeldUser) 336247_5 teambond 0 dp_hal_srme_match davef SE 0:00 1 (JobHeldUser) 336247_6 teambond 0 dp_hal_srme_match davef SE 0:00 1 (JobHeldUser) 336247_7 teambond 0 dp_hal_srme_match davef SE 0:00 1 (JobHeldUser) 336247_8 teambond 0 dp_hal_srme_match davef SE 0:00 1 (JobHeldUser) 336247_9 teambond 0 dp_hal_srme_match davef SE 0:00 1 (JobHeldUser) 336247_10 teambond 0 dp_hal_srme_match davef SE 0:00 1 (JobHeldUser) 336247_11 teambond 0 dp_hal_srme_match davef SE 0:00 1 (JobHeldUser) 336247_12 teambond 0 dp_hal_srme_match davef SE 0:00 1 (JobHeldUser) 336247_14 teambond 0 dp_hal_srme_match davef SE 0:00 1 (JobHeldUser) 336247_18
I see! The problem is actually one of syntax. "scontrol release job=336247" does nothing, silently. "scontrol release 336247" works.
The syntax is now consistent allowing 3 types of specification: scontrol release jobid=150698 scontrol requeuehold state=specialexit jobid=150698 scontrol release job=150698 scontrol requeuehold state=specialexit job=150698 scontrol release 150698 scontrol requeuehold state=specialexit 150698 scontrol requeue jobid=150698 scontrol requeue job=150698 scontrol requeue 150698 Thanks, David
Forgot to mention the commit: b1534743250. David