Hello Tim, Minor issue: one of our monitoring script broke today after migrating from Slurm 17 to 18 on Sherlock. It looks like the -o '%b' (GRES) format option of squeue doesn't add a trailing space anymore. See below: $ squeue -rh -o '%g %u %P %b %T %C %D %R' -j 33003323 rzia emmagg rzia N/ARUNNING 1 1 sh-106-12 $ squeue -rh -o '%g %u %P %b %T %C %D %R' -j 33236568 epop gabourie gpu gpu:1RUNNING 1 1 sh-112-04 Our current workaround is to specify a format length which is long enough to cover all gres types on the cluster: $ squeue -rh -o '%g %u %P %10b %T %C %D %R' -j 33236568 epop gabourie gpu gpu:1 RUNNING 1 1 sh-112-04 $ squeue -rh -o '%g %u %P %10b %T %C %D %R' -j 33003323 rzia emmagg rzia N/A RUNNING 1 1 sh-106-12 Thanks! Stephane
Supplied patch in bug report 6141 fixes the issue with N/A printing, curious if fixes actual issue when TRES is present.
*** Ticket 6141 has been marked as a duplicate of this ticket. ***
Created attachment 8496 [details] squeue tres print fix Trey Dockendorf (tdockendorf@osc.edu) fix / contribution from 6141.
Created attachment 8497 [details] Fixes missing suffixes for TRES.
Hi Stephane, Trey, > Supplied patch in bug report 6141 fixes the issue with N/A printing, curious > if fixes actual issue when TRES is present. Yes, the patch fixes both, the trailing spaces and the "DELIM" issues. In fact, anything after the %b and before any other specified was actually ignored. Trey's patch has been merged and properly authored at: https://github.com/SchedMD/slurm/commit/9b0399b84373b3360a61b68c6725b0770c7f9dab Thanks Trey! Albert
*** Ticket 6144 has been marked as a duplicate of this ticket. ***
Thanks!
We are currently on Slurm 18.08.5 and we are still seeing the issue when using the SQUEUE_FORMAT2/-O/--Format formatting options. -o works as expected after the fix. # -o, works as expected $ squeue -o "%i,%j" NAME,JOBID mesau,32994447_[77-399%20] schedule-,30983558 schedule-,30983557 schedule-,30983556 LFeHNTMSFeL_-1_m1_BP86_def2-TZVP_OptHess.mpi,32999232 LFeHNTMSFeL_-1_m9_BP86_def2-TZVP_OptHess.mpi,32999246 LFeHCTMSFeL_-1_m2_BP86_def2-TZVP_OptHess.mpi,32999250 LFeHCTMSFeL_-1_m10_BP86_def2-TZVP_OptHess.mpi,32999251 LFeSCHTMS_0_m1_BP86_def2-TZVP_OptHess_NotTight.mpi,33001227 # -O, missing delimiter $ squeue -O "name,jobid" NAME JOBID mesau 32994447 schedule- 30983558 schedule- 30983557 schedule- 30983556 LFeHNTMSFeL_-1_m1_BP32999232 LFeHNTMSFeL_-1_m9_BP32999246 LFeHCTMSFeL_-1_m2_BP32999250 LFeHCTMSFeL_-1_m10_B32999251 LFeSCHTMS_0_m1_BP86_33001227 # -o, works as expected $ squeue -o "%i %j" NAME JOBID mesau 32994447_[77-399%20] schedule- 30983558 schedule- 30983557 schedule- 30983556 LFeHNTMSFeL_-1_m1_BP86_def2-TZVP_OptHess.mpi 32999232 LFeHNTMSFeL_-1_m9_BP86_def2-TZVP_OptHess.mpi 32999246 LFeHCTMSFeL_-1_m2_BP86_def2-TZVP_OptHess.mpi 32999250 LFeHCTMSFeL_-1_m10_BP86_def2-TZVP_OptHess.mpi 32999251 LFeSCHTMS_0_m1_BP86_def2-TZVP_OptHess_NotTight.mpi 33001227 # -O, fails completely with any delimiter other than "," $ squeue -O "name jobid" squeue: error: Invalid job format specification: name jobid
Hi Kaylea, > We are currently on Slurm 18.08.5 and we are still seeing the issue when > using the SQUEUE_FORMAT2/-O/--Format formatting options. -o works as > expected after the fix. Actually what you are reporting in comment 14 is the expected behavior (or at least a known limitation) in the sense that as you can see in the manpage one of the differences between -o and -O is that -O "Requests a comma separated list of job information to be displayed". So, for -O we can only specify the desired fields, but not any other delimiter of character in the list of desired fields. Just each field, justification and size. The same happens with sinfo's pair of options. I can see that you may want this limitation to be removed to allow the same level of flexibility/delimiters between -o and -O though, so if you are still interested and because it's not actually related to the original bug, please file a new bug referencing your comment here as 5-Enhancement and there we will discuss it further. Is that fine for you? Thanks, Albert
Sounds good, will do!
Closing as fixed again.