Ticket 20932

Summary: Help with installing helper files slurm_completion.sh and slurm.vim
Product: Slurm Reporter: Ole.H.Nielsen <Ole.H.Nielsen>
Component: User CommandsAssignee: Ben Roberts <ben>
Status: RESOLVED FIXED QA Contact:
Severity: 4 - Minor Issue    
Priority: --- CC: skyler
Version: 23.11.10   
Hardware: Linux   
OS: Linux   
Site: DTU Physics Alineos Sites: ---
Atos/Eviden Sites: --- Confidential Site: ---
Coreweave sites: --- Cray Sites: ---
DS9 clusters: --- HPCnow Sites: ---
HPE Sites: --- IBM Sites: ---
NOAA SIte: --- NoveTech Sites: ---
Nvidia HWinf-CS Sites: --- OCF Sites: ---
Recursion Pharma Sites: --- SFW Sites: ---
SNIC sites: --- Linux Distro: ---
Machine Name: CLE Version:
Version Fixed: 24.05.4, 24.11.0rc1 Target Release: ---
DevPrio: --- Emory-Cloud Sites: ---

Description Ole.H.Nielsen@fysik.dtu.dk 2024-09-16 07:05:18 MDT
At SLUG'24 we were told about the very nice helper files slurm_completion.sh and slurm.vim available from 
https://github.com/SchedMD/slurm/tree/master/contribs/slurm_completion_help
These scripts are built in the source folder .../contribs/slurm_completion_help/ but unfortunately not installed by any of the slurm-* RPMs.

We would like to make these helper files available to our users automatically.

For EL8/EL9 Linux this procedure for slurm_completion.sh works immediately:

$ dnf install bash-completion
$ cp .../contribs/slurm_completion_help/slurm_completion.sh /etc/bash_completion.d/

Can you kindly update the README.md to add this?

As for the slurm.vim helper file, I have only been able to make this work if the file is copied manually by each user like:

$ mkdir -p ~/.vim/after/syntax/sh/
$ cp .../contribs/slurm_completion_help/slurm.vim ~/.vim/after/syntax/sh/

I've tried to copy slurm.vim to various subfolders under /usr/share/vim/ but none of that seems to work :-(

Question: Can you suggest a way to make slurm.vim available to all users by installing it in some system directory (IMHO only on login nodes)?

Thanks,
Ole
Comment 2 Ben Roberts 2024-09-17 11:02:25 MDT
Hi Ole,

I'm glad you can make use of the slurm_completion script.  We updated that to include new options available to modern releases relatively recently.  We try to keep installation instructions generic enough to work with any flavor of Linux, so rather than including an example that uses 'dnf', I am proposing a patch that will update the instructions for the completion script with the following:
-----------------------------------------
 ### Instalation
 
+You must have the bash-completion package installed in order for this to work.
 Source the [slurm_completion.sh][slurm-completion] script. It is
 recommended to automate this by adding a bash sourcing line to your
-`.bashrc` or `.profile`. Alternatively, add it to `/etc/profile.d/`
-to automatically source it for login shells.
+`.bashrc` or `.profile`. Alternatively, you can copy the file to
+`/etc/profile.d/` to automatically source it for login shells.
 
 Additionally, there are a number of environment variables that can be set to
 alter/customize completion behavior. They are documented in
-----------------------------------------

Does that look like it would be enough to get you pointed in the right direction?  

I'll also point out that there was recently work done to make the rpms include the slurm_completion script.  This will be available with 24.11.  You can see details of this in the following commit.
https://github.com/SchedMD/slurm/commit/3b0496b21d559675363fc5acf8ec58f983753abc

The slurm syntax highlighting question you have is interesting though.  This syntax file was created in 2012 and hasn't really been maintained since then, so there are a lot of missing options and it only has the full option name and not the single letter alternate syntax.  I'm hesitant to update instructions for this to make someone think that it's up to date and ready to use.  I can open a ticket to have someone work on updating this, but it's not going to be a high priority.  

Regarding your question of where you can put the file to have it recognized by more than a single user, I found an article that said you can put it in the $VIMRUNTIME/syntax/ directory, but I wasn't able to get that to work in my testing.  To find the $VIMRUNTIME directory you can run ':echo $VIMRUNTIME' from inside vim for it to show you the directory.  It's possible that I was missing some step, but I didn't have time to work on it more.  For your reference, the question/answer I was looking at is here, but please keep in mind that this is outside the scope of our support and I'm including it as a reference you can use.
https://vi.stackexchange.com/questions/6602/personal-syntax-highlighting

I hope this helps.  Let me know if you think the proposed doc changes would help  you get it installed.

Thanks,
Ben
Comment 3 Ole.H.Nielsen@fysik.dtu.dk 2024-09-18 04:05:32 MDT
Hi Ben,

Thanks a lot for explaining about the slurm_completion script:

(In reply to Ben Roberts from comment #2)
> I'm glad you can make use of the slurm_completion script.  We updated that
> to include new options available to modern releases relatively recently.  We
> try to keep installation instructions generic enough to work with any flavor
> of Linux, so rather than including an example that uses 'dnf', I am
> proposing a patch that will update the instructions for the completion
> script with the following:
> -----------------------------------------
>  ### Instalation
>  
> +You must have the bash-completion package installed in order for this to
> work.
>  Source the [slurm_completion.sh][slurm-completion] script. It is
>  recommended to automate this by adding a bash sourcing line to your
> -`.bashrc` or `.profile`. Alternatively, add it to `/etc/profile.d/`
> -to automatically source it for login shells.
> +`.bashrc` or `.profile`. Alternatively, you can copy the file to
> +`/etc/profile.d/` to automatically source it for login shells.

I don't think this is the best approach for Slurm prior to 24.11 on EL8/EL9 systems where I propose that one should manually copy the file from the source folder contribs/slurm_completion_help/ to /etc/bash_completion.d/ 

> I'll also point out that there was recently work done to make the rpms
> include the slurm_completion script.  This will be available with 24.11. 
> You can see details of this in the following commit.
> https://github.com/SchedMD/slurm/commit/
> 3b0496b21d559675363fc5acf8ec58f983753abc

Thanks a lot for the commit to the 24.11 slurm-contribs (?) RPM to include a /usr/share/bash-completion/completions/slurm_completion.sh file and defining a new prerequisite package "bash-completion"!

I did a "diff" of slurm_completion.sh between 23.11.10 (our current version) and 24.05.3 and found a lot of changes.  

Question: Should we use the slurm_completion.sh from 24.05 also on 23.11 and older installations, or are there any incompatible changes?

Thanks,
Ole
Comment 4 Ole.H.Nielsen@fysik.dtu.dk 2024-09-18 04:19:22 MDT
Hi Ben,

Regarding "vim" syntax highlighting:

(In reply to Ben Roberts from comment #2)
> The slurm syntax highlighting question you have is interesting though.  This
> syntax file was created in 2012 and hasn't really been maintained since
> then, so there are a lot of missing options and it only has the full option
> name and not the single letter alternate syntax.  I'm hesitant to update
> instructions for this to make someone think that it's up to date and ready
> to use.  I can open a ticket to have someone work on updating this, but it's
> not going to be a high priority.  

I think this is a fair approach, given that slurm.vim was last updated 5 years ago, and that major changes may be required!

Please do open an internal low-priority ticket on this, since users of vim may really appreciate a syntax highlighting feature.

> Regarding your question of where you can put the file to have it recognized
> by more than a single user, I found an article that said you can put it in
> the $VIMRUNTIME/syntax/ directory, but I wasn't able to get that to work in
> my testing.  To find the $VIMRUNTIME directory you can run ':echo
> $VIMRUNTIME' from inside vim for it to show you the directory.  It's
> possible that I was missing some step, but I didn't have time to work on it
> more.  For your reference, the question/answer I was looking at is here, but
> please keep in mind that this is outside the scope of our support and I'm
> including it as a reference you can use.
> https://vi.stackexchange.com/questions/6602/personal-syntax-highlighting

Yeah, adding a globally available file to vim seems to be a tricky question, and I wasn't able to find any useful guidance on the internet.  If anyone might be going to work on the above mentioned ticket, it would be good to first solve the issue of adding a slurm.vim file which is available to all users by default.

Thanks,
Ole
Comment 5 Ben Roberts 2024-09-19 15:12:01 MDT
Hi Ole,

> I don't think this is the best approach for Slurm prior to 24.11 on
> EL8/EL9 systems where I propose that one should manually copy the file
> from the source folder contribs/slurm_completion_help/ to
> /etc/bash_completion.d/ 

I ran some tests and confirmed that you can either source the file or put it in /etc/bash_completion.d/.  Since both ways are valid I'll move forward with the proposed wording I have and hopefully it will help clarify how to deploy the completion script.

> Question: Should we use the slurm_completion.sh from 24.05 also on 23.11
> and older installations, or are there any incompatible changes?

There are changes to commands that are version specific.  I would recommend you use the completion script that comes with the version you are running to avoid running into cases where it auto-completes using syntax for a newer release.

> Please do open an internal low-priority ticket on this, since users of vim
> may really appreciate a syntax highlighting feature.

I have opened an internal ticket for this file to be updated.  I can't make any guarantee of when it will be updated, but the request is there to show that there is interest.  I noted in that ticket that we should also look into whether there is a way to make this available to all users without copying it to each user's home directory.

Thanks,
Ben
Comment 7 Ole.H.Nielsen@fysik.dtu.dk 2024-09-20 01:02:09 MDT
Hi Ben,

Thanks a lot for your support.  I think these features will be well documented in the next release.  Please close the present case.

Best regards,
Ole


(In reply to Ben Roberts from comment #5)
> Hi Ole,
> 
> > I don't think this is the best approach for Slurm prior to 24.11 on
> > EL8/EL9 systems where I propose that one should manually copy the file
> > from the source folder contribs/slurm_completion_help/ to
> > /etc/bash_completion.d/ 
> 
> I ran some tests and confirmed that you can either source the file or put it
> in /etc/bash_completion.d/.  Since both ways are valid I'll move forward
> with the proposed wording I have and hopefully it will help clarify how to
> deploy the completion script.
> 
> > Question: Should we use the slurm_completion.sh from 24.05 also on 23.11
> > and older installations, or are there any incompatible changes?
> 
> There are changes to commands that are version specific.  I would recommend
> you use the completion script that comes with the version you are running to
> avoid running into cases where it auto-completes using syntax for a newer
> release.
> 
> > Please do open an internal low-priority ticket on this, since users of vim
> > may really appreciate a syntax highlighting feature.
> 
> I have opened an internal ticket for this file to be updated.  I can't make
> any guarantee of when it will be updated, but the request is there to show
> that there is interest.  I noted in that ticket that we should also look
> into whether there is a way to make this available to all users without
> copying it to each user's home directory.
> 
> Thanks,
> Ben
Comment 9 Ben Roberts 2024-09-20 09:07:26 MDT
Thanks Ole.  I've committed the updates to the README file and the other ticket to update the slurm.vim file is still open.  Let us know if there is anything else we can do to help.

Thanks,
Ben