*** Ticket 8808 has been marked as a duplicate of this ticket. ***

Valantis,

>In another ticket someone from SchedMD suggested to do this,
Could you please reference the ticket? I think that we need to fully understand what are you trying to achieve.

cheers,
Marcin

(In reply to Marcin Stolarek from comment #2)
> Valantis,
> 
> >In another ticket someone from SchedMD suggested to do this,
> Could you please reference the ticket? I think that we need to fully
> understand what are you trying to achieve.
> 
> cheers,
> Marcin

Hi Marcin!

Sorry but I cannot search my old tickets in your bugzilla... Why is that so? It's not the first time I complained about this :P Can you search my old tickets about option --hint or SLURM_HINT?

For some reasons, no need to explain in full details here why, we need to have an output env var in job's environment for option `--hint` or its input env var SLURM_HINT. We just want to export e.g. SLURM_HINT in job's env.

In the other ticket I have been told that I can do this with cli_filter. But I cannot find an easy way... As I explained the only working solution I can think of is yhe combination of a spank plugin and the lua cli_filter (which we have already).

Valantis,

Do you mean Bug 8554?

The --hint is really tricky in terms of how it works, it's really kind of meta option that just sets other options. You may want to check some explanation in Bug 8147 comment 15. The options set by specific values of --hint can be found in src/common/proc_args.c verify_hint function.

>For some reasons, no need to explain in full details here[...]
However, I cannot recommend manipulating SLURM_* variables because we don't know the case a code snippet you're looking for can look like:


>extern int pre_submit(slurm_opt_t *opt, int offset)                              
>{                                                                                
>        char *name = NULL;                                                       
>        char *value = NULL;                                                      
>        size_t st = 0;                                                           
>                                                                                 
>        st = 0;                                                                  
>        while (slurm_option_get_next_set(opt, &name, &value, &st)) {             
>                if(!xstrncmp(name,"hint",4)) {                                   
>                        setenv("MY_HINT", value, 1);                              
>                }                                                                
>                xfree(name);                                                     
>                xfree(value);                                                    
>        }                                                                        
>        return SLURM_SUCCESS;                                                    
>}      

The easiest way to use it on 19.05 is probably to update the code in cli_fliter/none plugin and run make/make install, add CliFilterPlugins=none to slurm.conf. The result looks like:
# scontrol show config | grep CliFi
CliFilterPlugins        = none
# env | grep MY_HINT
# srun -v  --hint=compute_bound /bin/bash -c "env" |& grep HIN
MY_HINT=compute_bound

Is there anything else I can help you with?

cheers,
Marcin

(In reply to Marcin Stolarek from comment #4)
> Valantis,
> 
> Do you mean Bug 8554?
> 
> The --hint is really tricky in terms of how it works, it's really kind of
> meta option that just sets other options. You may want to check some
> explanation in Bug 8147 comment 15. The options set by specific values of
> --hint can be found in src/common/proc_args.c verify_hint function.
> 
> >For some reasons, no need to explain in full details here[...]
> However, I cannot recommend manipulating SLURM_* variables because we don't
> know the case a code snippet you're looking for can look like:
> 
> 
> >extern int pre_submit(slurm_opt_t *opt, int offset)                              
> >{                                                                                
> >        char *name = NULL;                                                       
> >        char *value = NULL;                                                      
> >        size_t st = 0;                                                           
> >                                                                                 
> >        st = 0;                                                                  
> >        while (slurm_option_get_next_set(opt, &name, &value, &st)) {             
> >                if(!xstrncmp(name,"hint",4)) {                                   
> >                        setenv("MY_HINT", value, 1);                              
> >                }                                                                
> >                xfree(name);                                                     
> >                xfree(value);                                                    
> >        }                                                                        
> >        return SLURM_SUCCESS;                                                    
> >}      
> 
> The easiest way to use it on 19.05 is probably to update the code in
> cli_fliter/none plugin and run make/make install, add CliFilterPlugins=none
> to slurm.conf. The result looks like:
> # scontrol show config | grep CliFi
> CliFilterPlugins        = none
> # env | grep MY_HINT
> # srun -v  --hint=compute_bound /bin/bash -c "env" |& grep HIN
> MY_HINT=compute_bound
> 
> Is there anything else I can help you with?
> 
> cheers,
> Marcin

Yes this is the ticket and I couldn't find it. How can I search my old resolved tickets??? This is weird, I don't know what filters to put.

Also, we could name that env var MY_HINT or something else it's not that important.

We currently use the lua cli_filter, but it seems that it is impossible to export env vars with that plugin, right?

The only way is to use the none, or create another one..

Created attachment 13685 [details]
search_by_reporter_email

Valantis,

>Yes this is the ticket and I couldn't find it. How can I search my old resolved tickets??? This is weird, I don't know what filters to put.

You can find the form I used to find your cases in attached screenshot it wasn't that many so I did ctrl+f --hint after that.

>We currently use the lua cli_filter, but it seems that it is impossible to export env vars with that plugin, right?
Did you migrate to 20.02(the ticket is for 19.05 where we didn't have cli_filter/lua) or do you have your own cli_filter/lua?

cheers,
Marcin

(In reply to Marcin Stolarek from comment #6)
> Created attachment 13685 [details]
> search_by_reporter_email
> 
> Valantis,
> 
> >Yes this is the ticket and I couldn't find it. How can I search my old resolved tickets??? This is weird, I don't know what filters to put.
> 
> You can find the form I used to find your cases in attached screenshot it
> wasn't that many so I did ctrl+f --hint after that.
> 
> >We currently use the lua cli_filter, but it seems that it is impossible to export env vars with that plugin, right?
> Did you migrate to 20.02(the ticket is for 19.05 where we didn't have
> cli_filter/lua) or do you have your own cli_filter/lua?
> 
> cheers,
> Marcin

We had just backported/patched the lua cli_filter plugin on our 19.05 from the actual PR of its original creator (the patch was provided to us). If with current cli_filter/lua on 20.02 is possible to export env vars then I can check if it's easy to backport it.

OK I just checked, it is the same plugin more or less, so I guess it is not possible to export env vars from cli_filter/lua..

I think I will modify the cli_filter/none as it was suggested and use both plugins ;)

A devel question: in pre_submit() of cli_filter how can I check if SLURM_HINT input env var is defined and get it's contents?

Thank you!

Well.. if you have the same code, it should work. I thought that Lua is not an option because of Slurm version. You can use cli_filter.lua like:
>local M = reqlocal M = require 'posix'
>function slurm_cli_setup_defaults(options, cli_type)
>        return slurm.SUCCESS
>end
>function slurm_cli_post_submit(options, cli_type)
>        return slurm.SUCCESS
>end
>function slurm_cli_pre_submit(options, cli_type)
>        if options['hint'] ~= nil then
>                M.setenv("MY_HINT",options["hint"]);
>        end
>        return slurm.SUCCESS
>end

results:
# scontrol show config | grep CliFi
CliFilterPlugins        = none
# srun --hint=compute_bound "env" | grep -i hint
MY_HINT=compute_bound
# srun  "env" | grep -i hint

You just have to use a library providing posix bindings, since setenv is not ANSI C.

cheers,
Marcin

PS. I just found that first line of script got broken during copy&paste it should be:
>local M = require 'posix'

>A devel question: in pre_submit() of cli_filter how can I check if SLURM_HINT input env var is defined and get it's contents?

You can use os.getenv, just check if you really have to, for instance:

>local os = require 'os'
>function slurm_cli_pre_submit(options, cli_type)
>        local asd = os.getenv("SLURM_HINT")
>        slurm.log_info("%s",asd);
>        if options['hint'] ~= nil then
>                M.setenv("MY_HINT",options["hint"]);
>        end
>        return slurm.SUCCESS
>end


# env | grep HINT
SLURM_HINT=compute_bound
# srun env | grep -i hint
srun: lua: compute_bound
SLURM_HINT=compute_bound
MY_HINT=compute_bound

cheers,
Marcin

(In reply to Marcin Stolarek from comment #10)
> PS. I just found that first line of script got broken during copy&paste it
> should be:
> >local M = require 'posix'

OK I see... That was my original problem and this is why I created the ticket ;) I can solve it easily with lua I just need a setenv lua library with posix bindings... (e..g in centos 7 I will install lua-posix, right?)

Thanks for the fix, I was about to ask you about that line :P

Thank you very much Marcin!

Let's keep this ticket open until I finish with the plugin..

>(e..g in centos 7 I will install lua-posix, right?)
lua-posix from epel should work for you on CentOS7

cheers,
Marcin

OK it was tested and works as expected! Thank you Marcin :)

Now you can close this ticket.