Hi Daniel, right now there's no partition/qos limit to accomplish this. We can mark it as a possible future enhancement if needed.

Daniel, note that the job_submit plugin has access to the job_desc->script field to discern interactive from non-interactive jobs. You could check that field to control the submissions to specific partitions as desired. Dunno exactly what you want to limit:

- "Number of interactive jobs submitted to specific partition, computing also past jobs"
- "Number of interactive jobs RUN/PEND to a specific partition at the same time"
- "Reject any submission to specific partition if job is interactive"
- ...?

We are trying to reject interactive job submission to one of our partitions.

Can you point me to documentation/examples on how to use the job_submit plugin?

Sure, this is the documentation:

http://slurm.schedmd.com/job_submit_plugins.html

You should configure the JobSubmitPlugins=lua in your slurm.conf. There are a few examples in the contribs/lua directory:

https://github.com/SchedMD/slurm/tree/master/contribs/lua

I think something in this direction is what you want:

--[[ Example Lua script ]]--

function slurm_job_submit( job_desc, part_list, submit_uid )
        local partition = "part1"
        if ((job_desc.script == nil or job_desc.script == '') and job_desc.partition == partition) then
                slurm.log_info("slurm_job_submit: interactive job submitted by user_id:%d to partition:%s rejected", job_desc.user_id, job_desc.partition)
                return slurm.FAILURE
        end
        return slurm.SUCCESS
end

function slurm_job_modify( job_desc, job_rec, part_list, modify_uid )
        return slurm.SUCCESS
end

slurm.log_info("initialized")
return slurm.SUCCESS

This is great info.  Only one thing is unclear for me. Is the job_submit plugin build as part of the default installation, or something I need to compile myself?

By default Slurm does not load any Job Submit Plugins. To accomplish this task, set JobSubmitPlugins=lua in slurm.conf and reconfigure it. Then Slurm will dynamically execute the lua script named "job_submit.lua" located in the default script directory (typically the subdirectory "etc" of the installation directory). Lua is an interpreted scripting language and needs no compilation after source edition.

Thanks.

FYI,
I just tried the script that comes with 16.05.10 and got this
slurmctld[4133]: error: lua: /etc/slurm/job_submit.lua: /etc/slurm/job_submit.lua:5: ')' expected near '!'

It appears that != in lua is "~=" http://lua-users.org/wiki/LuaTypesTutorial

diff job_submit.lua job_submit.lua-new 
5c5
<   if ((job_desc.script == nil or job_desc.script == '') and job_desc.partition != partition) then
---
>   if ((job_desc.script == nil or job_desc.script == '') and job_desc.partition ~= partition) then