I want to modify cgroup.conf, but I'm not sure how this will affect running jobs. I have always made changes to cgroup.conf during maintenance periods. For example, now I have in cgroup.conf: CgroupAutomount=yes ConstrainCores=yes ConstrainRAMSpace=yes ConstrainDevices=yes and am considering going to: ConstrainKmemSpace=yes ConstrainCores=yes CgroupAutomount=yes ConstrainRAMSpace=yes ConstrainSwapSpace=yes TaskAffinity=no MaxSwapPercent=10 Can I do this with running jobs? If so, can I then just execute an "scontrol reconfig" or would I need to restart slurmd / slurmctld?
TaskAffinity was removed starting in 21.08.2 and has been obsolete for the last several releases. It is recommended to use TaskPlugin to enable these and stack both cgroup and affinity. >TaskPlugin=task/cgroup,task/affinity If you want to disable affinity you would remove that plugin from the TaskPlugin line, although I am curious why you want to do this. >TaskPlugin=task/cgroup > Can I do this with running jobs? If so, can I then just execute an "scontrol > reconfig" or would I need to restart slurmd / slurmctld? Modifying these setting should work with a reconfigure since you already use the cgroups plugin. You would run into problems if you tried to enable/disable the entirety of the plugin without a restart. I am assuming here but it sounds like you may also have task/affinity enabled. In this case, you are able to remove task/affinity but you will need to restart the slurmd's. [1] https://github.com/SchedMD/slurm/blob/master/NEWS#L1008 [2] https://slurm.schedmd.com/cgroups.html#task
Sorry, that was a cut-and-paste error, I was going the other direction. From: ConstrainKmemSpace=yes ConstrainCores=yes CgroupAutomount=yes ConstrainRAMSpace=yes ConstrainSwapSpace=yes TaskAffinity=no MaxSwapPercent=10 to: CgroupAutomount=yes ConstrainCores=yes ConstrainRAMSpace=yes ConstrainDevices=yes I really wanted to get rid of the "Swap" stuff and was worried running jobs might die during a reconfig. But I just did it and everything appears to have gone fine.
> I really wanted to get rid of the "Swap" stuff and was worried running jobs might die during a reconfig. But I just did it and everything appears to have gone fine. Yes, that is fine as well. The limits/cgroups are instantiated during step creation. Running jobs will clean these setting up once they finish.
