Dear SchedMD support, we have trouble getting the fairshare quite the way we want it. Our situation: 1. Most of the nodes we host are our own and are shared among all users. 2. Some departments buy their own nodes which they then exclusively use (they're still allowed to use our nodes) 3. Some departments buy nodes to be integrated among our nodes for everyone in exchange of a higher fairshare. The first two cases are fine, a problem comes in the third category. We currently have an "all" account, where all the users are in. Most users are just normal users of category 1 and they get 1000 shares. Users of category 2 get 1000 shares in all, but also an extra account to use on their own nodes (in their own partition). For the third category, we calculate how much their additional nodes add to our cluster and add that to their shares, e.g. if the department's nodes add 5% to our cluster, they will get 1050 shares. The problem with this setup is that the more people work in a department, the more users get higher fairshare, leading to an advantage for bigger departments. What we actually want is a setup where users of the third category share a pool of extra shares and once this pool is depleted, they use their default 1000 shares that everybody has. Is something like that possible with fairshare in Slurm? Best, Marcus
Marcus, There are two fairshare algorithms: The fair tree algorithm and the classic. The classic algorithm is only active if you have PriorityFlags=NO_FAIR_TREE in you slurm.conf. I will assume you are using the current default algorithm which is fair tree. For the fair tree, fair share is calculated recursively on each account level. Shares only matter in the ratio between sibling accounts. (https://slurm.schedmd.com/fair_tree.html) For the department to have 5% of the shares you could do something like this: >sacctmgr show assoc tree format=cluster,account,user,share,parentname > Cluster Account User Share Par Name >---------- -------------------- ---------- --------- ---------- > dragon root 1 > dragon all 95 root > dragon all scott 1000 > dragon all marcus 1000 > dragon department 5 root > dragon department scott 1 Since 'all' and 'department' are on the same level their shares interact. Since the children of 'all' each have 1000 shares %50 of whatever is allotted to 'all'. User scott would have two accounts to think about. He would have to decide which account to use for any given job. Sadly, there isn't a way to automatically draw from the ideal account. A user could run sshare to see which account has more fairshare at any time before running a job. If you don't want your users to have to think about which accout to use based on fairshare, you could remove the 'all' account and split it among departments directly: >sacctmgr show assoc tree format=cluster,account,user,share,parentname > Cluster Account User Share Par Name >---------- -------------------- ---------- --------- ---------- > dragon root 1 > dragon math 100 root > dragon math tom 2 > dragon math bob 1 > dragon engineering 110 root > dragon engineering scott 9534 > dragon physics 100 root > dragon physics marcus 1 In this example, Engineering might deserve more and so gets 110 shares out of 310. Tom and Bob each use the the math department but Tom might get more shares in that department. Scott and Marcus each have %100 in their department. Do you have any questions or concerns? Anything I should explain more? -Scott
If you have any follow up questions feel free to reopen this ticket. -Scott
