Ticket 12047

Summary:	How to limit total number of CPUs used by an account, same limit for all accounts?
Product:	Slurm	Reporter:	David Chin <dwc62>
Component:	Limits	Assignee:	Scott Hilton <scott>
Status:	RESOLVED INFOGIVEN	QA Contact:
Severity:	3 - Medium Impact
Priority:	---	CC:	scott
Version:	20.02.7
Hardware:	Linux
OS:	Linux
Site:	Drexel	Alineos Sites:	---
Atos/Eviden Sites:	---	Confidential Site:	---
Coreweave sites:	---	Cray Sites:	---
DS9 clusters:	---	HPCnow Sites:	---
HPE Sites:	---	IBM Sites:	---
NOAA SIte:	---	OCF Sites:	---
Recursion Pharma Sites:	---	SFW Sites:	---
SNIC sites:	---	Linux Distro:	---
Machine Name:		CLE Version:
Version Fixed:		Target Release:	---
DevPrio:	---	Emory-Cloud Sites:	---

Description David Chin 2021-07-15 14:55:29 MDT

Hi,

We would like to impose a limit (QOS) on the number of CPUs used in the "def" partition by any account, where all accounts have this same limit.

In our case, we want every account to be able to consume no more than 2,352 CPUs simultaneously in the "def" partition.

I tried defining this:

$ sacctmgr show qos  format=name,priority,grptres,usagefactor
      Name   Priority       GrpTRES UsageFactor
---------- ---------- ------------- -----------
    normal          0                  1.000000
       def          0      cpu=2352    1.000000

And then applying the "def" QOS to the "def" partition, but that seemed to limit the total number of CPUs used on the partition aggregating the usage of all accounts.

How do we accomplish this without applying a QOS to each account separately?

Thanks,
    Dave

Comment 1 Scott Hilton 2021-07-15 16:35:21 MDT

Dave,

I believe you are looking for MaxTRESPerAccount
(see https://slurm.schedmd.com/resource_limits.html)

Run something like this
>sacctmgr update qos def set MaxTRESPerAccount=cpu=2352

Then assign the qos to the partition as you planned.

-Scott

Comment 2 David Chin 2021-07-15 17:55:16 MDT

Hi Scott:

Thanks much.

I've just done that, and I'll do a bit of testing.

--DAve

Comment 3 Scott Hilton 2021-07-19 10:46:10 MDT

David, 

Did that work for you?

-Scott

Comment 4 David Chin 2021-07-19 10:56:06 MDT

Hi, Scott:

Yes, that did.

Thanks very much. Please close this ticket.

Regards,
    Dave

Comment 5 Scott Hilton 2021-07-20 15:55:37 MDT

Closing Ticket