Ticket 19912

Summary: MaxRss calculation in Accounting
Product: Slurm Reporter: j.naumann
Component: AccountingAssignee: Jacob Jenson <jacob>
Status: OPEN --- QA Contact:
Severity: 6 - No support contract    
Priority: ---    
Version: - Unsupported Older Versions   
Hardware: Linux   
OS: Linux   
Site: -Other- Alineos Sites: ---
Atos/Eviden Sites: --- Confidential Site: ---
Coreweave sites: --- Cray Sites: ---
DS9 clusters: --- HPCnow Sites: ---
HPE Sites: --- IBM Sites: ---
NOAA SIte: --- NoveTech Sites: ---
Nvidia HWinf-CS Sites: --- OCF Sites: ---
Recursion Pharma Sites: --- SFW Sites: ---
SNIC sites: --- Tzag Elita Sites: ---
Linux Distro: --- Machine Name:
CLE Version: Version Fixed:
Target Release: --- DevPrio: ---
Emory-Cloud Sites: ---

Description j.naumann 2024-05-17 04:47:46 MDT
Dear slurm support,

I encountered a weird behavior about the MaxRss field in the slurm account. For the concrete example sacct reports [1] a MaxRss of roughly 55.7G while the job had a mem limit of 32G. If one measures the MaxRss of the job script outside of slurm with the GNU time-tool (/usr/bin/time -v ...) you get a MaxRss of 28G which is about the half of the size sacct reports.  

Therefore, I want to ask you how the MaxRss field of sacct is calculated? Is there a factor inside the slurm value which one has to consider? How can the fact be explained that slurm reports a MaxRss of 56G which would be clear above the mem limit of 32G which is enforced by the slurm cgroups plugin?

Thank you very much!
Jan Naumann 

[1]

$ sacct  -j 273186_0 -o JobId,MaxVMSize,MaxVMSizeNode,MaxVMSizeNode,AveVMSize,MaxRSS,MaxRSSNode,MaxRSSTask,AveRSS,AllocCPUS,ReqMem,NTasks,NNodes
JobID         MaxVMSize  MaxVMSizeNode  MaxVMSizeNode  AveVMSize     MaxRSS MaxRSSNode MaxRSSTask     AveRSS  AllocCPUS     ReqMem   NTasks   NNodes 
------------ ---------- -------------- -------------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- -------- -------- 
273186_0                                                                                                              4        32G                 1 
273186_0.ba+  58504736K           x176           x176  58504736K  58362584K       x176          0  58362584K          4                   1        1 
273186_0.ex+        60K           x176           x176        60K        60K       x176          0        60K          4                   1        1