Ticket 15423

Summary: slurmddebug setting
Product: Slurm Reporter: Tom Skipper <ts840>
Component: ConfigurationAssignee: Ben Roberts <ben>
Status: RESOLVED INFOGIVEN QA Contact:
Severity: 4 - Minor Issue    
Priority: ---    
Version: - Unsupported Older Versions   
Hardware: Linux   
OS: Linux   
Site: Rutgers Slinky Site: ---
Alineos Sites: --- Atos/Eviden Sites: ---
Confidential Site: --- Coreweave sites: ---
Cray Sites: --- DS9 clusters: ---
Google sites: --- HPCnow Sites: ---
HPE Sites: --- IBM Sites: ---
NOAA SIte: --- NoveTech Sites: ---
Nvidia HWinf-CS Sites: --- OCF Sites: ---
Recursion Pharma Sites: --- SFW Sites: ---
SNIC sites: --- Tzag Elita Sites: ---
Linux Distro: --- Machine Name:
CLE Version: Version Fixed:
Target Release: --- DevPrio: ---
Emory-Cloud Sites: ---

Description Tom Skipper 2022-11-15 12:13:52 MST 
I see that SlurmctldDebug can be changed dynamically via scontrol; however, doesn't appear the same is true to change SlurmdDebug.  Can that only be modified in the slurm.conf file?

This is slurm 18.0.8
Comment 1 Tom Skipper 2022-11-15 12:15:49 MST 
Correct version is 18.08.8
Comment 2 Ben Roberts 2022-11-15 13:04:14 MST 
Hi Tom,

I'm afraid that's correct.  There isn't a mechanism to update the SlurmdDebug level with scontrol.  To change the log leve you would need to update the slurm.conf and restart slurmd on the node you want to collect debug logs for.  Assuming you have the slurm.conf on a shared location or it's automatically distributed to the nodes, you would want to be sure to change the log level back so you don't accidentally have more nodes running with debug logs.  It's also a good idea to set the log level back to 'info' (or whichever log level you typically use) on the node once you've collected the logging information you're looking for.

Let me know if you have any additional questions related to this.

Thanks,
Ben
Comment 3 Tom Skipper 2022-11-15 14:34:52 MST 
Noted.  Appreciate the confirmation.
Comment 4 Ben Roberts 2022-11-15 14:51:19 MST 
No problem.  I'll go ahead and close this ticket but let us know if there's anything else we can do to help.

Thanks,
Ben