Ticket 7127

Summary:	Tons of "Unable to read requested pattern in /proc/148502/status" in logs
Product:	Slurm	Reporter:	Anthony DelSorbo <anthony.delsorbo>
Component:	slurmd	Assignee:	Director of Support <support>
Status:	RESOLVED DUPLICATE	QA Contact:
Severity:	3 - Medium Impact
Priority:	---
Version:	18.08.7
Hardware:	Linux
OS:	Linux
Site:	NOAA	Slinky Site:	---
Alineos Sites:	---	Atos/Eviden Sites:	---
Confidential Site:	---	Coreweave sites:	---
Cray Sites:	---	DS9 clusters:	---
Google sites:	---	HPCnow Sites:	---
HPE Sites:	---	IBM Sites:	---
NOAA SIte:	NESCC	NoveTech Sites:	---
Nvidia HWinf-CS Sites:	---	OCF Sites:	---
Recursion Pharma Sites:	---	SFW Sites:	---
SNIC sites:	---	Tzag Elita Sites:	---
Linux Distro:	---	Machine Name:	Any
CLE Version:		Version Fixed:
Target Release:	---	DevPrio:	---
Emory-Cloud Sites:	---

Description Anthony DelSorbo 2019-05-28 09:29:14 MDT

This is the first of three tickets concerning slurm's inability to "read" various file systems.  Here, I'm seeing:

slurmstepd[148497]: debug3: jobacct_gather_linux: unable to read requested pattern in /proc/148502/status

Here's a sample count of hits for just part of today 

grep -c "unable to read requested pattern" /log/syslogs/2019.05.28/t05*
/log/syslogs/2019.05.28/t0500:23061
/log/syslogs/2019.05.28/t0501:22165
/log/syslogs/2019.05.28/t0502:26200
/log/syslogs/2019.05.28/t0503:20169
/log/syslogs/2019.05.28/t0504:21024
/log/syslogs/2019.05.28/t0505:41840
/log/syslogs/2019.05.28/t0506:29901
/log/syslogs/2019.05.28/t0507:29695

Would you please guide me in understanding why the slurmd daemons are spewing out so many of these and how to correct the issue?  Short of turning off debug3, there must be some setting that will permit me to reduce or eliminate these messages from the logs.

Thanks,

Tony.

Comment 2 Jason Booth 2019-05-28 10:19:49 MDT

Hi Tony,

This is resolved in the following commit which is part of the 19.05 release.

https://github.com/SchedMD/slurm/commit/585b81d0424526ce7906da8e0818476f1aa3ba4a

For now, I would suggest you lower your log level since these messages will be dumped at debug2 or higher and a production system should not go beyond "debug" unless absolutely needed.

*** This ticket has been marked as a duplicate of ticket 6501 ***