Ticket 17673 - sinfo --yaml dumps core
Summary: sinfo --yaml dumps core
Status: RESOLVED FIXED
Alias: None
Product: Slurm
Classification: Unclassified
Component: User Commands (show other tickets)
Version: 23.02.4
Hardware: Linux Linux
: 4 - Minor Issue
Assignee: Nate Rini
QA Contact:
URL:
Depends on:
Blocks:
 
Reported: 2023-09-12 14:43 MDT by Ole.H.Nielsen@fysik.dtu.dk
Modified: 2023-09-26 12:39 MDT (History)
1 user (show)

See Also:
Site: DTU Physics
Alineos Sites: ---
Atos/Eviden Sites: ---
Confidential Site: ---
Coreweave sites: ---
Cray Sites: ---
DS9 clusters: ---
HPCnow Sites: ---
HPE Sites: ---
IBM Sites: ---
NOAA SIte: ---
OCF Sites: ---
Recursion Pharma Sites: ---
SFW Sites: ---
SNIC sites: ---
Linux Distro: ---
Machine Name:
CLE Version:
Version Fixed: 23.02.6,23.11.0rc1
Target Release: ---
DevPrio: ---
Emory-Cloud Sites: ---


Attachments
The RPM configure log file ~/rpmbuild/BUILD/slurm-23.02.4/config.log (274.51 KB, text/plain)
2023-09-12 21:10 MDT, Ole.H.Nielsen@fysik.dtu.dk
Details

Note You need to log in before you can comment on or make changes to this ticket.
Description Ole.H.Nielsen@fysik.dtu.dk 2023-09-12 14:43:16 MDT
I wanted to test the YAML output option, but the commands dump core:

# sinfo --yaml
sinfo: error: load_plugins: unable to find plugin: serializer/yaml
Segmentation fault (core dumped)

I'm probably missing some required library in my Slurm RPM build.  The OS is CentOS 7.9 containing this libyaml RPM:

# rpm -q libyaml
libyaml-0.1.4-11.el7_0.x86_64

Ole
Comment 1 Ole.H.Nielsen@fysik.dtu.dk 2023-09-12 21:10:28 MDT
Created attachment 32226 [details]
The RPM configure log file ~/rpmbuild/BUILD/slurm-23.02.4/config.log
Comment 2 Ole.H.Nielsen@fysik.dtu.dk 2023-09-19 04:49:26 MDT
In the YAML Parser installation instructions at https://slurm.schedmd.com/download.html#yaml the libyaml version 0.2.5 gets installed.

However, in CentOS 7 and RHEL/Rocky/Alma 8 only some older versions 0.1.4 and 0.1.7 are available for installation.  The EL9 contains 0.2.5.

Can Slurm somehow use the older libyaml versions, or at least detect that an old version is installed?  Otherwise we might have to find a way to build libyaml RPM packages for version 0.2.5.

Ole
Comment 3 Nate Rini 2023-09-19 09:19:33 MDT
(In reply to Ole.H.Nielsen@fysik.dtu.dk from comment #2)
> Can Slurm somehow use the older libyaml versions, or at least detect that an
> old version is installed?  Otherwise we might have to find a way to build
> libyaml RPM packages for version 0.2.5.

It could select any one of them depending on which one is found first during configure.
Comment 4 Ole.H.Nielsen@fysik.dtu.dk 2023-09-19 13:13:27 MDT
(In reply to Nate Rini from comment #3)
> (In reply to Ole.H.Nielsen@fysik.dtu.dk from comment #2)
> > Can Slurm somehow use the older libyaml versions, or at least detect that an
> > old version is installed?  Otherwise we might have to find a way to build
> > libyaml RPM packages for version 0.2.5.
> 
> It could select any one of them depending on which one is found first during
> configure.

The system only has 1 libyaml version, namely the older 0.1.4/0.1.7 from the installed RPM.

I meant to ask whether Slurm's YAML plugin can be made to function correctly with libyaml 0.1.X?

If this is not possible, the minimum working version of libyaml ought to be documented.

Thanks,
Ole
Comment 5 Nate Rini 2023-09-19 13:37:59 MDT
(In reply to Ole.H.Nielsen@fysik.dtu.dk from comment #4)
> I meant to ask whether Slurm's YAML plugin can be made to function correctly
> with libyaml 0.1.X?
> If this is not possible, the minimum working version of libyaml ought to be
> documented.

From the logs, it looks like configure handled everything correctly:
> configure:21130: checking for libyaml installation
> configure:21172: result: 
> configure:21177: WARNING: unable to locate libyaml parser library

I will add a note about the minimum libyaml required.
Comment 6 Ole.H.Nielsen@fysik.dtu.dk 2023-09-19 13:53:43 MDT
(In reply to Nate Rini from comment #5)
> (In reply to Ole.H.Nielsen@fysik.dtu.dk from comment #4)
> > I meant to ask whether Slurm's YAML plugin can be made to function correctly
> > with libyaml 0.1.X?
> > If this is not possible, the minimum working version of libyaml ought to be
> > documented.
> 
> From the logs, it looks like configure handled everything correctly:
> > configure:21130: checking for libyaml installation
> > configure:21172: result: 
> > configure:21177: WARNING: unable to locate libyaml parser library

Yes, I saw this too, despite libyaml version 0.1.4 (CentOS 7 RPM) being installed and available in the expected location:

$ ls -l /usr/lib64/libyaml*
lrwxrwxrwx. 1 root root     18 Dec 15  2016 /usr/lib64/libyaml-0.so.2 -> libyaml-0.so.2.0.4
-rwxr-xr-x. 1 root root 131096 Jan 30  2015 /usr/lib64/libyaml-0.so.2.0.4

> I will add a note about the minimum libyaml required.

Thanks.  Following that documentation we will have to make workarounds to build and install the libyaml RPM packages, since these are not provided by EL7 and EL8 Linux.

Ole
Comment 10 Ole.H.Nielsen@fysik.dtu.dk 2023-09-20 02:24:31 MDT
I looked at the Slurm configure script and found that it determines whether libyaml is installed or not by searching for the yaml.h file, not the library itself.

On CentOS and RHEL (and clones) the yaml.h file is only installed with the libyaml-devel RPM package:

$ rpm -qf /usr/include/yaml.h
libyaml-devel-0.1.4-11.el7_0.x86_64

Likewise, I suppose that the libjwt-devel RPM package must also be installed. 

I've rebuilt Slurm RPM packages with these prerequisites, and "sinfo --yaml" now works correctly.

The core dump reported in comment 0 should nevertheless be fixed.  It occurs when the libyaml-devel RPM package wasn't installed when building Slurm RPMs.

I would like to ask that the download page https://slurm.schedmd.com/download.html#json should be updated with an extra paragraph at the end:

For RPM based installations the development libraries should be installed by:

$ yum install http-parser-devel json-c-devel libjwt-devel libyaml-devel

Thanks,
Ole
Comment 11 Ole.H.Nielsen@fysik.dtu.dk 2023-09-20 02:36:47 MDT
It would also be good to add this documentation to https://slurm.schedmd.com/faq.html#json_serializer

For RPM based installations the development libraries should be installed by:

$ yum install http-parser-devel json-c-devel libjwt-devel libyaml-devel
Comment 12 Nate Rini 2023-09-20 12:51:50 MDT
Ole

I created bug#17731 to handle the RPM specific issues to avoid confusing that and the segfault originally reported in comment#0.
Comment 13 Ole.H.Nielsen@fysik.dtu.dk 2023-09-20 12:58:23 MDT
(In reply to Nate Rini from comment #12)
> I created bug#17731 to handle the RPM specific issues to avoid confusing
> that and the segfault originally reported in comment#0.

Thanks, sounds good!
Ole
Comment 21 Nate Rini 2023-09-26 11:25:10 MDT
Ole

The segfault has now been fixed and minimum versions have been noted for libyaml/libjson-c for the upcoming Slurm-23.02.6 release:
> 42a08d9caf interfaces/data_parser - Avoid passing NULL to printf()
> 3df118258b plugrack - Always allocate functions array
> a0f9e6275d Docs - Note min version for libyaml and libjson-c

Closing out the ticket as rpm-related issues are to be handled in bug#17731.

Please respond if there are any more issues or related questions.

--Nate
Comment 22 Ole.H.Nielsen@fysik.dtu.dk 2023-09-26 12:39:02 MDT
Hi Nate

(In reply to Nate Rini from comment #21)
> The segfault has now been fixed and minimum versions have been noted for
> libyaml/libjson-c for the upcoming Slurm-23.02.6 release:
> > 42a08d9caf interfaces/data_parser - Avoid passing NULL to printf()
> > 3df118258b plugrack - Always allocate functions array
> > a0f9e6275d Docs - Note min version for libyaml and libjson-c

Sounds good! Thanks a lot for the patches!

> Closing out the ticket as rpm-related issues are to be handled in bug#17731.

OK, I look forward to resolution of those issues as well.

Ole