Ticket 2134

Summary:	installation improvements related mostly to systemd and suse
Product:	Slurm	Reporter:	Dimitar Pashov <d.pashov>
Component:	Other	Assignee:	Jacob Jenson <jacob>
Status:	RESOLVED TIMEDOUT	QA Contact:
Severity:	6 - No support contract
Priority:	---	CC:	adam.huffman
Version:	15.08.3
Hardware:	Linux
OS:	Linux
Site:	-Other-	Alineos Sites:	---
Atos/Eviden Sites:	---	Confidential Site:	---
Coreweave sites:	---	Cray Sites:	---
DS9 clusters:	---	HPCnow Sites:	---
HPE Sites:	---	IBM Sites:	---
NOAA SIte:	---	OCF Sites:	---
Recursion Pharma Sites:	---	SFW Sites:	---
SNIC sites:	---	Linux Distro:	---
Machine Name:		CLE Version:
Version Fixed:		Target Release:	---
DevPrio:	---	Emory-Cloud Sites:	---
Attachments:	combined spec file patch for diff commits see the github compare page service service slurm.spec.patch slurmctld.service.in.patch slurmd.service.in.patch slurmdbd.service.in.patch slurm.conf.example.patch slurm-d.conf.patch slurm-ctld.conf.patch qrerun.pl.patch qalter.pl.patch

Description Dimitar Pashov 2015-11-11 09:23:38 MST

Hi, these changes are mostly things I carried around for a while (some of them more than a year) which make slurm rpm buildable and usable on suse.

1. Add dependency on munge in the service files.
2. Add tmpfiles.d to ensure state directories are there on service startup
3. Move the slurm.conf.example state dirs away from /tmp to /var/spool since it is useful to preserve things after reboot in my installation (and possibly others')
4. Add a bunch of dependencies for various SUSE versions to the spec file while making sure things work in RHEL 6 and 7.
5. Add all the systemd tweaks to the spec file in a sort of general way with minimal disruption.
6. Heed advice from SUSE's fairly comprehensive rpmlint reports.
7. Escape commented out macros since they are still expanded and may cause trouble.
8. Change line endings in a couple of q scripts from windows to unix style.
9. Probably fixed a typo here or there in the spec file.

I'm hosting the changes in a forked repo on https://github.com/dmt4/slurm.git branch systemd-tweaks. I'm attaching patches against the current (as of now) official branch slurm-15.08. They shall apply cleanly as shown in:

https://github.com/SchedMD/slurm/compare/slurm-15.08...dmt4:systemd-tweaks

Ready made rpms using these changes, and their respective build logs are available from:
https://build.opensuse.org/package/show/home:pashov_d/slurm

Click on a distribution name for the rpms and on the 'succeeded' links for the respective build log.

I cannot really test on RHEL/CentOS/etc... here and can only go as far compiling (don't have the time to setup virtual machines etc and I stay away from the rhel machines around). I do however run slurm on a handful of suse machines and it seems to be fine. I doubt there is major messup on rhel builds but everything is possible.

Cheers!

Comment 1 Dimitar Pashov 2015-11-11 09:25:16 MST

Created attachment 2407 [details]
combined spec file patch for diff commits see the github compare page

Comment 2 Dimitar Pashov 2015-11-11 09:26:38 MST

Created attachment 2408 [details]
service

Comment 3 Dimitar Pashov 2015-11-11 09:27:07 MST

Created attachment 2409 [details]
service

Comment 4 Dimitar Pashov 2015-11-11 09:28:18 MST

Created attachment 2410 [details]
slurm.spec.patch

Comment 5 Dimitar Pashov 2015-11-11 09:28:52 MST

Created attachment 2411 [details]
slurmctld.service.in.patch

Comment 6 Dimitar Pashov 2015-11-11 09:29:18 MST

Created attachment 2412 [details]
slurmd.service.in.patch

Comment 7 Dimitar Pashov 2015-11-11 09:29:53 MST

Created attachment 2413 [details]
slurmdbd.service.in.patch

Comment 8 Dimitar Pashov 2015-11-11 09:30:27 MST

Created attachment 2414 [details]
slurm.conf.example.patch

Comment 9 Dimitar Pashov 2015-11-11 09:30:55 MST

Created attachment 2415 [details]
slurm-d.conf.patch

Comment 10 Dimitar Pashov 2015-11-11 09:31:20 MST

Created attachment 2416 [details]
slurm-ctld.conf.patch

Comment 11 Dimitar Pashov 2015-11-11 09:31:48 MST

Created attachment 2417 [details]
qrerun.pl.patch

Comment 12 Dimitar Pashov 2015-11-11 09:32:11 MST

Created attachment 2418 [details]
qalter.pl.patch

Comment 13 Dimitar Pashov 2015-11-11 09:33:15 MST

Th way to add files to bugzilla is bonkers.

Comment 15 Moe Jette 2016-10-24 15:20:33 MDT

You will not need to keep the patches to the *service.in* files in Slurm version 17.02.

I don't know when we'll have time to review the other changes, which are fairly large.


$ grep munge *service*
slurmctld.service.in:After=network.target munge.service
slurmdbd.service.in:After=network.target munge.service
slurmd.service.in:After=network.target munge.service

Comment 16 Dimitar Pashov 2016-10-25 07:10:11 MDT

That's a good development!

As for the other changes, most of them are actually quite small. You will convince yourself if you read the points. For example the q scripts look like massive patches but are nothing more than the result dos2unix being run on them. Escaping of the .spec file comment variables is trivial stuff and so is the tmpfiles.d addition too. 

When I submitted all this a year ago I did it with the intent to help you get a more polished product, which I have used for free. Carrying out the changes for my local setup wasn't such a big deal.

To be honest, after the "Importance: --- 6 - No support contract" change I got the impression you could not be bothered to read a report from someone not paying for support even if it is more of a contribution than a complaint, and almost entirely in your benefit. 

Thanks for taking the time to respond anyway. Maybe my impression was incorrect.

Comment 17 Moe Jette 2016-10-25 08:49:35 MDT

(In reply to Dimitar Pashov from comment #16)
> That's a good development!
> 
> As for the other changes, most of them are actually quite small. You will
> convince yourself if you read the points. For example the q scripts look
> like massive patches but are nothing more than the result dos2unix being run
> on them. Escaping of the .spec file comment variables is trivial stuff and
> so is the tmpfiles.d addition too. 

Thanks for that information

> When I submitted all this a year ago I did it with the intent to help you
> get a more polished product, which I have used for free. Carrying out the
> changes for my local setup wasn't such a big deal.
> 
> To be honest, after the "Importance: --- 6 - No support contract" change I
> got the impression you could not be bothered to read a report from someone
> not paying for support even if it is more of a contribution than a
> complaint, and almost entirely in your benefit. 
> 
> Thanks for taking the time to respond anyway. Maybe my impression was
> incorrect.

Thank you for your contribution. It's just a matter of doing what we can with the staffing available. The backlog of bugs has tripled over the past year. I'm just making a quick pass over the severity 6 bugs looking for contributions like yours.

Comment 18 Moe Jette 2016-10-25 09:48:13 MDT

dos2imox was already run on qrerun and qalter.

I committed your slurm.conf.example patch here:
https://github.com/SchedMD/slurm/commit/21276086caa435ce794e6e331d711adcfdc15944

The other changes are configuration or build dependent and we probably don't want to make them.

In any case, your local changes will be much smaller going forward.

Comment 19 Dimitar Pashov 2016-10-25 10:34:06 MDT

On Tuesday, 25 October 2016 15:48:13 BST you wrote:
> https://bugs.schedmd.com/show_bug.cgi?id=2134
> 
> --- Comment #18 from Moe Jette <jette@schedmd.com> ---
> dos2imox was already run on qrerun and qalter.
> 
> I committed your slurm.conf.example patch here:
> https://github.com/SchedMD/slurm/commit/21276086caa435ce794e6e331d711adcfdc1
> 5944
> 
> The other changes are configuration or build dependent and we probably don't
> want to make them.
> 
> In any case, your local changes will be much smaller going forward.

Fair enough. I've passed maintenance responsibility for the small cluster I've 
setup to someone else here and haven't checked the state of affairs this year. 
Glad to hear that small issues like the dos2unix are getting ironed out. I had 
a fix for a crash in smap back then which I thought was pointless reporting on 
but I see now that this too has been fixed about 6 months ago. It might make 
sense to close this report for being too old to be of any further use.

I'll have to setup a queuing system on a small Knight's Landing based cluster  
soon. It will be an opportunity to test a 17 pre-release and look again at 
that spec file.

Comment 20 Moe Jette 2016-10-25 10:42:06 MDT

(In reply to Dimitar Pashov from comment #19)
> It might make 
> sense to close this report for being too old to be of any further use.

Done


> I'll have to setup a queuing system on a small Knight's Landing based
> cluster  
> soon. It will be an opportunity to test a 17 pre-release and look again at 
> that spec file.

See:
http://slurm.schedmd.com/intel_knl.html

I'd hold off on version 17.02 pre-releases for now.
There's new code in version 16.05.6 that can reboot the KNL nodes in different NUMA and MCDDRAM modes
(Similar functionality to what we've been doing on Cray's for several months, but different plugin and mechanism used to reconfigure and reboot nodes).