Ticket 2134 - installation improvements related mostly to systemd and suse
Summary: installation improvements related mostly to systemd and suse
Status: RESOLVED TIMEDOUT
Alias: None
Product: Slurm
Classification: Unclassified
Component: Other (show other tickets)
Version: 15.08.3
Hardware: Linux Linux
: 6 - No support contract
Assignee: Jacob Jenson
QA Contact:
URL:
Depends on:
Blocks:
 
Reported: 2015-11-11 09:23 MST by Dimitar Pashov
Modified: 2016-10-25 10:42 MDT (History)
1 user (show)

See Also:
Site: -Other-
Slinky Site: ---
Alineos Sites: ---
Atos/Eviden Sites: ---
Confidential Site: ---
Coreweave sites: ---
Cray Sites: ---
DS9 clusters: ---
Google sites: ---
HPCnow Sites: ---
HPE Sites: ---
IBM Sites: ---
NOAA SIte: ---
NoveTech Sites: ---
Nvidia HWinf-CS Sites: ---
OCF Sites: ---
Recursion Pharma Sites: ---
SFW Sites: ---
SNIC sites: ---
Tzag Elita Sites: ---
Linux Distro: ---
Machine Name:
CLE Version:
Version Fixed:
Target Release: ---
DevPrio: ---
Emory-Cloud Sites: ---

Attachments
combined spec file patch for diff commits see the github compare page (28.66 KB, patch)
2015-11-11 09:25 MST, Dimitar Pashov 		Details | Diff
service (319 bytes, patch)
2015-11-11 09:26 MST, Dimitar Pashov 		Details | Diff
service (301 bytes, patch)
2015-11-11 09:27 MST, Dimitar Pashov 		Details | Diff
slurm.spec.patch (28.66 KB, patch)
2015-11-11 09:28 MST, Dimitar Pashov 		Details | Diff
slurmctld.service.in.patch (319 bytes, patch)
2015-11-11 09:28 MST, Dimitar Pashov 		Details | Diff
slurmd.service.in.patch (301 bytes, patch)
2015-11-11 09:29 MST, Dimitar Pashov 		Details | Diff
slurmdbd.service.in.patch (322 bytes, patch)
2015-11-11 09:29 MST, Dimitar Pashov 		Details | Diff
slurm.conf.example.patch (711 bytes, patch)
2015-11-11 09:30 MST, Dimitar Pashov 		Details | Diff
slurm-d.conf.patch (251 bytes, patch)
2015-11-11 09:30 MST, Dimitar Pashov 		Details | Diff
slurm-ctld.conf.patch (266 bytes, patch)
2015-11-11 09:31 MST, Dimitar Pashov 		Details | Diff
qrerun.pl.patch (9.02 KB, patch)
2015-11-11 09:31 MST, Dimitar Pashov 		Details | Diff
qalter.pl.patch (11.99 KB, patch)
2015-11-11 09:32 MST, Dimitar Pashov 		Details | Diff
Show Obsolete (3) Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this ticket.
Description Dimitar Pashov 2015-11-11 09:23:38 MST 
Hi, these changes are mostly things I carried around for a while (some of them more than a year) which make slurm rpm buildable and usable on suse. 

1. Add dependency on munge in the service files.
2. Add tmpfiles.d to ensure state directories are there on service startup
3. Move the slurm.conf.example state dirs away from /tmp to /var/spool since it is useful to preserve things after reboot in my installation (and possibly others')
4. Add a bunch of dependencies for various SUSE versions to the spec file while making sure things work in RHEL 6 and 7.
5. Add all the systemd tweaks to the spec file in a sort of general way with minimal disruption.
6. Heed advice from SUSE's fairly comprehensive rpmlint reports.
7. Escape commented out macros since they are still expanded and may cause trouble.
8. Change line endings in a couple of q scripts from windows to unix style.
9. Probably fixed a typo here or there in the spec file.

I'm hosting the changes in a forked repo on https://github.com/dmt4/slurm.git branch systemd-tweaks. I'm attaching patches against the current (as of now) official branch slurm-15.08. They shall apply cleanly as shown in:

https://github.com/SchedMD/slurm/compare/slurm-15.08...dmt4:systemd-tweaks

Ready made rpms using these changes, and their respective build logs are available from:
https://build.opensuse.org/package/show/home:pashov_d/slurm

Click on a distribution name for the rpms and on the 'succeeded' links for the respective build log.

I cannot really test on RHEL/CentOS/etc... here and can only go as far compiling (don't have the time to setup virtual machines etc and I stay away from the rhel machines around). I do however run slurm on a handful of suse machines and it seems to be fine. I doubt there is major messup on rhel builds but everything is possible.

Cheers!
Comment 1 Dimitar Pashov 2015-11-11 09:25:16 MST 
Created attachment 2407 [details]
combined spec file patch for diff commits see the github compare page
Comment 2 Dimitar Pashov 2015-11-11 09:26:38 MST 
Created attachment 2408 [details]
service
Comment 3 Dimitar Pashov 2015-11-11 09:27:07 MST 
Created attachment 2409 [details]
service
Comment 4 Dimitar Pashov 2015-11-11 09:28:18 MST 
Created attachment 2410 [details]
slurm.spec.patch
Comment 5 Dimitar Pashov 2015-11-11 09:28:52 MST 
Created attachment 2411 [details]
slurmctld.service.in.patch
Comment 6 Dimitar Pashov 2015-11-11 09:29:18 MST 
Created attachment 2412 [details]
slurmd.service.in.patch
Comment 7 Dimitar Pashov 2015-11-11 09:29:53 MST 
Created attachment 2413 [details]
slurmdbd.service.in.patch
Comment 8 Dimitar Pashov 2015-11-11 09:30:27 MST 
Created attachment 2414 [details]
slurm.conf.example.patch
Comment 9 Dimitar Pashov 2015-11-11 09:30:55 MST 
Created attachment 2415 [details]
slurm-d.conf.patch
Comment 10 Dimitar Pashov 2015-11-11 09:31:20 MST 
Created attachment 2416 [details]
slurm-ctld.conf.patch
Comment 11 Dimitar Pashov 2015-11-11 09:31:48 MST 
Created attachment 2417 [details]
qrerun.pl.patch
Comment 12 Dimitar Pashov 2015-11-11 09:32:11 MST 
Created attachment 2418 [details]
qalter.pl.patch
Comment 13 Dimitar Pashov 2015-11-11 09:33:15 MST 
Th way to add files to bugzilla is bonkers.
Comment 15 Moe Jette 2016-10-24 15:20:33 MDT 
You will not need to keep the patches to the *service.in* files in Slurm version 17.02.

I don't know when we'll have time to review the other changes, which are fairly large.


$ grep munge *service*
slurmctld.service.in:After=network.target munge.service
slurmdbd.service.in:After=network.target munge.service
slurmd.service.in:After=network.target munge.service
Comment 16 Dimitar Pashov 2016-10-25 07:10:11 MDT 
That's a good development!

As for the other changes, most of them are actually quite small. You will convince yourself if you read the points. For example the q scripts look like massive patches but are nothing more than the result dos2unix being run on them. Escaping of the .spec file comment variables is trivial stuff and so is the tmpfiles.d addition too. 

When I submitted all this a year ago I did it with the intent to help you get a more polished product, which I have used for free. Carrying out the changes for my local setup wasn't such a big deal.

To be honest, after the "Importance: --- 6 - No support contract" change I got the impression you could not be bothered to read a report from someone not paying for support even if it is more of a contribution than a complaint, and almost entirely in your benefit. 

Thanks for taking the time to respond anyway. Maybe my impression was incorrect.
Comment 17 Moe Jette 2016-10-25 08:49:35 MDT 
(In reply to Dimitar Pashov from comment #16)
> That's a good development!
> 
> As for the other changes, most of them are actually quite small. You will
> convince yourself if you read the points. For example the q scripts look
> like massive patches but are nothing more than the result dos2unix being run
> on them. Escaping of the .spec file comment variables is trivial stuff and
> so is the tmpfiles.d addition too. 

Thanks for that information

> When I submitted all this a year ago I did it with the intent to help you
> get a more polished product, which I have used for free. Carrying out the
> changes for my local setup wasn't such a big deal.
> 
> To be honest, after the "Importance: --- 6 - No support contract" change I
> got the impression you could not be bothered to read a report from someone
> not paying for support even if it is more of a contribution than a
> complaint, and almost entirely in your benefit. 
> 
> Thanks for taking the time to respond anyway. Maybe my impression was
> incorrect.

Thank you for your contribution. It's just a matter of doing what we can with the staffing available. The backlog of bugs has tripled over the past year. I'm just making a quick pass over the severity 6 bugs looking for contributions like yours.
Comment 18 Moe Jette 2016-10-25 09:48:13 MDT 
dos2imox was already run on qrerun and qalter.

I committed your slurm.conf.example patch here:
https://github.com/SchedMD/slurm/commit/21276086caa435ce794e6e331d711adcfdc15944

The other changes are configuration or build dependent and we probably don't want to make them.

In any case, your local changes will be much smaller going forward.
Comment 19 Dimitar Pashov 2016-10-25 10:34:06 MDT 
On Tuesday, 25 October 2016 15:48:13 BST you wrote:
> https://bugs.schedmd.com/show_bug.cgi?id=2134
> 
> --- Comment #18 from Moe Jette <jette@schedmd.com> ---
> dos2imox was already run on qrerun and qalter.
> 
> I committed your slurm.conf.example patch here:
> https://github.com/SchedMD/slurm/commit/21276086caa435ce794e6e331d711adcfdc1
> 5944
> 
> The other changes are configuration or build dependent and we probably don't
> want to make them.
> 
> In any case, your local changes will be much smaller going forward.

Fair enough. I've passed maintenance responsibility for the small cluster I've 
setup to someone else here and haven't checked the state of affairs this year. 
Glad to hear that small issues like the dos2unix are getting ironed out. I had 
a fix for a crash in smap back then which I thought was pointless reporting on 
but I see now that this too has been fixed about 6 months ago. It might make 
sense to close this report for being too old to be of any further use.

I'll have to setup a queuing system on a small Knight's Landing based cluster  
soon. It will be an opportunity to test a 17 pre-release and look again at 
that spec file.
Comment 20 Moe Jette 2016-10-25 10:42:06 MDT 
(In reply to Dimitar Pashov from comment #19)
> It might make 
> sense to close this report for being too old to be of any further use.

Done


> I'll have to setup a queuing system on a small Knight's Landing based
> cluster  
> soon. It will be an opportunity to test a 17 pre-release and look again at 
> that spec file.

See:
http://slurm.schedmd.com/intel_knl.html

I'd hold off on version 17.02 pre-releases for now.
There's new code in version 16.05.6 that can reboot the KNL nodes in different NUMA and MCDDRAM modes
(Similar functionality to what we've been doing on Cray's for several months, but different plugin and mechanism used to reconfigure and reboot nodes).