Ticket 3293

Summary:	Upgrade from 15.08.10 to 16.05.6
Product:	Slurm	Reporter:	David Richardson <david.richardson>
Component:	slurmdbd	Assignee:	Tim Wickberg <tim>
Status:	RESOLVED INFOGIVEN	QA Contact:
Severity:	4 - Minor Issue
Priority:	---
Version:	16.05.6
Hardware:	Linux
OS:	Linux
Site:	University of Utah	Alineos Sites:	---
Atos/Eviden Sites:	---	Confidential Site:	---
Coreweave sites:	---	Cray Sites:	---
DS9 clusters:	---	HPCnow Sites:	---
HPE Sites:	---	IBM Sites:	---
NOAA SIte:	---	OCF Sites:	---
Recursion Pharma Sites:	---	SFW Sites:	---
SNIC sites:	---	Linux Distro:	---
Machine Name:		CLE Version:
Version Fixed:		Target Release:	---
DevPrio:	---	Emory-Cloud Sites:	---

Description David Richardson 2016-11-28 17:19:28 MST

Hi,

In my environment, we have four clusters, each with its own schedctld. Those four schedctld's all talk to a single slurmdbd. We are preparing to update our clusters from CentOS 6 to 7.

Is there any concern about updating the slurmdbd to 16, updating one of the schedctlds and one cluster to 16, but leaving the others back at 15 (perhaps for an extended time)?

Are there any significant caveats to the upgrade procedure from 15 to 16 (beyond update the dbd [and wait for the DB schema updates], update the ctld, and update the individual cluster nodes)?

Thanks,
DR
University of Utah CHPC

Comment 1 Tim Wickberg 2016-11-28 18:57:09 MST

(In reply to David Richardson from comment #0)
> Hi,
> 
> In my environment, we have four clusters, each with its own schedctld. Those
> four schedctld's all talk to a single slurmdbd. We are preparing to update
> our clusters from CentOS 6 to 7.
> 
> Is there any concern about updating the slurmdbd to 16, updating one of the
> schedctlds and one cluster to 16, but leaving the others back at 15 (perhaps
> for an extended time)?
> 
> Are there any significant caveats to the upgrade procedure from 15 to 16
> (beyond update the dbd [and wait for the DB schema updates], update the
> ctld, and update the individual cluster nodes)?
> 
> Thanks,
> DR
> University of Utah CHPC

Nope, that's the preferred upgrade path. Keep in mind that you'll need to eventually get things in sync, but have another release or so until that's a problem - the 16.05 slurmdbd will only talk to 16.05/15.08/14.11 slurmctld/slurmd/user commands.

The 15.08 to 16.05 database update should happen pretty quickly; 14.11 to 15.08 took a lot longer. 16.05 to 17.02 should also be pretty quick once its released.

One note - if you're using cgroups, the ReleaseAgent setting is no longer needed on 16.05.5 and up, which solves some complications with systemd. If you're using pam_slurm_adopt you may also need to look into disabling pam_systemd to have it function correctly.

Anything else I can answer?

Comment 2 David Richardson 2016-12-02 13:25:20 MST

Thanks for your help, guys. The upgrade of the slurmdbd went perfectly!

Thanks,
DR