Ticket 5927

Summary:	Questions regarding Slurm upgrade
Product:	Slurm	Reporter:	NYU HPC Team <hpc-staff>
Component:	Other	Assignee:	Nate Rini <nate>
Status:	RESOLVED INFOGIVEN	QA Contact:
Severity:	3 - Medium Impact
Priority:	---
Version:	- Unsupported Older Versions
Hardware:	Linux
OS:	Linux
See Also:	https://bugs.schedmd.com/show_bug.cgi?id=4918
Site:	NYU	Alineos Sites:	---
Atos/Eviden Sites:	---	Confidential Site:	---
Coreweave sites:	---	Cray Sites:	---
DS9 clusters:	---	HPCnow Sites:	---
HPE Sites:	---	IBM Sites:	---
NOAA SIte:	---	OCF Sites:	---
Recursion Pharma Sites:	---	SFW Sites:	---
SNIC sites:	---	Linux Distro:	---
Machine Name:		CLE Version:
Version Fixed:		Target Release:	---
DevPrio:	---	Emory-Cloud Sites:	---

Description NYU HPC Team 2018-10-25 13:56:08 MDT

Hi Experts:

We are running on the SLURM version 17.02.11, with slurmdbd/mysql in the backend. It has been operating smoothly. (Thanks a lot!). We are thinking of upgrading, and seeking your advice. We are aware of the announcement yesterday on the availability of Slurm versions 18.08.3 and 17.11.12.

1. Is it okay to upgrade from version 17.02.11 to 18.08.3, without going through 17.11.12? Will there be any issue on migrating the DB records?

2. Anything we should pay attention to on MPI (PMI2, PMIX etc)?

3. Any other points you may want to remind us.


Thanks very much,
Wensheng

Comment 2 Nate Rini 2018-10-25 15:55:59 MDT

Wensheng

> 1. Is it okay to upgrade from version 17.02.11 to 18.08.3, without going
> through 17.11.12? Will there be any issue on migrating the DB records?

The upgrade will be jumping from 17.11 -> 18.08 which is within the two version limit and is supported.

https://slurm.schedmd.com/quickstart_admin.html#upgrade
> Slurm permits upgrades between any two versions whose major release numbers differ by two or less

> 2. Anything we should pay attention to on MPI (PMI2, PMIX etc)?

Slurm 18.08+ supports PMIx v1.2+, v2.x and v3.x per https://slurm.schedmd.com/mpi_guide.html#pmix

Verify that OpenMPI (and your other MPI stacks) are correctly linking against the upgraded Slurm's libslurm.so.XX shared library.

> 3. Any other points you may want to remind us.

Please take care to create full backups of your Slurm install including your database to allow for quick reversal in the event of unexpected issues.

--Nate

Comment 3 Nate Rini 2018-11-01 10:49:55 MDT

Closing ticket. Please respond if you have any more questions.