Ticket 17317

Summary:	upgrade site + add accounting
Product:	Slurm	Reporter:	keven.haynes
Component:	Build System and Packaging	Assignee:	Ben Roberts <ben>
Status:	RESOLVED INFOGIVEN	QA Contact:
Severity:	4 - Minor Issue
Priority:	---
Version:	23.11.x
Hardware:	Linux
OS:	Linux
Site:	Rollins School of Public Health	Alineos Sites:	---
Atos/Eviden Sites:	---	Confidential Site:	---
Coreweave sites:	---	Cray Sites:	---
DS9 clusters:	---	HPCnow Sites:	---
HPE Sites:	---	IBM Sites:	---
NOAA SIte:	---	OCF Sites:	---
Recursion Pharma Sites:	---	SFW Sites:	---
SNIC sites:	---	Linux Distro:	---
Machine Name:		CLE Version:
Version Fixed:		Target Release:	---
DevPrio:	---	Emory-Cloud Sites:	---

Description keven.haynes 2023-07-31 09:28:15 MDT

Greetings, 

We need to upgrade our site-wide slurm installation (from 20.11) and add the slurmdb accounting database to our site.  If someone would reach out to us regarding the process we'd be much obliged. 

Sincerely, 
Keven Haynes
HPC Manager, RSPH (not Emory-Cloud)

Comment 1 Ben Roberts 2023-07-31 10:35:13 MDT

Hi Keven,

We support upgrades from up to 2 versions prior to the version you are going to. So in your case, if you were going to go to version 23.02 you would need to take an intermediate step when coming from 20.11. This requirement for upgrades to be within 2 versions is so that Slurm can interpret the information from the previous version correctly. When you are using slurmdbd there is a database conversion that happens, which sounds like it may not be applicable in your case. For slurmctld there are state files that contain information about things like nodes and any queued jobs or reservations that need to be handled.

We do have a document that outlines the steps you need to take to upgrade your system. I would recommend you review this document, if you haven't already, before beginning your upgrade.
https://slurm.schedmd.com/quickstart_admin.html#upgrade

We also have a document that goes over configuring your system to use slurmdbd for accounting that I would recommend you also review.
https://slurm.schedmd.com/accounting.html

The primary thing I would point out is that if you are going to be using AccountingStorageEnforce (which I would recommend) you will need to make sure that you create accounts and user associations for all the users who will be using the cluster.

Since you are going to be upgrading around the same time you are introducing slurmdbd, I would recommend you perform the upgrade first and then add accounting. I would get slurmdbd up and running, use sacctmgr to create the cluster, create the account hierarchy so that it matches what you need for your organization, and then create the users in the appropriate accounts. This process is outlined here:
https://slurm.schedmd.com/accounting.html#database-configuration

Let me know if you have any questions about the information I sent.

Thanks,
Ben

Comment 2 Ben Roberts 2023-08-23 13:47:12 MDT

Hi Keven,

I wanted to follow up and make sure you don't have any follow up questions about this.  Let me know if there is anything else I can do to help or if this ticket is ok to close.

Thanks,
Ben

Comment 3 Ben Roberts 2023-09-26 12:34:47 MDT

Hi Keven,

I believe the information I sent should have helped and I haven't heard any additional questions.  I'll go ahead and close this ticket but let us know if there's anything else we can do to help.

Thanks,
Ben