Ticket 9111

Summary:	Issues with sacctmgr dump/load; get rid of it, expand it, or add a new option?
Product:	Slurm	Reporter:	Marshall Garey <marshall>
Component:	Database	Assignee:	Unassigned Developer <dev-unassigned>
Status:	OPEN ---	QA Contact:
Severity:	5 - Enhancement
Priority:	---	CC:	bas.vandervlies, cinek, martijn.kruiten, nate, sts, tim, tmerritt
Version:	20.11.x
Hardware:	Linux
OS:	Linux
See Also:	https://bugs.schedmd.com/show_bug.cgi?id=14311 https://bugs.schedmd.com/show_bug.cgi?id=7450 https://bugs.schedmd.com/show_bug.cgi?id=4325
Site:	SchedMD	Alineos Sites:	---
Atos/Eviden Sites:	---	Confidential Site:	---
Coreweave sites:	---	Cray Sites:	---
DS9 clusters:	---	HPCnow Sites:	---
HPE Sites:	---	IBM Sites:	---
NOAA SIte:	---	NoveTech Sites:	---
Nvidia HWinf-CS Sites:	---	OCF Sites:	---
Recursion Pharma Sites:	---	SFW Sites:	---
SNIC sites:	---	Tzag Elita Sites:	---
Linux Distro:	---	Machine Name:
CLE Version:		Version Fixed:
Target Release:	---	DevPrio:	---
Emory-Cloud Sites:	---

Description Marshall Garey 2020-05-26 16:47:39 MDT

Tim, I've added you to CC since this bug has potential enhancements or deprecations. I set this as a sev-4 but it might be a sev-5 instead.

Coming from bug 8910 comment 8:

> * sacctmgr dump doesn't dump QOS, so sacctmgr load will fail
> unless the needed QOS are in the database.

I think that this makes using sacctmgr dump/load pretty useless, since the dump files won't load at all without all the QOS that were attached to associations in the database previously. This requires you to manully dump/restore the qos_table (using mysql commands) to get sacctmgr load to succeed.


> * sacctmgr dump doesn't dump TRES, so you need to manually dump
> and restore the tres_table to get that.

If trying to restore a database from using sacctmgr dump/load and sacctmgr archive load, you'll be missing all the added TRES, which will be an issue for rollups and maybe a problem beyond that. (What about the TRES fields for all the jobs requesting a TRES/GRES that's not one of the default ones?)


As I explained to the customer in bug 8910, simply dumping the entire production database and then doing sacctmgr archive load is a much better method than trying to restore from scratch. However, I think we need to address these problems.

I can think of a few options:
* Deprecate sacctmgr dump/load because it's not practically useful without manually (using mysql) dumping/restoring the qos_table in the database.
* Enhance sacctmgr dump/load to include QOS (and TRES).
* Add new commands to dump/load QOS and TRES.
* Document these limitations. I personally don't like this option, because we'd basically be documenting that a command is broken without fixing it.

I'm sure there are other options I haven't considered.