Summary: | sacctmgr: procedure to dump/restore QoS settings. | ||
---|---|---|---|
Product: | Slurm | Reporter: | Brad Viviano <viviano.brad> |
Component: | Configuration | Assignee: | Nate Rini <nate> |
Status: | RESOLVED DUPLICATE | QA Contact: | |
Severity: | 5 - Enhancement | ||
Priority: | --- | CC: | jess |
Version: | 21.08.8 | ||
Hardware: | Linux | ||
OS: | Linux | ||
See Also: |
https://bugs.schedmd.com/show_bug.cgi?id=14441 https://bugs.schedmd.com/show_bug.cgi?id=9111 |
||
Site: | EPA | Alineos Sites: | --- |
Atos/Eviden Sites: | --- | Confidential Site: | --- |
Coreweave sites: | --- | Cray Sites: | --- |
DS9 clusters: | --- | HPCnow Sites: | --- |
HPE Sites: | --- | IBM Sites: | --- |
NOAA SIte: | --- | NoveTech Sites: | --- |
Nvidia HWinf-CS Sites: | --- | OCF Sites: | --- |
Recursion Pharma Sites: | --- | SFW Sites: | --- |
SNIC sites: | --- | Tzag Elita Sites: | --- |
Linux Distro: | --- | Machine Name: | |
CLE Version: | Version Fixed: | ||
Target Release: | --- | DevPrio: | --- |
Emory-Cloud Sites: | --- |
Description
Brad Viviano
2022-06-14 06:30:32 MDT
Brad
Currently, QOS dumping/loading does not exist in sacctmgr load/dump commands. It does however exist in slurmrestd:
> https://slurm.schedmd.com/rest_api.html#slurmdbdGetQos
Please tell me if you want specific examples of how to dump and load the QOS configs.
--Nate
> Please tell me if you want specific examples of how to dump and load the QOS configs.
Yes. I'd like a specific example or a reference to a FAQ of how I would dump the QoS, similar to what I would do with "sacctmgr dump ...." and "sacctmgr load ...."
Thanks.
(In reply to Brad Viviano from comment #3) > > Please tell me if you want specific examples of how to dump and load the QOS configs. > Yes. I'd like a specific example or a reference to a FAQ of how I would > dump the QoS, similar to what I would do with "sacctmgr dump ...." and > "sacctmgr load ...." Does your site have JWT auth setup? We're not running slurmrestd currently. Is there a solution that would work using sacctmgr? (In reply to Brad Viviano from comment #5) > We're not running slurmrestd currently. As long as it is compiled and installed, it's not required to be running as a systemd service. Start the daemon. I'm just having it listen on a UNIX socket so we can use curl: > $ slurmrestd unix:$HOME/.slurmrestd.sock To dump the current config of QOS: > curl --unix-socket $HOME/.slurmrestd.sock "http://localhost/slurmdb/v0.0.38/qos" > qos.json To load: > curl --unix-socket $HOME/.slurmrestd.sock -X POST -H "Content-Type: application/json" "http://localhost/slurmdb/v0.0.38/qos" --data-binary @qos.json Note that if you're attempting to load this, it may reject the request if the "id" field is set but there is already an existing QOS with the same id. > Is there a solution that would work using sacctmgr? We can see about turning this ticket into an RFE to have that functionality added if you prefer. Please note the URLS should be v0.0.37 instead of v0.0.38. > We can see about turning this ticket into an RFE to have that functionality added if you prefer.
It seems to me there should be a standard method to move ALL settings, MINUS job history between slurmdbd instances.
There are a few tickets in bugs.schedmd.com going back to 2017 were others have asked for the capability, to have "sacctmgr dump/load" include QoS. Really not sure why it hasn't been added in the last 5 years.
That said, I don't need the capability built in/automatic. But there should be a clear procedure in the documentation for situations where a user wants to move all settings, EXCEPT job history from Cluster A -> Cluster B.
There certainly are times when I can see this being useful. Setting up a testing environment being the most common. Maybe I want to spin up a handful of VMs and test functionality in 22.05.X before upgrading my live production cluster from 21.08. I need all my database settings (Users, Accounts, Associations, QoS, etc) to be able to test correctly, but I don't want to drag the 5 million+ job entries along with me via a full mysqldump from my production environment.
So, however that gets resolved, RFE as a feature in the code or just better documentation is fine with me :).
Thanks.
(In reply to Brad Viviano from comment #8) > > We can see about turning this ticket into an RFE to have that functionality added if you prefer. > > It seems to me there should be a standard method to move ALL settings, MINUS > job history between slurmdbd instances. Please note that it is possible to dump all of the configurations in slurmdbd (done via sacctmgr or slurmrestd) via: > curl --unix-socket $HOME/.slurmrestd.sock "http://localhost/slurmdb/v0.0.37/config" > config.json Then the config can be applied via > curl --unix-socket $HOME/.slurmrestd.sock -X POST -H "Content-Type: application/json" "http://localhost/slurmdb/v0.0.37/config" --data-binary @config.json > There are a few tickets in bugs.schedmd.com going back to 2017 were others > have asked for the capability, to have "sacctmgr dump/load" include QoS. > Really not sure why it hasn't been added in the last 5 years. We will discuss this internally. Is adding the QOS dumping functionality to `sacctmgr dump` something your site is interested in sponsoring? I'm not sure what "sponsoring" means. When and if your site would like to sponsor this, please do reply, and we can start the normal RFE process again. |