Ticket 6877

Summary: Schedmd/UW/Google Hybrid Cloud Bursting / Federation project using API Keys
Product: Slurm Reporter: Adam Hough <ahough>
Component: CloudAssignee: Ben Roberts <ben>
Status: RESOLVED INFOGIVEN QA Contact:
Severity: 4 - Minor Issue    
Priority: --- CC: kilian, mattjay
Version: 18.08.3   
Hardware: Linux   
OS: Linux   
Site: University of Washington Alineos Sites: ---
Atos/Eviden Sites: --- Confidential Site: ---
Coreweave sites: --- Cray Sites: ---
DS9 clusters: --- HPCnow Sites: ---
HPE Sites: --- IBM Sites: ---
NOAA SIte: --- OCF Sites: ---
Recursion Pharma Sites: --- SFW Sites: ---
SNIC sites: --- Linux Distro: ---
Machine Name: CLE Version:
Version Fixed: Target Release: ---
DevPrio: --- Emory-Cloud Sites: ---
Attachments: Mox Slurm config file.
scontrol show config

Description Adam Hough 2019-04-17 16:14:28 MDT
Created attachment 9936 [details]
Mox Slurm config file.

Per previous discussion this is to track setting up Slurm to burst to Google's cloud compute using API keys to allow users to have their own Google account but run jobs on either Hyak or Google cloud at the user's directive.
Comment 1 Adam Hough 2019-04-17 16:16:21 MDT
Created attachment 9937 [details]
scontrol show config
Comment 4 Ben Roberts 2019-05-06 15:33:42 MDT
Hi Adam,

Brian let me know that you were discussing setting up a cluster in the cloud to complement your on-premise cluster.  He said we would send some documentation on setting this up, which I'm happy to provide.  

You can start with the github page for the slurm-gcp project, which provides some details on setting up a stand alone cluster with this tool.
https://github.com/SchedMD/slurm-gcp

Codelabs also has a good tutorial on setting this up:
https://codelabs.developers.google.com/codelabs/hpc-slurm-on-gcp/#0

We have a documentation page on configuring Slurm for multiple clusters:
https://slurm.schedmd.com/multi_cluster.html

If you're going to have separate clusters the more secure way of handling this is to have multiple munge keys.  The accounting documentation has some details on configuring multiple munge keys in the section about infrastructure:
https://slurm.schedmd.com/accounting.html#infrastructure

With multiple munge keys you would need to specify which munge socket was listening for which key.  You can do this with the AuthInfo parameter in the slurmdbd.conf and slurmctld.conf files.
You can read more about AuthInfo in the slurmdbd.conf file in the documentation here:
https://slurm.schedmd.com/accounting.html#slurmdbd-configuration

...and here:
https://slurm.schedmd.com/slurmdbd.conf.html#OPT_AuthInfo

And this is the section on setting AuthInfo in the slurm.conf:
https://slurm.schedmd.com/slurm.conf.html#OPT_AuthInfo

Let me know if you have questions about this.

Thanks,
Ben
Comment 5 Ben Roberts 2019-05-30 12:43:21 MDT
Hi Adam,

I wanted to make sure the information I sent you about cloud bursting was helpful.  Let me know if you still need help with this ticket.

Thanks,
Ben
Comment 6 Ben Roberts 2019-06-19 13:52:32 MDT
Hi Adam,

I believe the documentation links I sent should provide the information you were looking for.  I'm going to close this ticket, but if you have any follow up questions feel free to let me know and we can reopen the ticket.

Thanks,
Ben