Ticket 17276

Summary:	Question about reservation syntax and GPUs
Product:	Slurm	Reporter:	Brent G <brent.gawryluik>
Component:	Other	Assignee:	Jason Booth <jbooth>
Status:	RESOLVED INFOGIVEN	QA Contact:
Severity:	4 - Minor Issue
Priority:	---
Version:	22.05.8
Hardware:	Linux
OS:	Linux
Site:	Recursion Pharma	Alineos Sites:	---
Atos/Eviden Sites:	---	Confidential Site:	---
Coreweave sites:	---	Cray Sites:	---
DS9 clusters:	---	HPCnow Sites:	---
HPE Sites:	---	IBM Sites:	---
NOAA SIte:	---	OCF Sites:	---
Recursion Pharma Sites:	---	SFW Sites:	---
SNIC sites:	---	Linux Distro:	---
Machine Name:		CLE Version:
Version Fixed:		Target Release:	---
DevPrio:	---	Emory-Cloud Sites:	---

Description Brent G 2023-07-25 10:18:28 MDT

I created the following reservation for two nodes:

╭─root@bright1 ~
╰─# scontrol create reservation reservationname=neuroatlas_1 starttime=2023-08-01T07:00:00 endtime=2023-08-05T07:00:00 Nodes=vlt1,vlt2 Users=[redacted], flags=ignore_jobs
Reservation created: neuroatlas_1

This is my verification:

╭─root@bright1 ~
╰─# scontrol show reservation
ReservationName=neuroatlas_1 StartTime=2023-08-01T07:00:00 EndTime=2023-08-05T07:00:00 Duration=4-00:00:00
   Nodes=vlt[1-2] NodeCnt=2 CoreCnt=64 Features=(null) PartitionName=(null) Flags=IGNORE_JOBS,SPEC_NODES
   TRES=cpu=64
   Users=[redacted], Groups=(null) Accounts=(null) Licenses=(null) State=INACTIVE BurstBuffer=(null) Watts=n/a
   MaxStartDelay=(null)


The question I have is: do I need to specify anything else to ensure that the GPUs associated with these nodes are available to the specified users? Or do these users get access to all resources on these nodes for this duration?

The other question is that I have not specified a partition. Does this mean that they can use any partition when submitting jobs during the reservation period?

Comment 1 Jason Booth 2023-07-25 11:20:31 MDT

> The question I have is: do I need to specify anything else to ensure that the GPUs 
> associated with these nodes are available to the specified users? Or do these users 
> get access to all resources on these nodes for this duration?

The user will have access to the node and GRES/GPUs of that node during the duration. 
Right now reservations are not tightly coupled to GPU's though we are working on a 23.11 
feature to account for these and reservations.

> The other question is that I have not specified a partition. Does this mean that they 
> can use any partition when submitting jobs during the reservation period?

That is correct. Any partition that the user submits too that also has these nodes will 
give exclusive access.

Comment 2 Brent G 2023-07-25 12:27:13 MDT

Thank you for the clarification. This was extremely helpful.



On Tue, Jul 25, 2023 at 10:20 AM <bugs@schedmd.com> wrote:

> Comment # 1 on bug 17276 from Jason Booth > The question I have is: do I
> need to specify anything else to ensure that the GPUs > associated with
> these nodes are available to the specified users? Or do these users > get
> access to all
> 
>
> *Comment # 1
> <https://urldefense.com/v3/__https://bugs.schedmd.com/show_bug.cgi?id=17276*c1__;Iw!!Bpf-Q4l8AQ!BghpyBp6JzZEv6hFUtdEZdVoiRNJCsIwp4tUKPcahjyQ8NZ_fcD_5A1qvpfSU7JqgxE4UcIKsO5p0CeEuZ60Z4Ur3w$>
> on bug 17276
> <https://urldefense.com/v3/__https://bugs.schedmd.com/show_bug.cgi?id=17276__;!!Bpf-Q4l8AQ!BghpyBp6JzZEv6hFUtdEZdVoiRNJCsIwp4tUKPcahjyQ8NZ_fcD_5A1qvpfSU7JqgxE4UcIKsO5p0CeEuZ7diuIn6A$>
> from Jason Booth <jbooth@schedmd.com> *
>
> > The question I have is: do I need to specify anything else to ensure that the GPUs
> > associated with these nodes are available to the specified users? Or do these users
> > get access to all resources on these nodes for this duration?
>
> The user will have access to the node and GRES/GPUs of that node during the
> duration.
> Right now reservations are not tightly coupled to GPU's though we are working
> on a 23.11
> feature to account for these and reservations.
> > The other question is that I have not specified a partition. Does this mean that they
> > can use any partition when submitting jobs during the reservation period?
>
> That is correct. Any partition that the user submits too that also has these
> nodes will
> give exclusive access.
>
> ------------------------------
> You are receiving this mail because:
>
>    - You reported the bug.
>
>

Comment 3 Jason Booth 2023-07-26 11:58:58 MDT

Resolving.