| Summary: | Requirement for user access to a reservation for an account | ||
|---|---|---|---|
| Product: | Slurm | Reporter: | Renata Dart <renata> |
| Component: | reservations | Assignee: | Ben Roberts <ben> |
| Status: | RESOLVED INFOGIVEN | QA Contact: | |
| Severity: | 4 - Minor Issue | ||
| Priority: | --- | ||
| Version: | 22.05.2 | ||
| Hardware: | Linux | ||
| OS: | Linux | ||
| Site: | SLAC | Slinky Site: | --- |
| Alineos Sites: | --- | Atos/Eviden Sites: | --- |
| Confidential Site: | --- | Coreweave sites: | --- |
| Cray Sites: | --- | DS9 clusters: | --- |
| Google sites: | --- | HPCnow Sites: | --- |
| HPE Sites: | --- | IBM Sites: | --- |
| NOAA SIte: | --- | NoveTech Sites: | --- |
| Nvidia HWinf-CS Sites: | --- | OCF Sites: | --- |
| Recursion Pharma Sites: | --- | SFW Sites: | --- |
| SNIC sites: | --- | Tzag Elita Sites: | --- |
| Linux Distro: | --- | Machine Name: | |
| CLE Version: | Version Fixed: | ||
| Target Release: | --- | DevPrio: | --- |
| Emory-Cloud Sites: | --- | ||
|
Description
Renata Dart
2023-02-14 19:26:41 MST
Hi Renata,
It's true that without the AccountingStorageEnforce flag Slurm will allow users to specify any account they want and allow the jobs to run. Reservations are slightly different though since it's reserving nodes for a specific type of job. When you set an account as the access list for a reservation it will enforce that requirement for jobs to be able to run in the reservation.
If you are looking for a way to allow more users to access a reservation you could specify an account that is denied access to the reservation, which would then implicitly allow all other accounts. Here's an example of how that would look.
I have an account called 'sub6' and I say that jobs from that account are denied access to reservation.
$ scontrol create reservation accounts=-sub6 starttime=now duration=1:00:00 nodecnt=1
Reservation created: sub6_5
$ scontrol show reservations sub6_5
ReservationName=sub6_5 StartTime=2023-02-15T11:57:09 EndTime=2023-02-15T12:57:09 Duration=01:00:00
Nodes=node01 NodeCnt=1 CoreCnt=6 Features=(null) PartitionName=debug Flags=
TRES=cpu=12
Users=(null) Groups=(null) Accounts=-sub6 Licenses=(null) State=ACTIVE BurstBuffer=(null) Watts=n/a
MaxStartDelay=(null)
If I submit a job to a reservation that is not defined on my system it will allow the job to run in that reservation.
$ sbatch -n1 -Afake --reservation=sub6_5 --wrap='srun sleep 30'
Submitted batch job 1454
$ squeue
JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)
1454 debug wrap user4 R 0:00 1 node01
You should be aware if a user specifies the account that was excluded from the reservation, their job will be rejected.
$ sbatch -n1 -Asub6 --reservation=sub6_5 --wrap='srun sleep 30'
sbatch: error: Batch job submission failed: Access denied to requested reservation
I remember from the previous ticket that you were working on getting accounts created the way you wanted. Is that still moving forward? Let me know if you have any questions about the reservation example I showed and if that sounds like something that will work.
Thanks,
Ben
Hi Ben, thanks for this clarification, it is what I was expecting based on my tests. The decision still hasn't been made to move forward with AccountingStorageEnforce, so these kinds of questions are cropping up. Renata On Wed, 15 Feb 2023, bugs@schedmd.com wrote: >https://bugs.schedmd.com/show_bug.cgi?id=16032 > >--- Comment #1 from Ben Roberts <ben@schedmd.com> --- >Hi Renata, > >It's true that without the AccountingStorageEnforce flag Slurm will allow users >to specify any account they want and allow the jobs to run. Reservations are >slightly different though since it's reserving nodes for a specific type of >job. When you set an account as the access list for a reservation it will >enforce that requirement for jobs to be able to run in the reservation. > >If you are looking for a way to allow more users to access a reservation you >could specify an account that is denied access to the reservation, which would >then implicitly allow all other accounts. Here's an example of how that would >look. > >I have an account called 'sub6' and I say that jobs from that account are >denied access to reservation. >$ scontrol create reservation accounts=-sub6 starttime=now duration=1:00:00 >nodecnt=1 >Reservation created: sub6_5 > >$ scontrol show reservations sub6_5 >ReservationName=sub6_5 StartTime=2023-02-15T11:57:09 >EndTime=2023-02-15T12:57:09 Duration=01:00:00 > Nodes=node01 NodeCnt=1 CoreCnt=6 Features=(null) PartitionName=debug Flags= > TRES=cpu=12 > Users=(null) Groups=(null) Accounts=-sub6 Licenses=(null) State=ACTIVE >BurstBuffer=(null) Watts=n/a > MaxStartDelay=(null) > > > > >If I submit a job to a reservation that is not defined on my system it will >allow the job to run in that reservation. >$ sbatch -n1 -Afake --reservation=sub6_5 --wrap='srun sleep 30' >Submitted batch job 1454 > >$ squeue > JOBID PARTITION NAME USER ST TIME NODES >NODELIST(REASON) > 1454 debug wrap user4 R 0:00 1 node01 > > > > >You should be aware if a user specifies the account that was excluded from the >reservation, their job will be rejected. >$ sbatch -n1 -Asub6 --reservation=sub6_5 --wrap='srun sleep 30' >sbatch: error: Batch job submission failed: Access denied to requested >reservation > > > > >I remember from the previous ticket that you were working on getting accounts >created the way you wanted. Is that still moving forward? Let me know if you >have any questions about the reservation example I showed and if that sounds >like something that will work. > >Thanks, >Ben > >-- >You are receiving this mail because: >You reported the bug. I understand. Hopefully excluding an account will work better for what you're trying to do. Let me know if you have any other questions about this or if we can close the ticket. Thanks, Ben Hi Ben, thanks for your help. A somewhat related question....is it possible for a slurm operator or admin to submit a job as someone else, without sudo? Renata On Thu, 16 Feb 2023, bugs@schedmd.com wrote: >https://bugs.schedmd.com/show_bug.cgi?id=16032 > >--- Comment #3 from Ben Roberts <ben@schedmd.com> --- >I understand. Hopefully excluding an account will work better for what you're >trying to do. Let me know if you have any other questions about this or if we >can close the ticket. > >Thanks, >Ben > >-- >You are receiving this mail because: >You reported the bug. We do have a flag that is available for the root user to submit jobs as other users, but I'm afraid it's not an option for admins or operators. https://slurm.schedmd.com/sbatch.html#OPT_uid When you attempt to use it as a non-root user it will print out this message: $ sbatch --uid=user2 --wrap='srun sleep 30' sbatch: error: --uid only permitted by root user Thanks, Ben Hi Renata, I believe the information I sent addressed your questions and I haven't heard any follow up questions. I'll go ahead and close this ticket. Let us know if there's anything else we can do to help. Thanks, Ben |