Ticket 11109

Summary:	Support multiple BasePath with job_container/tmpfs
Product:	Slurm	Reporter:	Trey Dockendorf <tdockendorf>
Component:	Configuration	Assignee:	Nate Rini <nate>
Status:	RESOLVED DUPLICATE	QA Contact:
Severity:	5 - Enhancement
Priority:	---	CC:	bas.vandervlies, marshall, plazonic, tim, troy
Version:	20.11.5
Hardware:	Linux
OS:	Linux
See Also:	https://bugs.schedmd.com/show_bug.cgi?id=11123
Site:	Ohio State OSC	Alineos Sites:	---
Atos/Eviden Sites:	---	Confidential Site:	---
Coreweave sites:	---	Cray Sites:	---
DS9 clusters:	---	HPCnow Sites:	---
HPE Sites:	---	IBM Sites:	---
NOAA SIte:	---	NoveTech Sites:	---
Nvidia HWinf-CS Sites:	---	OCF Sites:	---
Recursion Pharma Sites:	---	SFW Sites:	---
SNIC sites:	---	Linux Distro:	---
Machine Name:		CLE Version:
Version Fixed:		Target Release:	---
DevPrio:	---	Emory-Cloud Sites:	---

Description Trey Dockendorf 2021-03-16 16:50:40 MDT

OSC has been using https://github.com/hpc2n/spank-private-tmp/tree/20.02.2 with SLURM 20.11 to do a private /dev/shm and it has worked nicely.  That SPANK plugin supports multiple private mounts and we are evaluating also doing a private /tmp.  If we moved to job_container/tmpfs it looks like we'd be limited to /dev/shm only and not both /dev/shm and /tmp.  Is it possible with job_container/tmpfs to setup multiple private locations like /dev/shm and /tmp ? My read of the config docs and my initial read of code is that only one BasePath per either config or node group is allowed.  Maybe this needs to be an enhancement if that's not possible.

Comment 1 Nate Rini 2021-03-16 17:05:13 MDT

Trey,

I'm going to put this as a SEV5 (RFE) while I look into it. This plugin is new, so the limitations have not all been tested.

Thanks,
--Nate

Comment 2 Nate Rini 2021-03-17 15:30:01 MDT

Trey,

The paths (/dev/shm and /tmp) are hardcoded in the plugin currently. I will start the normal process for RFE review.

Thanks,
--Nate

Comment 6 Josko Plazonic 2021-03-18 15:39:48 MDT

As far as I can tell /dev/shm is already handled by tmpfs plugin and is not configurable (i.e., plugin always does it):

job_container_tmpfs.c:static int _mount_private_shm(void)
job_container_tmpfs.c:  rc = umount("/dev/shm");
job_container_tmpfs.c:          error("%s: umount /dev/shm failed: %s\n",
job_container_tmpfs.c:  rc = mount("tmpfs", "/dev/shm", "tmpfs", 0, NULL);
job_container_tmpfs.c:          error("%s: mounting private /dev/shm failed: %s\n",
job_container_tmpfs.c:                  rc = _mount_private_shm();
job_container_tmpfs.c:                          error("%s: could not mount private shm",

Our problem is that we want to have additional "private" temp directories. We currently do this with /tmp, /var/tmp and /var/lock. Maybe an option Dirs that can default to /tmp but allows more than that.

We actually had an earlier version of NERSC plugin that we modified to do that but the version that got merged has diverged too much for this to be directly useful.

Comment 7 Trey Dockendorf 2021-03-18 15:53:51 MDT

For OSC, this RFE maybe not useful since this was initially opened not fully understanding how the job_container/tmpfs plugin worked.  I opened RFE #11135 that would actually allow OSC to use this plugin in our current environment.

Comment 8 Nate Rini 2021-03-18 15:58:06 MDT

Trey,

I going to mark this as a duplicate of bug#11135 then.

Thanks,
--Nate

*** This ticket has been marked as a duplicate of ticket 11135 ***