Ticket 11109 - Support multiple BasePath with job_container/tmpfs
Summary: Support multiple BasePath with job_container/tmpfs
Status: RESOLVED DUPLICATE of ticket 11135
Alias: None
Product: Slurm
Classification: Unclassified
Component: Configuration (show other tickets)
Version: 20.11.5
Hardware: Linux Linux
: 5 - Enhancement
Assignee: Nate Rini
QA Contact:
URL:
Depends on:
Blocks:
 
Reported: 2021-03-16 16:50 MDT by Trey Dockendorf
Modified: 2021-03-18 15:58 MDT (History)
5 users (show)

See Also:
Site: Ohio State OSC
Alineos Sites: ---
Atos/Eviden Sites: ---
Confidential Site: ---
Coreweave sites: ---
Cray Sites: ---
DS9 clusters: ---
HPCnow Sites: ---
HPE Sites: ---
IBM Sites: ---
NOAA SIte: ---
NoveTech Sites: ---
Nvidia HWinf-CS Sites: ---
OCF Sites: ---
Recursion Pharma Sites: ---
SFW Sites: ---
SNIC sites: ---
Linux Distro: ---
Machine Name:
CLE Version:
Version Fixed:
Target Release: ---
DevPrio: ---
Emory-Cloud Sites: ---

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this ticket.
Description Trey Dockendorf 2021-03-16 16:50:40 MDT 
OSC has been using https://github.com/hpc2n/spank-private-tmp/tree/20.02.2 with SLURM 20.11 to do a private /dev/shm and it has worked nicely.  That SPANK plugin supports multiple private mounts and we are evaluating also doing a private /tmp.  If we moved to job_container/tmpfs it looks like we'd be limited to /dev/shm only and not both /dev/shm and /tmp.  Is it possible with job_container/tmpfs to setup multiple private locations like /dev/shm and /tmp ? My read of the config docs and my initial read of code is that only one BasePath per either config or node group is allowed.  Maybe this needs to be an enhancement if that's not possible.
Comment 1 Nate Rini 2021-03-16 17:05:13 MDT 
Trey,

I'm going to put this as a SEV5 (RFE) while I look into it. This plugin is new, so the limitations have not all been tested.

Thanks,
--Nate
Comment 2 Nate Rini 2021-03-17 15:30:01 MDT 
Trey,

The paths (/dev/shm and /tmp) are hardcoded in the plugin currently. I will start the normal process for RFE review.

Thanks,
--Nate
Comment 6 Josko Plazonic 2021-03-18 15:39:48 MDT 
As far as I can tell /dev/shm is already handled by tmpfs plugin and is not configurable (i.e., plugin always does it):

job_container_tmpfs.c:static int _mount_private_shm(void)
job_container_tmpfs.c:  rc = umount("/dev/shm");
job_container_tmpfs.c:          error("%s: umount /dev/shm failed: %s\n",
job_container_tmpfs.c:  rc = mount("tmpfs", "/dev/shm", "tmpfs", 0, NULL);
job_container_tmpfs.c:          error("%s: mounting private /dev/shm failed: %s\n",
job_container_tmpfs.c:                  rc = _mount_private_shm();
job_container_tmpfs.c:                          error("%s: could not mount private shm",

Our problem is that we want to have additional "private" temp directories. We currently do this with /tmp, /var/tmp and /var/lock. Maybe an option Dirs that can default to /tmp but allows more than that.

We actually had an earlier version of NERSC plugin that we modified to do that but the version that got merged has diverged too much for this to be directly useful.
Comment 7 Trey Dockendorf 2021-03-18 15:53:51 MDT 
For OSC, this RFE maybe not useful since this was initially opened not fully understanding how the job_container/tmpfs plugin worked.  I opened RFE #11135 that would actually allow OSC to use this plugin in our current environment.
Comment 8 Nate Rini 2021-03-18 15:58:06 MDT 
Trey,

I going to mark this as a duplicate of bug#11135 then.

Thanks,
--Nate

*** This ticket has been marked as a duplicate of ticket 11135 ***