Ticket 4190 - Consider extending X11 forwarding to allow arbitrary port forwarding and tunneling
Summary: Consider extending X11 forwarding to allow arbitrary port forwarding and tunn...
Status: OPEN
Alias: None
Product: Slurm
Classification: Unclassified
Component: Other (show other tickets)
Version: 17.11.x
Hardware: Linux Linux
: 5 - Enhancement
Assignee: Unassigned Developer
QA Contact:
URL:
Depends on:
Blocks:
 
Reported: 2017-09-26 11:52 MDT by Paul Edmon
Modified: 2019-10-16 08:34 MDT (History)
8 users (show)

See Also:
Site: Harvard University
Alineos Sites: ---
Atos/Eviden Sites: ---
Confidential Site: ---
Coreweave sites: ---
Cray Sites: ---
DS9 clusters: ---
HPCnow Sites: ---
HPE Sites: ---
IBM Sites: ---
NOAA SIte: ---
OCF Sites: ---
Recursion Pharma Sites: ---
SFW Sites: ---
SNIC sites: ---
Linux Distro: ---
Machine Name:
CLE Version:
Version Fixed:
Target Release: ---
DevPrio: ---
Emory-Cloud Sites: ---


Attachments

Note You need to log in before you can comment on or make changes to this ticket.
Description Paul Edmon 2017-09-26 11:52:07 MDT
https://github.com/harvardinformatics/spunnel

Here is what our informatics group wrote for this based on the Spank X11 plugin.
Comment 1 Tim Wickberg 2017-09-26 15:31:49 MDT
Thanks for the link Paul. Retagging as an enhancement request; it won't be part of the X11 implementation in 17.11 but may be something we explore in a future release.

cheers,
- Tim
Comment 2 Gordon Dexter 2019-10-15 16:04:26 MDT
Would this possibly include SSH Agent forwarding?
Comment 3 Tim Wickberg 2019-10-15 16:10:14 MDT
As a general update: the revamped X11 forwarding code in the Slurm 19.05 release could be extended to cover arbitrary network forwarding if desired. I haven't heard of a specific use case behind this as of yet, but the underlying primitives are all there.

(In reply to Gordon Dexter from comment #2)
> Would this possibly include SSH Agent forwarding?

Possibly, although I don't completely understand the use case for this.

This bug represents an outstanding enhancement request - there is no commitment from SchedMD to develop anything related to this at present.

- Tim
Comment 4 S Senator 2019-10-15 16:55:58 MDT
One possibly use case could be to set up tunnels for front-end views into an existing running jobs. Use case that we are exploring are a Jupyter notebook application and a back-end, steerable visualization job (Paraview.)

Thank you,
-Steve Senator

________________________________________
From: bugs@schedmd.com <bugs@schedmd.com>
Sent: Tuesday, October 15, 2019 4:10:14 PM
To: Senator, Steven Terry
Subject: [Bug 4190] Consider extending X11 forwarding to allow arbitrary port forwarding and tunneling

Tim Wickberg<mailto:tim@schedmd.com> changed bug 4190<https://bugs.schedmd.com/show_bug.cgi?id=4190>
What    Removed Added
CC              jess@schedMD.com

Comment # 3<https://bugs.schedmd.com/show_bug.cgi?id=4190#c3> on bug 4190<https://bugs.schedmd.com/show_bug.cgi?id=4190> from Tim Wickberg<mailto:tim@schedmd.com>

As a general update: the revamped X11 forwarding code in the Slurm 19.05
release could be extended to cover arbitrary network forwarding if desired. I
haven't heard of a specific use case behind this as of yet, but the underlying
primitives are all there.

(In reply to Gordon Dexter from comment #2<show_bug.cgi?id=4190#c2>)
> Would this possibly include SSH Agent forwarding?

Possibly, although I don't completely understand the use case for this.

This bug represents an outstanding enhancement request - there is no commitment
from SchedMD to develop anything related to this at present.

- Tim

________________________________
You are receiving this mail because:

  *   You are on the CC list for the bug.
Comment 5 Paul Edmon 2019-10-16 08:26:04 MDT
That is our use case here.  We wrote a spank plugin to do it based off 
the extant spank X11 plugin.  Since X11 is now native to slurm we figure 
this would be natural extension.

For most of our Jupyter notebook needs we've been using Open OnDemand, 
but the ability to generate tunnels is still handy for some cases.

-Paul Edmon-

On 10/15/19 6:55 PM, bugs@schedmd.com wrote:
>
> *Comment # 4 <https://bugs.schedmd.com/show_bug.cgi?id=4190#c4> on bug 
> 4190 <https://bugs.schedmd.com/show_bug.cgi?id=4190> from S Senator 
> <mailto:sts@lanl.gov> *
> One possibly use case could be to set up tunnels for front-end views into an
> existing running jobs. Use case that we are exploring are a Jupyter notebook
> application and a back-end, steerable visualization job (Paraview.)
>
> Thank you,
> -Steve Senator
>
> ________________________________________
> From:bugs@schedmd.com  <mailto:bugs@schedmd.com>  <bugs@schedmd.com  <mailto:bugs@schedmd.com>>
> Sent: Tuesday, October 15, 2019 4:10:14 PM
> To: Senator, Steven Terry
> Subject: [Bug 4190  <show_bug.cgi?id=4190>] Consider extending X11 forwarding to allow arbitrary port
> forwarding and tunneling
>
> Tim Wickberg<mailto:tim@schedmd.com> changed bug
> 4190<https://bugs.schedmd.com/show_bug.cgi?id=4190  <show_bug.cgi?id=4190>>
> What    Removed Added
> CCjess@schedMD.com  <mailto:jess@schedMD.com>
>
> Comment # 3  <show_bug.cgi?id=4190#c3><https://bugs.schedmd.com/show_bug.cgi?id=4190#c3  <show_bug.cgi?id=4190#c3>> on bug
> 4190<https://bugs.schedmd.com/show_bug.cgi?id=4190  <show_bug.cgi?id=4190>> from Tim
> Wickberg<mailto:tim@schedmd.com>
>
> As a general update: the revamped X11 forwarding code in the Slurm 19.05
> release could be extended to cover arbitrary network forwarding if desired. I
> haven't heard of a specific use case behind this as of yet, but the underlying
> primitives are all there.
>
> (In reply to Gordon Dexter fromcomment #2  <show_bug.cgi?id=4190#c2><show_bug.cgi?id=4190#c2>)
> > Would this possibly include SSH Agent forwarding?
>
> Possibly, although I don't completely understand the use case for this.
>
> This bug represents an outstanding enhancement request - there is no commitment
> from SchedMD to develop anything related to this at present.
>
> - Tim
>
> ________________________________
> You are receiving this mail because:
>
>    *   You are on the CC list for the bug.
> ------------------------------------------------------------------------
> You are receiving this mail because:
>
>   * You reported the bug.
>
Comment 6 Gordon Dexter 2019-10-16 08:34:43 MDT
(In reply to Tim Wickberg from comment #3)
> 
> (In reply to Gordon Dexter from comment #2)
> > Would this possibly include SSH Agent forwarding?
> 
> Possibly, although I don't completely understand the use case for this.

Agents are useful for keys that are encrypted, or not stored on the compute cluster.  They allow a developer to decrypt a key on their desktop and use it from a remote system (e.g. for git push/pull) without ever having it on disk (or even in memory) on that system.

I can definitely see a use for generic port forwarding (e.g. for Jupyter) as well.