Ticket 3625

Summary: Improve mpi/PMIx plugin performance
Product: Slurm Reporter: Artem Polyakov <artpol84>
Component: ContributionsAssignee: Unassigned Developer <dev-unassigned>
Status: RESOLVED DUPLICATE QA Contact:
Severity: 4 - Minor Issue    
Priority: ---    
Version: 17.11.x   
Hardware: Linux   
OS: Linux   
Site: -Other- Slinky Site: ---
Alineos Sites: --- Atos/Eviden Sites: ---
Confidential Site: --- Coreweave sites: ---
Cray Sites: --- DS9 clusters: ---
Google sites: --- HPCnow Sites: ---
HPE Sites: --- IBM Sites: ---
NOAA SIte: --- NoveTech Sites: ---
Nvidia HWinf-CS Sites: --- OCF Sites: ---
Recursion Pharma Sites: --- SFW Sites: ---
SNIC sites: --- Tzag Elita Sites: ---
Linux Distro: --- Machine Name:
CLE Version: Version Fixed:
Target Release: --- DevPrio: ---
Emory-Cloud Sites: ---
Attachments: Set of patches

Description Artem Polyakov 2017-03-27 13:05:08 MDT
Created attachment 4252 [details]
Set of patches

This set of patches introduce two important features:
1. Direct TCP connection between stepd's
2. Non-blocking send/receives.

Those two features are self-contained and improve the performance of the PMIX working in the direct-modex mode.
They also are strategic steps for subsequent near future improvements.

We have a week+ of successful MTT runs for our internal test suite containing 1500+ MPI applications, so we have some level of confidence that it no regression was introduced. We will be running this on daily basis.

Application of this patchset:
* Please apply with git am to preserve commit messages. Some of them include related performance data that I'd like to store for the records.
* Note that after patch #6 (0006-mpi-pmix-Rework-communication-framework-step-1.patch) autogen.sh should be invoked because this patch introduces changes to the build system (because of new *.c/*.h files).
Comment 1 Danny Auble 2017-03-27 13:18:28 MDT
Artem, is this any different than bug 3103?  If not please attach this set there and close this one down.  If they are different please let me know if we can close 3103 down.
Comment 2 Artem Polyakov 2017-03-27 13:21:36 MDT
That is right, I forgot about the original one. Sorry.
Comment 3 Artem Polyakov 2017-03-27 13:23:01 MDT
marking this one as duplicated

*** This ticket has been marked as a duplicate of ticket 3103 ***