Ticket 3625 - Improve mpi/PMIx plugin performance
Summary: Improve mpi/PMIx plugin performance
Status: RESOLVED DUPLICATE of ticket 3103
Alias: None
Product: Slurm
Classification: Unclassified
Component: Contributions (show other tickets)
Version: 17.11.x
Hardware: Linux Linux
: 4 - Minor Issue
Assignee: Unassigned Developer
QA Contact:
URL:
Depends on:
Blocks:
 
Reported: 2017-03-27 13:05 MDT by Artem Polyakov
Modified: 2017-03-27 19:35 MDT (History)
0 users

See Also:
Site: -Other-
Slinky Site: ---
Alineos Sites: ---
Atos/Eviden Sites: ---
Confidential Site: ---
Coreweave sites: ---
Cray Sites: ---
DS9 clusters: ---
Google sites: ---
HPCnow Sites: ---
HPE Sites: ---
IBM Sites: ---
NOAA SIte: ---
NoveTech Sites: ---
Nvidia HWinf-CS Sites: ---
OCF Sites: ---
Recursion Pharma Sites: ---
SFW Sites: ---
SNIC sites: ---
Tzag Elita Sites: ---
Linux Distro: ---
Machine Name:
CLE Version:
Version Fixed:
Target Release: ---
DevPrio: ---
Emory-Cloud Sites: ---


Attachments
Set of patches (37.54 KB, application/x-bzip)
2017-03-27 13:05 MDT, Artem Polyakov
Details

Note You need to log in before you can comment on or make changes to this ticket.
Description Artem Polyakov 2017-03-27 13:05:08 MDT
Created attachment 4252 [details]
Set of patches

This set of patches introduce two important features:
1. Direct TCP connection between stepd's
2. Non-blocking send/receives.

Those two features are self-contained and improve the performance of the PMIX working in the direct-modex mode.
They also are strategic steps for subsequent near future improvements.

We have a week+ of successful MTT runs for our internal test suite containing 1500+ MPI applications, so we have some level of confidence that it no regression was introduced. We will be running this on daily basis.

Application of this patchset:
* Please apply with git am to preserve commit messages. Some of them include related performance data that I'd like to store for the records.
* Note that after patch #6 (0006-mpi-pmix-Rework-communication-framework-step-1.patch) autogen.sh should be invoked because this patch introduces changes to the build system (because of new *.c/*.h files).
Comment 1 Danny Auble 2017-03-27 13:18:28 MDT
Artem, is this any different than bug 3103?  If not please attach this set there and close this one down.  If they are different please let me know if we can close 3103 down.
Comment 2 Artem Polyakov 2017-03-27 13:21:36 MDT
That is right, I forgot about the original one. Sorry.
Comment 3 Artem Polyakov 2017-03-27 13:23:01 MDT
marking this one as duplicated

*** This ticket has been marked as a duplicate of ticket 3103 ***