Ticket 5727

I think you're interested in the --hint=[no]multithread option for srun and sbatch. Leave the nodes with hyperthreading enabled, and users can control how their tasks are bound themselves (or you as an admin can use a job submit plugin to do it for them):

A quick example with this node config:

NodeName=DEFAULT RealMemory=3000 Sockets=1 CoresPerSocket=4 ThreadsPerCore=2


$ srun -n8 whereami
0001 v1 - Cpus_allowed: 11      Cpus_allowed_list:      0,4
0006 v1 - Cpus_allowed: 88      Cpus_allowed_list:      3,7
0000 v1 - Cpus_allowed: 11      Cpus_allowed_list:      0,4
0005 v1 - Cpus_allowed: 44      Cpus_allowed_list:      2,6
0007 v1 - Cpus_allowed: 88      Cpus_allowed_list:      3,7
0003 v1 - Cpus_allowed: 22      Cpus_allowed_list:      1,5
0004 v1 - Cpus_allowed: 44      Cpus_allowed_list:      2,6
0002 v1 - Cpus_allowed: 22      Cpus_allowed_list:      1,5

$ srun -n8 --hint=multithread whereami
0007 v1 - Cpus_allowed: 80      Cpus_allowed_list:      7
0001 v1 - Cpus_allowed: 10      Cpus_allowed_list:      4
0005 v1 - Cpus_allowed: 40      Cpus_allowed_list:      6
0003 v1 - Cpus_allowed: 20      Cpus_allowed_list:      5
0004 v1 - Cpus_allowed: 04      Cpus_allowed_list:      2
0006 v1 - Cpus_allowed: 08      Cpus_allowed_list:      3
0000 v1 - Cpus_allowed: 01      Cpus_allowed_list:      0
0002 v1 - Cpus_allowed: 02      Cpus_allowed_list:      1

$ srun -n8 --hint=nomultithread whereami
0005 v2 - Cpus_allowed: 10      Cpus_allowed_list:      4
0006 v2 - Cpus_allowed: 02      Cpus_allowed_list:      1
0007 v2 - Cpus_allowed: 20      Cpus_allowed_list:      5
0000 v1 - Cpus_allowed: 01      Cpus_allowed_list:      0
0004 v2 - Cpus_allowed: 01      Cpus_allowed_list:      0
0002 v1 - Cpus_allowed: 02      Cpus_allowed_list:      1
0001 v1 - Cpus_allowed: 10      Cpus_allowed_list:      4
0003 v1 - Cpus_allowed: 20      Cpus_allowed_list:      5

Is this an acceptable solution?

A note on the output from comment 1: notice that without --hint=[no]multithread, the tasks are bound to the whole core, but there could be as many tasks on a core as there are hyperthreads on a core. With --hint=multithread, each task is bound to a single hyperthread. With --hint=nomultithread, each task is bound to a single core, and only a single task per core is permitted.


Also, I'm dropping the severity to sev-4. I was told that your site joined support a little early (before training in October) but that all your bugs should be sev-4 for now. This is just a friendly reminder to keep things at sev-4 for now.

I haven't read the contract or email, so I don't know exactly when you'll be able to submit severity 1-3 bugs - just keep in contact with whoever is managing the contract with you (I assume Jacob or Jess).

I'm closing this as resolved/infogiven.

I found out that your "official" support contract period starts on Oct 1 (when you can submit any severity ticket), and we're supporting sev-4 tickets until then.

(Forgot to close the ticket - actually closing)

Feel free to reopen if you have further questions regarding hyperthreading.