Ticket 3234 - Dynamic bf_interval feature
Summary: Dynamic bf_interval feature
Status: RESOLVED INFOGIVEN
Alias: None
Product: Slurm
Classification: Unclassified
Component: Scheduling (show other tickets)
Version: 16.05.6
Hardware: Linux Linux
: 4 - Minor Issue
Assignee: Felip Moll
QA Contact:
URL:
Depends on:
Blocks:
 
Reported: 2016-11-03 02:28 MDT by CSC sysadmins
Modified: 2019-06-03 00:58 MDT (History)
2 users (show)

See Also:
Site: CSC - IT Center for Science
Slinky Site: ---
Alineos Sites: ---
Atos/Eviden Sites: ---
Confidential Site: ---
Coreweave sites: ---
Cray Sites: ---
DS9 clusters: ---
Google sites: ---
HPCnow Sites: ---
HPE Sites: ---
IBM Sites: ---
NOAA SIte: ---
NoveTech Sites: ---
Nvidia HWinf-CS Sites: ---
OCF Sites: ---
Recursion Pharma Sites: ---
SFW Sites: ---
SNIC sites: ---
Tzag Elita Sites: ---
Linux Distro: ---
Machine Name:
CLE Version:
Version Fixed:
Target Release: ---
DevPrio: ---
Emory-Cloud Sites: ---

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this ticket.
Description CSC sysadmins 2016-11-03 02:28:42 MDT 
Hi,

I really hate that every now and then I need manually tune backfill parameters depending on the queue depth. Slurm knows internally (sdiag) how long generally backfill loop takes and it could adjust it shorter or longer depending on the load.
Something like bf_dynamic_interval_window=20-1200
Comment 1 Tim Wickberg 2016-11-07 16:18:37 MST 
(In reply to Tommi Tervo from comment #0)
> Hi,
> 
> I really hate that every now and then I need manually tune backfill
> parameters depending on the queue depth. Slurm knows internally (sdiag) how
> long generally backfill loop takes and it could adjust it shorter or longer
> depending on the load.
> Something like bf_dynamic_interval_window=20-1200

If I understand you correctly, you'd like the backfill scheduler to always run a complete pass, and then immediately start again once that pass has completed?
Comment 2 CSC sysadmins 2016-11-14 07:00:57 MST 
Yes, backfill could run continuously.
Comment 3 Tim Wickberg 2017-03-07 18:38:42 MST 
I'm remarking this as a possible enhancement for 17.11 to add a flag to keep the backfill scheduler running continually.
Comment 4 Doug Jacobsen 2017-05-15 10:23:48 MDT 
See bug 3808 for a possible (user contributed) implementation that is similar to the functionality discussed here.
Comment 5 Felip Moll 2019-05-31 04:01:40 MDT 
Hi Tommi,

This enhancement has been sitting there for a long time.
What Doug suggested in bug 3808 would solve the commented problem here.

The idea is to set

bf_interval = Time between backfill cycles
bf_max_time = Max time spent in the backfill including sleeps.

Setting a large bf_max_time will help to reach the queue. To avoid responsiveness problem, setting max_rpc_cnt is advised.


After you confirm there's not any objection on your side, I will close this bug.
Comment 6 CSC sysadmins 2019-06-03 00:58:07 MDT 
Hi,
 
I'll close this bug