Hi, We would like request new fields to be added to the ElasticSearch plugin so that we can do better long accounting for our jobs. The fields we would like are: — Critical to have: MaxRSS ReqMem TotalCPU — Would really like to have: MaxDiskRead MaxDiskWrite MaxVMSize ReqTres — Nice to have: AllocGRES AllocTres MaxPages Priority SystemCPU Kind Regards --Mick
Hi Mick - We received your feature request. We currently do not have plans to expand this right now. Normally features like these are paid engagements. Is this something that your site would like to sponsor?
Hi Jason, I need to talk to my team and see how to proceed. I'll follow-up in a few days. --Mick
Hi Mick - I have discussed this feature request internally and we believe we can accommodate this request without any sponsorship from your end, although it would have to be part of 20.11 or later. We will keep you posted on the slated release.
(In reply to Jason Booth from comment #4) > Hi Mick - I have discussed this feature request internally and we believe we > can accommodate this request without any sponsorship from your end, although > it would have to be part of 20.11 or later. We will keep you posted on the > slated release. Hi Jason, That's great, please let me know an ETA and I'll watch for new releases. We do plan to upgrade later this year and we'll probably wait until this feature-set is included, unless you advise otherwise. Kind regards --Mick
Mick, >We do plan to upgrade later this year and we'll probably wait until this feature-set is included, unless you advise otherwise. 20.02 will be released very soon so you will want to upgrade to at least 19.05 to stay within our support model. When 20.02 is released 18.08 will no longer receive support. In regards to the timeline, that will have to wait until after we release 20.02 since we are working on getting that release out the door.
Hi Jason, We're working on upgrading our Slurm instance real soon now. Do you expect these ES features in an upcoming release? Thanks --Mick
Hi Jason, Was this implemented in 20.11, I didn't find any specifics in release notes. Thanks!
Any progress on this? Thanks --Mick
Hi, To summarize: tres_[alloc|req] fields are available since Slurm 20.11.0 (commit 564768643a). tres_[alloc|req]_raw fields are available since Slurm 23.11.1 (commit a53d2afa3e). priority will be available since the next Slurm major release (commit d02c4d65cf). We discarded adding a separate field analogous to what sacct displays for ReqMem since that info could be derived from tres_req[_raw]. The addition of the above fields are natural additions in concordance of the jobcomp plugin design, meaning they are accessible from the job_record_t structure handed to its interface. Unfortunately, the rest of the requested fields can't directly be added as their information is not part of the job_record_t, but instead derived from slurmdb_stats_t, slurmdb_job_rec_t or various slurmdb_step_rec_t aggregations, which are structs designed to work with the accounting_storage plugins and sacct client-side. Making them available to the jobcomp plugins would fall out of the scope of this bug. They could get potentially added in the future, but it'd require refactor the jobcomp plugin interface to make this universally available instead of splitting it up, and that's a bunch more work, so would fall under RFE territory. Please, let us know if you need anything else from this bug. Otherwise, I'll go ahead and tag it as resolved. And sorry this took a little longer than expected.
