in the contributed seff module this line of perl code exists if (exists $step->{'stats'} && exists $step->{'stats'}{'tres_usage_in_tot'}) { i'm trying to build a tool for job efficiency which kinda of does the what seff does, but using the --json output from sacct. however, i cannot figure out how the tres_usage_in data is produced. when i look at the sacct json for a job step, i see something like (sorry can't cut/paste) { job_id: 1 steps: [ { tres: { requested: { max: [] min: [] average: [] total: [] }, consumed: { max: [] min: [] average: [] total: [] }, allocated: [ ] } } ] under each of the max/min/avg/tot there are subs for cpu/disk/mem/energy/pages but it seems inconsistent. i naively would think i could look in the consumed section of a step and compare that to the allocated/requested part and see how efficient a job was based on the resources asked for however, in my steps the consumed part only lists energy/disk, no cpu/mem/nodes/pages/etc so my question is, is something missing from the json output from sacct? if not can you tell me where/how the calculations for 'tres_usage' are done
Please upload your slurm.conf, as this contains information about what information is tracked.
(In reply to Ethan Simmons from comment #1) > Please upload your slurm.conf, as this contains information about what > information is tracked. i can't do that, the environment is on a protected network. however, i believe what your looking for is accountingstoragetres = cpu,mem,energy,node,billing,fs/disk,vmem,pages,gres/gpu,gres/gpumem,gres/gpuutil jobacctgatherfrequency=task=50,energy=60,network=60,filesystem=60 jobacctgathertype=jobacct_gather/cgroup i did poke at this a little more after i opened the ticket. it seems like what's actually missing from the JSON is the TRESUsage fields. when i run sacct --helpformat | grep TRES i can see a list of TRESUsageIn/Out and when i run something like 'sacct -o jobid,tresusageinmax' i do see values. however those fields doesn't seem to end up natively in the --json output for sacct. or atleast i don't understand how to calculate it ideally, every field listed in --helpformat for sacct should be in the --json output
Coming back to this. The json option uses the data parser, which is different than the options available using --format (see link 1). If you want specific values, use --format. If you want the output to be machine friendly, use --parsable (link 2). The json option is available for those that want output similar to the rest d, as this uses the data parser extensively. For your uses, I would recommend a command like this: > $ sacct -j 24 -P -o jobid,partition,account > JobID|Partition|Account > 24|primary|test3 Where you can change the format parameter to meet your needs. The format isn't json, but it is closer to csv. Let me know if any other questions come up here. [1] https://slurm.schedmd.com/sacct.html#OPT_json [2] https://slurm.schedmd.com/sacct.html#OPT_parsable
Following up, are we ok to close this ticket?
i don't agree with your assessment, but i don't have time to look at this at the moment, so you can close this ticket.
Feel free to open the ticket again if you want to revisit this. Closing
