Ticket 12948 - Getting number of jobs per account with sacct and sreport
Summary: Getting number of jobs per account with sacct and sreport
Status: RESOLVED INFOGIVEN
Alias: None
Product: Slurm
Classification: Unclassified
Component: Accounting (show other tickets)
Version: 21.08.4
Hardware: Linux Linux
: 4 - Minor Issue
Assignee: Scott Hilton
QA Contact:
URL:
Depends on:
Blocks:
 
Reported: 2021-12-02 08:39 MST by ARC Admins
Modified: 2021-12-08 13:08 MST (History)
0 users

See Also:
Site: University of Michigan
Slinky Site: ---
Alineos Sites: ---
Atos/Eviden Sites: ---
Confidential Site: ---
Coreweave sites: ---
Cray Sites: ---
DS9 clusters: ---
Google sites: ---
HPCnow Sites: ---
HPE Sites: ---
IBM Sites: ---
NOAA SIte: ---
NoveTech Sites: ---
Nvidia HWinf-CS Sites: ---
OCF Sites: ---
Recursion Pharma Sites: ---
SFW Sites: ---
SNIC sites: ---
Tzag Elita Sites: ---
Linux Distro: ---
Machine Name:
CLE Version:
Version Fixed:
Target Release: ---
DevPrio: ---
Emory-Cloud Sites: ---

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this ticket.
Description ARC Admins 2021-12-02 08:39:57 MST 
Hello,

I have, what I believe is, a simple question. I want to get the number of non-duplicate jobs for an account in a given month with sacct. Presently, I use something like:

```
sacct -nPXDT -S $STARTDATE -E $ENDDATE -M $CLUSTER -A $ACCOUNT
```

This seems to be straight-forward and reliable. And, as I understand it, would match what sreport does on the back-end when calculating usage for a given account in a given time frame. 

If I were to then try to use sreport to get that same information (i.e. number of jobs for a given account in a given time frame), would the following be expected to work and deliver analogous results:

```
sreport -nP job SizesByAccount -M $CLUSTER Start=$STARTDATE End=$ENDDATE PrintJobCount accounts=$ACCOUNT
```

Regards,

David
Comment 1 Scott Hilton 2021-12-02 16:21:32 MST 
David,

First of all sacct -D literally means include the duplicates. Specifically the duplicate jobids.
https://slurm.schedmd.com/sacct.html#OPT_duplicates

In the case a job is requeued and started again, you will have two records with the same jobid both of which used resources, though the first didn't run to completion. This is the most common way to get duplicate job ids

The sreport script skips all jobs that don't have any elapsed time. It also includes duplicate jobs.

sacct -nPXDT -S $STARTDATE -E $ENDDATE -M $CLUSTER -A $ACCOUNT --format=jobid,elapsed | grep -x 00:00:00

-Scott
Comment 2 ARC Admins 2021-12-03 06:45:36 MST 
(In reply to Scott Hilton from comment #1)

> In the case a job is requeued and started again, you will have two records
> with the same jobid both of which used resources, though the first didn't
> run to completion. This is the most common way to get duplicate job ids

This is what we (Umich) want, in the case of billing (which is something our site has to do), since a job may use resources, get re-queued, and earn a charge.

> The sreport script skips all jobs that don't have any elapsed time. It also
> includes duplicate jobs.
> 
> sacct -nPXDT -S $STARTDATE -E $ENDDATE -M $CLUSTER -A $ACCOUNT
> --format=jobid,elapsed | grep -x 00:00:00

At first I was confused on what you are getting at. I'm only one cup of coffee into the day, so let's see if I get this right. What I *think* you're getting at is that sreport grabs the duplicate jobs by default, whereas sacct has to be _told_ to do that (with the -D flag). And sreport also omits jobs with 0 ElapsedRaw by default, and sacct does _not_. Given these two statements, it is not reasonable to expect analogous output/results from the two commands I initially sent. Would that be true?


David
Comment 3 Scott Hilton 2021-12-03 11:01:54 MST 
David,

I see that my last paragraph was rather confusing.

Yes, the two commands you sent will not match. 

However, the command I sent at the end of the last message should match sreport. Except I wrote grep -x instead of grep -v which is what I meant to write. Also grep -c will give the line count, which is what you are looking for anyway.
>sacct -nPXDT -S $STARTDATE -E $ENDDATE -M $CLUSTER -A $ACCOUNT --format=jobid,elapsed | grep -cv 00:00:00

-Scott
Comment 4 Scott Hilton 2021-12-08 10:53:54 MST 
David,

Do you have anymore questions on this ticket?

-Scott
Comment 5 ARC Admins 2021-12-08 11:27:50 MST 
(In reply to Scott Hilton from comment #4)
> David,
> 
> Do you have anymore questions on this ticket?
> 
> -Scott

Scott,

I do not. I apologize. I thought I closed it yesterday as RESOLVED INFO GIVEN.

Thanks for checking.

David
Comment 6 Scott Hilton 2021-12-08 13:08:47 MST 
Closing Ticket