Ticket 7626

Hi, Matt can you tell me what you have set in your slurm.conf for:

AccountingStorageEnforce=

Also, would you clarify the associations a bit for me.

Usera has accounts accounta however, Usera submits to accountb but the jobs account is automatically switched back to accounta for this user?

If you are enforcing association, limits or qos via the AccountingStorageEnforce parameters then this would not be possible.

AccountingStorageEnforce=safe,associations,limits,qos

https://slurm.schedmd.com/accounting.html#slurm-accounting-configuration-after-build

sbatch -t1 --wrap="sleep 20" -A biot
sbatch: error: Batch job submission failed: Invalid account or account/partition combination specified

Thanks Jason - I'll take a look at the documentation around AccountingStorageEnforce.  In testing, I'm getting the same behavior (sbatch rejects my job when submitting to a real account which I don't have access to or a made-up account).

The case I'm looking at is the user who submitted to one account (vikramg) ended up running on a different account (engin).  I'm not seeing anything interesting in the slurmctld or slurmdbd logs for this job which would suggest why the charged account changed.  User has access to both (as of this morning) ; I'm not sure of the time sequence when this account was created and the user was given access to it (vikramg).

From Elasticsearch:
```
#!/bin/bash
#SBATCH --job-name=6_1_10fs
#SBATCH --mail-type=BEGIN, END
#SBATCH --cpus-per-node=4
#SBATCH --nodes=1
#SBATCH --mem-per-cpu=3000m
#SBATCH --time=2:00:00
#SBATCH --account=vikramg
#SBATCH --partition=standard
python compute_1_10fs.py
```

sacct output:
```
       JobID    JobName  Partition    Account  AllocCPUS      State ExitCode
------------ ---------- ---------- ---------- ---------- ---------- --------
53772          6_1_10fs   standard      engin          1 OUT_OF_ME+    0:125
53772.batch       batch                 engin          1 OUT_OF_ME+    0:125
53772.extern     extern                 engin          1  COMPLETED      0:0
```

Thanks,
  - Matt

Matt - I doubled checked in the source to verify expected behavior. If you are not doing any enforcement then the behavior is to copy the default account so that we do not have invalid data in the account.

This happens in _job_create

We check the association and if it is not valid then we null out the account and set it based on the association we find in the database.
...
>                (void) assoc_mgr_fill_in_assoc(acct_db_conn, &assoc_rec,
>                                               accounting_enforce, &assoc_ptr,
>                                               false);
>                if (assoc_ptr) {
>                        info("%s: account '%s' has no association for user %u "
>                             "using default account '%s'",
>                             __func__, job_desc->account, job_desc->user_id,
>                             assoc_rec.acct);
>                        xfree(job_desc->account);

...
>        if (job_desc->account == NULL)
>                job_desc->account = xstrdup(assoc_rec.acct);
...

So if you wish to enforce and deny submit then you will need to set AccountingStorageEnforce as mentioned in comment 1. I will proceed to close this out but if you have followup questions please feel free to reopen as needed.

Thanks Jason - I'll keep looking.

fwiw, we have had those options set in our slurm.conf.

AccountingStorageEnforce=associations,limits,safe,qos

Thanks,
   - Matt