Ticket 13968 - association exists in subaccount but not shown when using WithSubAccounts
Summary: association exists in subaccount but not shown when using WithSubAccounts
Status: RESOLVED TIMEDOUT
Alias: None
Product: Slurm
Classification: Unclassified
Component: Database (show other tickets)
Version: 21.08.4
Hardware: Linux Linux
: 3 - Medium Impact
Assignee: Oriol Vilarrubi
QA Contact:
URL:
Depends on:
Blocks:
 
Reported: 2022-04-29 09:22 MDT by ARC Admins
Modified: 2022-06-10 06:08 MDT (History)
0 users

See Also:
Site: University of Michigan
Slinky Site: ---
Alineos Sites: ---
Atos/Eviden Sites: ---
Confidential Site: ---
Coreweave sites: ---
Cray Sites: ---
DS9 clusters: ---
Google sites: ---
HPCnow Sites: ---
HPE Sites: ---
IBM Sites: ---
NOAA SIte: ---
NoveTech Sites: ---
Nvidia HWinf-CS Sites: ---
OCF Sites: ---
Recursion Pharma Sites: ---
SFW Sites: ---
SNIC sites: ---
Tzag Elita Sites: ---
Linux Distro: ---
Machine Name:
CLE Version:
Version Fixed:
Target Release: ---
DevPrio: ---
Emory-Cloud Sites: ---


Attachments

Note You need to log in before you can comment on or make changes to this ticket.
Description ARC Admins 2022-04-29 09:22:27 MDT
We are trying to make a web frontend to support self service for end users.  We just had a weird case we have not seen before where an association for a user in a subaccount doesn't show up when we pull the root account:

vikramg0 and vikramg1 are subaccounts of vikramg_root which is a child of the default root account.

sacctmgr list assoc account=vikramg0 format=User  | grep keneaton
keneaton   # user exists

sacctmgr list assoc account=vikramg1 format=User  | grep keneaton
  keneaton # user exists


sacctmgr list assoc account=vikramg_root format=User,Account WithSubAccounts | grep keneaton
  keneaton   vikramg1 


It is expected and behaves for all other users that we should see two associations for the user having access to both subaccounts.

Slurmdbd.log  doesn't show any errors related to this user which is used as testing as we are developing this portal.
Comment 1 Oriol Vilarrubi 2022-05-03 08:19:33 MDT
(In reply to ARC Admins from comment #0)
> We are trying to make a web frontend to support self service for end users. 

If you are doing this to develop a web frontend, maybe you are more interested is using slurmrestd rather than issuing sacctmgr commands, slurmrestd is a rest api from which you can access the same data as if using slurm commands.

> We just had a weird case we have not seen before where an association for a
> user in a subaccount doesn't show up when we pull the root account:
> 
> vikramg0 and vikramg1 are subaccounts of vikramg_root which is a child of
> the default root account.
> 
> sacctmgr list assoc account=vikramg0 format=User  | grep keneaton
> keneaton   # user exists
> 
> sacctmgr list assoc account=vikramg1 format=User  | grep keneaton
>   keneaton # user exists
> 
> 
> sacctmgr list assoc account=vikramg_root format=User,Account WithSubAccounts
> | grep keneaton
>   keneaton   vikramg1 
> 
> 
> It is expected and behaves for all other users that we should see two
> associations for the user having access to both subaccounts.
> 
> Slurmdbd.log  doesn't show any errors related to this user which is used as
> testing as we are developing this portal.

If I got you right the issue you are seeing here is that in the last command (sacctmgr list assoc account=vikramg_root format=User,Account WithSubAccounts) you do not see the second assocaition (the one with vikramg0) is that so?

I'll recreate your account structure and test it myslef to get more information about it.
Comment 2 Oriol Vilarrubi 2022-05-03 08:20:56 MDT
(In reply to Oriol Vilarrubi from comment #1)

> If you are doing this to develop a web frontend, maybe you are more
> interested is using slurmrestd rather than issuing sacctmgr commands,
> slurmrestd is a rest api from which you can access the same data as if using
> slurm commands.

I forgot to add here a link to the slurmrestd webpage: https://slurm.schedmd.com/slurmrestd.html if you are interested in it I can give you a more in depth explanation on how to set up and use it.
Comment 3 ARC Admins 2022-05-04 08:01:09 MDT
Yes correct, if we look at each subaccount it has the user associated with it, but when we look at the parent with the WithSubAccounts  option one isn't appearing.

We have been using this for hundreds of slurm accounts this is the first time we have seen this that we know of.  So I doubt you will be able to easily reproduce it because we don't know what we did that caused that behavior. We were testing our draft POST and DELETE API to add users to and from the subaccount in a test instance and got very confused when we started getting results we didn't expect.
Comment 4 Oriol Vilarrubi 2022-05-12 08:51:52 MDT
Hello,
I just tested the same with the latest 21.08 version (just because is the one I already had installed) and it is working fine for me:
jvilarru@torre:~$ sacctmgr show account sub1 sub2
   Account                Descr                  Org 
---------- -------------------- -------------------- 
      sub1                 sub1               parent 
      sub2                 sub2               parent 
jvilarru@torre:~$ sacctmgr list assoc account=sub1 format=User
      User 
---------- 
           
       foo 
jvilarru@torre:~$ sacctmgr list assoc account=sub2 format=User
      User 
---------- 
           
       foo 
jvilarru@torre:~$ sacctmgr list assoc account=parent format=User,Account WithSubAccounts
      User    Account 
---------- ---------- 
               parent 
                 sub1 
       foo       sub1 
                 sub2 
       foo       sub2 

I'll install slurm 21.08.4 in order to check if this is an already fixed bug, if that is the case then I'll identify it and tell you which 21.08 release you need for this to be fixed, otherwise that would mean that you have something not wright in your configuration or accounting. I'll get back to you with an answer.


Greetings.
Comment 5 Oriol Vilarrubi 2022-05-17 09:02:04 MDT
Hello,

I've tested 21.08.4 and sadly it still shows the correct way for me, that 99% means that you have something in your configuration that prevents that from working properly, could you please attach the output of the following commands so that I can better reproduce your environment:

scontrol show config

sacctmgr show config

Regards.
Comment 6 Oriol Vilarrubi 2022-05-31 08:13:30 MDT
Also, could you please create some dummy accounts and users just to test if this is a global issue or just with these accounts?

It would be something like this:

sacctmgr -i add account test_parent_account
sacctmgr -i add account test_sub1 parent=test_parent_account
sacctmgr -i add account test_sub2 parent=test_parent_account
sacctmgr -i add user account=test_sub1 foo
sacctmgr -i add user account=test_sub2 foo
sacctmgr list assoc account=test_parent_account format=User,Account WithSubAccounts
Comment 7 Oriol Vilarrubi 2022-06-10 06:08:52 MDT
Hello, 

I'm timing this bug out as we have not received response from you in more than 2 weeks. It can be reopened if needed without problem.

Regards.