Ticket 8605

Hi,

(In reply to Vlad Doster from comment #0)
> Is there any way to restore an ElasticSearch index from SlurmDB?

Not at present.

The jobcomp/elasticsearch plugin is loaded in and run from the slurmctld daemon. When enabled in the configuration, the plugin is given a subset of the information for each finished job and attempts to index it to the elastic server. There is no communication happening between the plugin and slurmdbd, and Slurm doesn't have any functionality to pull job entries from the database, adequate them to a JSON format and send them to the elastic server.

sacct client command can pull jobs and steps from the Slurm database, but it doesn't interact with the elastic server.

> There seems to be some fields aren't saved in SlurmDB, but are saved in 
> ElasticSearch? 

Yes. This is also true the other way around; there are fields saved in the Slurm database and not in the elastic server. 

There are also synthetic fields created by the elastic plugin during its execution, like i.e. parent_accounts which is a string in the form "/accountA/accountB/accountC" representing the hierarchy path to the account used by the job.

> Is the only option to write a script to transfer the data over and then set
> missing values to NULL?

Slurm doesn't provide such functionality, so the "transfer" would need to happen manually by using an external script or tool. I say "transfer" because such a task wouldn't be just pulling off info from the database and send it to elastic. 

This would require preprocessing the information into a JSON format. There would also be missing fields that ctld doesn't send to slurmdbd and thus isn't stored in the database (but it is in the elastic server if the plugin is enabled) and there would also be missing synthetic fields.

On top of that, we usually discourage to directly access the Slurm database.
 
Do you have any other questions? thanks.

Hello, thank you for the insight.