Ticket 5661 - error: Database settings not recommended values: innodb_buffer_pool_size
Summary: error: Database settings not recommended values: innodb_buffer_pool_size
Status: RESOLVED FIXED
Alias: None
Product: Slurm
Classification: Unclassified
Component: slurmdbd (show other tickets)
Version: 18.08.0
Hardware: Linux Linux
: 4 - Minor Issue
Assignee: Broderick Gardner
QA Contact:
URL:
Depends on:
Blocks:
 
Reported: 2018-08-31 18:53 MDT by Kilian Cavalotti
Modified: 2020-06-25 23:26 MDT (History)
1 user (show)

See Also:
Site: Stanford
Slinky Site: ---
Alineos Sites: ---
Atos/Eviden Sites: ---
Confidential Site: ---
Coreweave sites: ---
Cray Sites: ---
DS9 clusters: ---
Google sites: ---
HPCnow Sites: ---
HPE Sites: ---
IBM Sites: ---
NOAA SIte: ---
NoveTech Sites: ---
Nvidia HWinf-CS Sites: ---
OCF Sites: ---
Recursion Pharma Sites: ---
SFW Sites: ---
SNIC sites: ---
Tzag Elita Sites: ---
Linux Distro: ---
Machine Name:
CLE Version:
Version Fixed: 18.08.1
Target Release: ---
DevPrio: ---
Emory-Cloud Sites: ---

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this ticket.
Description Kilian Cavalotti 2018-08-31 18:53:26 MDT 
Hi there,

We're using a datacenter planned maintenance over the week-end to upgrade Sherlock to 18.08.0.

One of the first things I noted when starting the new slurmdbd is this error message:

error: Database settings not recommended values: innodb_buffer_pool_size


In our MySQL config, we have:

innodb_buffer_pool_size=32G


Looking at the code (_check_database_variables() in accounting_storage_mysql.c), it looks like the error message is displayed if the values are not at least half the recommendation. 
The recommended value for innodb_buffer_pool_size appears to be 1GB, and 32GB is definitely larger than half of 1GB. :)

Maybe the _check_database_variables() function expects values from the config to be in bytes? Or is there something else?

I tried to set DebugLevelSyslog to debug2, but didn't see any additional message in the logs, only this:

Aug 31 17:52:10 sh-sl01 systemd: Started Slurm DBD accounting daemon.
Aug 31 17:52:10 sh-sl01 slurmdbd[171888]: error: Database settings not recommended values: innodb_buffer_pool_size
Aug 31 17:52:10 sh-sl01 slurmdbd[171888]: pre-converting step table for sherlock


Thanks!
--
Kilian
Comment 1 Tim Wickberg 2018-08-31 19:01:44 MDT 
Do you mind dropping into the database and running a quick query?

What we're relying on is the output from 

show variables like 'innodb_buffer_pool_size';

which should be in bytes. Unless for some reason it isn't on your system?

There should be some minor additional details at debug2, I'm not sure why you aren't seeing that though.

As you've presumably noticed, this is just an advisory and you can safely ignore the warning if you've done this configuration already.

- Tim
Comment 2 Kilian Cavalotti 2018-08-31 19:04:33 MDT 
(In reply to Tim Wickberg from comment #1)
> Do you mind dropping into the database and running a quick query?
> 
> What we're relying on is the output from 
> 
> show variables like 'innodb_buffer_pool_size';
> 
> which should be in bytes. Unless for some reason it isn't on your system?

You;re right, it's in bytes:

MariaDB [slurm_acct_db]> show variables like 'innodb_buffer_pool_size';
+-------------------------+-------------+
| Variable_name           | Value       |
+-------------------------+-------------+
| innodb_buffer_pool_size | 34359738368 |
+-------------------------+-------------+

That still seems larger than the hard-coded (mmpf) recommended value of 1073741824.

> There should be some minor additional details at debug2, I'm not sure why
> you aren't seeing that though.

Don't know either.
 
> As you've presumably noticed, this is just an advisory and you can safely
> ignore the warning if you've done this configuration already.

I assumed so. May be in that case it could be better to display this as a warning rather than an error?

Cheers,
--
Kilian
Comment 3 Tim Wickberg 2018-08-31 19:10:19 MDT 
> You;re right, it's in bytes:
> 
> MariaDB [slurm_acct_db]> show variables like 'innodb_buffer_pool_size';
> +-------------------------+-------------+
> | Variable_name           | Value       |
> +-------------------------+-------------+
> | innodb_buffer_pool_size | 34359738368 |
> +-------------------------+-------------+
> 
> That still seems larger than the hard-coded (mmpf) recommended value of
> 1073741824.

I see the mistake, we'll have this fixed in 18.08.1. You can ignore this for now, it's not worth patching out.
Comment 4 Kilian Cavalotti 2018-08-31 19:12:37 MDT 
(In reply to Tim Wickberg from comment #3)
> > You;re right, it's in bytes:
> > 
> > MariaDB [slurm_acct_db]> show variables like 'innodb_buffer_pool_size';
> > +-------------------------+-------------+
> > | Variable_name           | Value       |
> > +-------------------------+-------------+
> > | innodb_buffer_pool_size | 34359738368 |
> > +-------------------------+-------------+
> > 
> > That still seems larger than the hard-coded (mmpf) recommended value of
> > 1073741824.
> 
> I see the mistake, we'll have this fixed in 18.08.1. You can ignore this for
> now, it's not worth patching out.

Got it, thanks!

Cheers,
-- 
Kilian
Comment 12 Broderick Gardner 2018-10-04 09:52:38 MDT 
The fix is in 18.08.1, commit ba446ec62333eeec

Closing
Comment 13 Kilian Cavalotti 2018-10-04 10:04:39 MDT 
(In reply to Broderick Gardner from comment #12)
> The fix is in 18.08.1, commit ba446ec62333eeec

Thank you!
-- 
Kilian
Comment 14 Issam SAID 2020-06-25 23:26:10 MDT 
we are still seeing this in 20.02.x