When originally adding these nodes, I incorrectly set TmpDisk=1000000 when the node has 10TB for TmpDisk. Today someone noticed and I corrected the nodes config in slurm.conf to be TmpDisk=10000000, but sinfo still thinks it is 1000000 [I am root!@frankie:(bare):gitolite3@localhost:slurm-config-bruno(master):etc]# scontrol show node cpu-a-1 | grep TmpDisk State=IDLE ThreadsPerCore=1 TmpDisk=10000000 Weight=200 Owner=N/A MCS_label=N/A [I am root!@frankie:(bare):gitolite3@localhost:slurm-config-bruno(master):etc]# sinfo -No "%n %d" | grep cpu-a-1 cpu-a-1 1000000 cpu-a-1 1000000 cpu-a-1 1000000 [I am root!@frankie:(bare):gitolite3@localhost:slurm-config-bruno(master):etc]# sinfo -NO 'NodeList,Disk' | grep cpu-a-1 cpu-a-1 1000000 cpu-a-1 1000000 cpu-a-1 1000000 The Slurm config for these nodes is: NodeName=cpu-a-1 CPUs=128 Boards=1 SocketsPerBoard=2 CoresPerSocket=64 ThreadsPerCore=1 RealMemory=4096000 TmpDisk=10000000 Feature=cpu,compute,largemem,amd_7h12 Weight=200 State=UNKNOWN NodeName=cpu-a-2 CPUs=128 Boards=1 SocketsPerBoard=2 CoresPerSocket=64 ThreadsPerCore=1 RealMemory=4096000 TmpDisk=10000000 Feature=cpu,compute,largemem,amd_7h12 Weight=200 State=UNKNOWN I've tried changing the format options for sinfo, but I always wind up a 0 short of the actual setting. What am I doing wrong here?
Looks like i'm late to this party, this is probably a duplicate of 16248 and may not have anything to do with me having originally set the TmpDisk value incorrectly.
Hello griznog, I can replicate this and I'm working on it. I'll keep you updated.
Hello griznog, This issue was due to the character buffer that would display the TmpDisk values in sinfo was a little short. We've increased the character buffer, so this behavior has been fixed in commit b8c6c9c0, which is in ahead of 24.11.1. I'll be closing this ticket now.
