When trying to determine the source of a memory leak in our code, I ran the clang static analyzer tool (http://clang-analyzer.llvm.org/) on Slurm compiled with --enable-debug and --enable-native-cray. It found 218 potential issues, with 9 of them in our plugin code. The file is 10MB, too large to add as an attachment, so I uploaded it to my google drive. Let me know if you cannot access it. https://docs.google.com/file/d/0B9d4ZHTQ9H_FTW85bHNrWXVxcTQ/edit?usp=sharing After unpacking, simply open the index.html file in a web browser and you should be able to view all of the issues. Some may be false alarms, but they are all worth checking out in my opinion.
Unless you build slurm (run configure) with "--enable-memory-leak-debug" then a lot of memory buffers get cached. With that flag, Slurm will run considerably slower. Please rebuild and retest. Unless clang finds leaks that valgrind does not, Slurm should be free of memory leaks.
I'll rebuild with that configure flag set. And yes, clang can find many leaks that valgrind does not, since clang is a static analyzer. It runs during compile time, not during runtime, so it can explore paths that aren't taken during runtime.
Here are the results when compiled with --enable-memory-leak-debug: https://docs.google.com/file/d/0B9d4ZHTQ9H_FejRlYTdFNG1GdDA/edit?usp=sharing Still found 214 errors.
FYI: I've reviewed about 50 of these (ignoring the dead assignments) without finding anything yet that will cause harm. Good stuff to harden the code, but nothing yet that's a game changer.
An update: Ignoring the dead assignments (which are arguably really low priority), it's down to 37 problems as reported by CLANG, many of which are false positives.
I'm going to close this ticket. Clang did find a couple of real bugs, but virtually all of its findings were false positives. There are a couple of false positives that I can't make go away and there are still about 100 dead assignments that I'm not going to worry about.