[CP2K-user] [CP2K:13765] CP2K 7.1 memory leak

[Paul Schwarz]

Hi Ole,

Regarding toolchain:
Yes, that was the right issue. I added the codelines from the updated 
toolchain script to my own and compiled a pdbg binary.

Regarding the memory leaks:
I ran a geometry optimization for two different systems with the pdbg 
binary: one that does not run out of memory before finishing, and one that 
does run out of memory before finishing.

In the first case, I got messages from the leak sanitizer in the end, and I 
attach the output at the end of this post.
Since I run my calculations with 'nohup ... &', I got the sanitizer 
messages written to nohup.out.
The leaks seem to come from
    /usr/lib/x86_64-linux-gnu/liblsan.so
    /opt/intel_.../linux/mpi/intel64/libfabric/lib/prov/librxm-fi.so
    /lib/x86_64-linux-gnu/libc.so

However, I'm not familiar with debugging code, and it seems to me that 1280 
bytes of leaked memory in 8 objects is not much.

In the second case, though, the leak sanitizer did not write anything to 
nohup.out and the simulation just crashed as before, as it ran out of 
memory.
I thought that there might be some problem, since I'm running this 
calculation via the ASE interface and maybe some information got lost, I 
don't know.
So I restarted the simulation, but now with the standard cp2k.pdbg, and not 
cp2k_shell.pdbg.
Here, I'm currently tracking the memory usage of my machine and I see that 
the memory usage increases from ~24 GB to 35 GB over rougly 170 minutes.

Even though I did not track the memory usage for the first case, I think 
that the memory usage must have grown in the same way.
And I don't really understand how the reported 1280 bytes of leaked memory 
could lead to such a large increase of memory usage.


Best regards
Paul


========================================================
Leak sanitizer output:

==42842==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 1280 byte(s) in 8 object(s) allocated from:
    #0 0x7f09c65047e3  (/usr/lib/x86_64-linux-gnu/liblsan.so.0+0xf7e3)
    #1 0x7f09318f36b7 in rxm_cmap_alloc_handle 
(/opt/devel/intel/compilers_and_libraries_2019.5.281/linux/mpi/intel64/libfabric/lib/prov/librxm-fi.so+0x86b7)

Direct leak of 27 byte(s) in 5 object(s) allocated from:
    #0 0x7f09c6503acb in __interceptor_malloc 
(/usr/lib/x86_64-linux-gnu/liblsan.so.0+0xeacb)
    #1 0x7f09c34a8a29 in __strdup (/lib/x86_64-linux-gnu/libc.so.6+0x9da29)

SUMMARY: LeakSanitizer: 1307 byte(s) leaked in 13 allocation(s).

=================================================================
==42843==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 1280 byte(s) in 8 object(s) allocated from:
    #0 0x7fe1887c47e3  (/usr/lib/x86_64-linux-gnu/liblsan.so.0+0xf7e3)
    #1 0x7fe0f3bf36b7 in rxm_cmap_alloc_handle 
(/opt/devel/intel/compilers_and_libraries_2019.5.281/linux/mpi/intel64/libfabric/lib/prov/librxm-fi.so+0x86b7)

Direct leak of 27 byte(s) in 5 object(s) allocated from:
    #0 0x7fe1887c3acb in __interceptor_malloc 
(/usr/lib/x86_64-linux-gnu/liblsan.so.0+0xeacb)
    #1 0x7fe185768a29 in __strdup (/lib/x86_64-linux-gnu/libc.so.6+0x9da29)

SUMMARY: LeakSanitizer: 1307 byte(s) leaked in 13 allocation(s).

=================================================================
==42844==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 1280 byte(s) in 8 object(s) allocated from:
    #0 0x7f4564a917e3  (/usr/lib/x86_64-linux-gnu/liblsan.so.0+0xf7e3)
    #1 0x7f44cfef36b7 in rxm_cmap_alloc_handle 
(/opt/devel/intel/compilers_and_libraries_2019.5.281/linux/mpi/intel64/libfabric/lib/prov/librxm-fi.so+0x86b7)

Direct leak of 27 byte(s) in 5 object(s) allocated from:
    #0 0x7f4564a90acb in __interceptor_malloc 
(/usr/lib/x86_64-linux-gnu/liblsan.so.0+0xeacb)
    #1 0x7f4561a35a29 in __strdup (/lib/x86_64-linux-gnu/libc.so.6+0x9da29)

SUMMARY: LeakSanitizer: 1307 byte(s) leaked in 13 allocation(s).

=================================================================
==42845==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 1280 byte(s) in 8 object(s) allocated from:
    #0 0x7faa81dbf7e3  (/usr/lib/x86_64-linux-gnu/liblsan.so.0+0xf7e3)
    #1 0x7fa9ed1f36b7 in rxm_cmap_alloc_handle 
(/opt/devel/intel/compilers_and_libraries_2019.5.281/linux/mpi/intel64/libfabric/lib/prov/librxm-fi.so+0x86b7)

Direct leak of 27 byte(s) in 5 object(s) allocated from:

=================================================================
==42846==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 1280 byte(s) in 8 object(s) allocated from:
    #0 0x7f54b0dcb7e3  (/usr/lib/x86_64-linux-gnu/liblsan.so.0+0xf7e3)
    #1 0x7f541c1f36b7 in rxm_cmap_alloc_handle 
(/opt/devel/intel/compilers_and_libraries_2019.5.281/linux/mpi/intel64/libfabric/lib/prov/librxm-fi.so+0x86b7)

Direct leak of 27 byte(s) in 5 object(s) allocated from:
    #0 0x7f54b0dcaacb in __interceptor_malloc 
(/usr/lib/x86_64-linux-gnu/liblsan.so.0+0xeacb)
    #1 0x7f54add6fa29 in __strdup (/lib/x86_64-linux-gnu/libc.so.6+0x9da29)

SUMMARY: LeakSanitizer: 1307 byte(s) leaked in 13 allocation(s).

=================================================================
==42848==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 1280 byte(s) in 8 object(s) allocated from:
    #0 0x7ff87ac327e3  (/usr/lib/x86_64-linux-gnu/liblsan.so.0+0xf7e3)
    #1 0x7ff7e5ff36b7 in rxm_cmap_alloc_handle 
(/opt/devel/intel/compilers_and_libraries_2019.5.281/linux/mpi/intel64/libfabric/lib/prov/librxm-fi.so+0x86b7)

Direct leak of 27 byte(s) in 5 object(s) allocated from:
    #0 0x7ff87ac31acb in __interceptor_malloc 
(/usr/lib/x86_64-linux-gnu/liblsan.so.0+0xeacb)
    #1 0x7ff877bd6a29 in __strdup (/lib/x86_64-linux-gnu/libc.so.6+0x9da29)

SUMMARY: LeakSanitizer: 1307 byte(s) leaked in 13 allocation(s).
    #0 0x7faa81dbeacb in __interceptor_malloc 
(/usr/lib/x86_64-linux-gnu/liblsan.so.0+0xeacb)
    #1 0x7faa7ed63a29 in __strdup (/lib/x86_64-linux-gnu/libc.so.6+0x9da29)

SUMMARY: LeakSanitizer: 1307 byte(s) leaked in 13 allocation(s).

=================================================================
==42847==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 1280 byte(s) in 8 object(s) allocated from:
    #0 0x7f1b63ff97e3  (/usr/lib/x86_64-linux-gnu/liblsan.so.0+0xf7e3)
    #1 0x7f1acf3f36b7 in rxm_cmap_alloc_handle 
(/opt/devel/intel/compilers_and_libraries_2019.5.281/linux/mpi/intel64/libfabric/lib/prov/librxm-fi.so+0x86b7)

Direct leak of 27 byte(s) in 5 object(s) allocated from:
    #0 0x7f1b63ff8acb in __interceptor_malloc 
(/usr/lib/x86_64-linux-gnu/liblsan.so.0+0xeacb)
    #1 0x7f1b60f9da29 in __strdup (/lib/x86_64-linux-gnu/libc.so.6+0x9da29)

SUMMARY: LeakSanitizer: 1307 byte(s) leaked in 13 allocation(s).
=================================================================
==42841==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 1280 byte(s) in 8 object(s) allocated from:
    #0 0x7f50707517e3  (/usr/lib/x86_64-linux-gnu/liblsan.so.0+0xf7e3)
    #1 0x7f4fdbaf36b7 in rxm_cmap_alloc_handle 
(/opt/devel/intel/compilers_and_libraries_2019.5.281/linux/mpi/intel64/libfabric/lib/prov/librxm-fi.so+0x86b7)

Direct leak of 27 byte(s) in 5 object(s) allocated from:
    #0 0x7f5070750acb in __interceptor_malloc 
(/usr/lib/x86_64-linux-gnu/liblsan.so.0+0xeacb)
    #1 0x7f506d6f5a29 in __strdup (/lib/x86_64-linux-gnu/libc.so.6+0x9da29)

SUMMARY: LeakSanitizer: 1307 byte(s) leaked in 13 allocation(s).


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.cp2k.org/archives/cp2k-user/attachments/20200818/8696f489/attachment.htm>