<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> <meta name="Generator" content="Microsoft Word 15 (filtered medium)"> <style></style> </head> <body lang="en-CH" link="blue" vlink="purple" style="word-wrap:break-word"> <div class="WordSection1"> <p class="MsoNormal"><span lang="DE-CH" style="mso-fareast-language:EN-US">Dear Matthew<o:p></o:p></span></p> <p class="MsoNormal"><span lang="DE-CH" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p> <p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">The memory growth for v2022.1 looks not too bad and it might be fine to survive longer runs.<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">I remember issues with memory leaks caused by the MPI implementation. Especially OpenMPI showed such problems in the past and that is why I used only MPICH for years, because leak checking in CP2K was impossible with OpenMPI. Have a look at this <a href="https://github.com/cp2k/cp2k/issues/1830"> issue</a> from Jan this year for instance.<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">The presence of memory leaks usually does not imply that the results are wrong.<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p> <p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Best regards<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p> <p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Matthias<o:p></o:p></span></p> <p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p> <div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm"> <p class="MsoNormal" style="margin-left:36.0pt"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">"cp2k@googlegroups.com" <cp2k@googlegroups.com> on behalf of Matthew Emerson <mrson@uiowa.edu><br> <b>Reply to: </b>"cp2k@googlegroups.com" <cp2k@googlegroups.com><br> <b>Date: </b>Friday, 23 September 2022 at 16:55<br> <b>To: </b>"cp2k@googlegroups.com" <cp2k@googlegroups.com><br> <b>Subject: </b>Re: [CP2K:17749] Possible Memory Leak<o:p></o:p></span></p> </div> <div> <p class="MsoNormal" style="margin-left:36.0pt"><o:p> </o:p></p> </div> <p class="MsoNormal" style="mso-margin-top-alt:0cm;margin-right:0cm;margin-bottom:12.0pt;margin-left:36.0pt"> Dear Dr. Matthias, <br> <br> Sorry for the late response, I wanted to test these things carefully. Below is what I have found.<br> <br> All versions of CP2K that I have tested appear to still have a memory leak. The issue is less severe on version 2022.1, but as you can see, in the graph "CP2K-2022.1.png", the program keeps growing with time and our runs are long. For version 8.1, in graph "CP2K-8.1.png", the problem is much more severe (on the same hardware - 56 MPI ranks).<br> <br> My primary concern is with the correctness of the calculations. We can of course restart the program, but how do I know that a code that looks like it is leaking is not corrupting data. If you continue the run you started, I predict it will continue growing in memory usage. I thank you for your time.<br> <br> <br> Sincerely,<br> Matthew S. Emerson<o:p></o:p></p> <div> <div> <p class="MsoNormal" style="margin-left:36.0pt">On Wednesday, September 21, 2022 at 9:23:41 AM UTC-5 Matthias Krack wrote:<o:p></o:p></p> </div> <blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm"> <div> <div> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:36.0pt"> <span lang="EN-US">Hi Matthew</span><o:p></o:p></p> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:36.0pt"> <span lang="EN-US"> </span><o:p></o:p></p> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:36.0pt"> <span lang="EN-US">I used <a href="https://github.com/cp2k/cp2k/blob/master/arch/Linux-gnu-x86_64.psmp" target="_blank"> this arch file</a> to build the current cp2k release version 2022.1 on our local cluster, basically by running</span><o:p></o:p></p> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt;text-indent:-18.0pt;mso-list:l0 level1 lfo1"> <![if !supportLists]><span style="font-size:10.0pt;font-family:Symbol"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman""> </span></span></span><![endif]><span lang="FR-CH">source arch/Linux-gnu-x86_64.psmp</span><o:p></o:p></p> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:36.0pt"> <span lang="EN-US">in the main cp2k folder and then run make as proposed after the cp2k toolchain has been built successfully. This is also done for the continuous regression testing (see first two entries in the <a href="https://dashboard.cp2k.org/index.html" target="_blank">CP2K dashboard</a>, just click on the “OK” link to see the details).</span><o:p></o:p></p> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:36.0pt"> <span lang="EN-US"> </span><o:p></o:p></p> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:36.0pt"> <span lang="FR-CH">HTH</span><o:p></o:p></p> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:36.0pt"> <span lang="FR-CH"> </span><o:p></o:p></p> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:36.0pt"> <span lang="FR-CH">Matthias</span><o:p></o:p></p> </div> </div> <div> <div> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:36.0pt"> <span lang="FR-CH"> </span><o:p></o:p></p> <div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm"> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt"> <b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">"<span class="MsoHyperlink">cp...@googlegroups.com</span>" <<span class="MsoHyperlink">cp...@googlegroups.com</span>> on behalf of Matthew Emerson <<span class="MsoHyperlink">mr...@uiowa.edu</span>><br> <b>Reply to: </b>"<span class="MsoHyperlink">cp...@googlegroups.com</span>" <<span class="MsoHyperlink">cp...@googlegroups.com</span>><br> <b>Date: </b>Wednesday, 21 September 2022 at 15:38<br> <b>To: </b>"<span class="MsoHyperlink">cp...@googlegroups.com</span>" <<span class="MsoHyperlink">cp...@googlegroups.com</span>><br> <b>Subject: </b>Re: [CP2K:17727] Possible Memory Leak</span><o:p></o:p></p> </div> <div> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt"> <o:p></o:p></p> </div> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt"> Dear Dr. Matthias, <br> <br> Do you have an ARCH file for this build that you could point me to? I would like to build using the same settings to test at ORNL.<br> <br> Sincerely, <o:p></o:p></p> <div> <p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt;margin-left:72.0pt"> Matthew S. Emerson<o:p></o:p></p> </div> </div> </div> <div> <div> <div> <div> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt"> On Wednesday, September 21, 2022 at 4:00:21 AM UTC-5 Matthias Krack wrote:<o:p></o:p></p> </div> </div> </div> </div> <div> <div> <div> <blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt"> <div> <div> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt"> <span lang="EN-US">Hello Matthew</span><o:p></o:p></p> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt"> <span lang="EN-US"> </span><o:p></o:p></p> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt"> <span lang="EN-US">I have run your case on our local compute cluster with CP2K v2022.1 (gnu 11.2.0, OpenMPI 4.1.3) using 144 CPU cores. I observe only a small increase in memory usage after the usual initial growth during the first MD steps (see attached plot).</span><o:p></o:p></p> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt"> <span lang="EN-US"> </span><o:p></o:p></p> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt"> <span lang="EN-US">Best regards</span><o:p></o:p></p> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt"> <span lang="EN-US"> </span><o:p></o:p></p> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt"> <span lang="EN-US">Matthias</span><o:p></o:p></p> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt"> <span lang="EN-US"> </span><o:p></o:p></p> </div> </div> </blockquote> </div> </div> </div> <div> <div> <div> <blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt"> <div> <div> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt"> <b><span lang="EN-GB" style="border:solid windowtext 1.0pt;padding:0cm">Error! Filename not specified.</span></b><o:p></o:p></p> </div> </div> </blockquote> </div> </div> </div> <div> <div> <div> <blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt"> <div> <div> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt"> <span lang="EN-US"> </span><o:p></o:p></p> <div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm"> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:108.0pt"> <b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">"<span class="MsoHyperlink">cp...@googlegroups.com</span>" <<span class="MsoHyperlink">cp...@googlegroups.com</span>> on behalf of Matthew Emerson <<span class="MsoHyperlink">mr...@uiowa.edu</span>><br> <b>Reply to: </b>"<span class="MsoHyperlink">cp...@googlegroups.com</span>" <<span class="MsoHyperlink">cp...@googlegroups.com</span>><br> <b>Date: </b>Tuesday, 20 September 2022 at 23:21<br> <b>To: </b>"<span class="MsoHyperlink">cp...@googlegroups.com</span>" <<span class="MsoHyperlink">cp...@googlegroups.com</span>><br> <b>Subject: </b>[CP2K:17721] Possible Memory Leak</span><o:p></o:p></p> </div> <div> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:108.0pt"> <o:p></o:p></p> </div> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:108.0pt"> Dear CP2K Developers/Community, <o:p></o:p></p> <div> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:108.0pt"> <o:p></o:p></p> </div> <div> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:108.0pt"> I have attached an input file which I believe shows an example of a possible memory leak in CP2K. It is a typical NVT DFT simulation of molten MgCl2 with PBE-D3 dispersion corrections. <o:p></o:p></p> </div> <div> <p style="margin-left:108.0pt">I have tried well-tested CP2K builds on our local cluster (v6.1, v8.1, v2022.1), our university supercomputer (v6.1, v8.1), and even the system-wide installations of CP2K on Cori at Oak Ridge National Lab (v8.1 and v9.1) and memory usage grows linear with time until either the node locks up/dies from insufficient memory usage or the job dies from maximum walltime (ORNL). I've done enough testing that I can almost tell how many MD steps before the job will die for a given machine with X amount of RAM and Y amount of MPI ranks. <br> <br> I normally wouldn’t email about things like this but I’ve tried multiple combinations (w/unit-testing) of GCC, OpenMPI, OpenBLAS/MKL, etc. and nothing seems to work. I am hoping this is simply an input file issue or my own error.<o:p></o:p></p> <p style="margin-left:108.0pt"> <o:p></o:p></p> <p style="margin-left:108.0pt">Any help will be much appreciated.<br> <br> Matthew S. Emerson<br> Margulis Research Group<br> Department of Chemistry<br> The University of Iowa<o:p></o:p></p> </div> </div> </div> <div> <div> <p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt;margin-left:108.0pt"> -- <br> You received this message because you are subscribed to the Google Groups "cp2k" group.<br> To unsubscribe from this group and stop receiving emails from it, send an email to <span class="MsoHyperlink">cp2k+uns...@googlegroups.com</span>.<br> To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/cp2k/0c20a8f2-3658-4429-a2e9-7f4aa0edb321n%40googlegroups.com?utm_medium=email&utm_source=footer" target="_blank"> https://groups.google.com/d/msgid/cp2k/0c20a8f2-3658-4429-a2e9-7f4aa0edb321n%40googlegroups.com</a>.<o:p></o:p></p> </div> </div> </blockquote> </div> </div> </div> <div> <div> <p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:72.0pt"> -- <br> You received this message because you are subscribed to the Google Groups "cp2k" group.<br> To unsubscribe from this group and stop receiving emails from it, send an email to <span class="MsoHyperlink">cp2k+uns...@googlegroups.com</span>.<o:p></o:p></p> </div> </div> <div> <div> <p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt;margin-left:72.0pt"> To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/cp2k/c69d3c0d-e3d2-489c-8040-19ac7c4e396en%40googlegroups.com?utm_medium=email&utm_source=footer" target="_blank"> https://groups.google.com/d/msgid/cp2k/c69d3c0d-e3d2-489c-8040-19ac7c4e396en%40googlegroups.com</a>.<o:p></o:p></p> </div> </div> </blockquote> </div> <p class="MsoNormal" style="margin-left:36.0pt">-- <br> You received this message because you are subscribed to the Google Groups "cp2k" group.<br> To unsubscribe from this group and stop receiving emails from it, send an email to <a href="mailto:cp2k+unsubscribe@googlegroups.com">cp2k+unsubscribe@googlegroups.com</a>.<br> To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/cp2k/3bb44147-b5e4-4edf-aa0c-14b6eb9195den%40googlegroups.com?utm_medium=email&utm_source=footer"> https://groups.google.com/d/msgid/cp2k/3bb44147-b5e4-4edf-aa0c-14b6eb9195den%40googlegroups.com</a>.<br> <br> <o:p></o:p></p> </div> </body> </html> <p></p> -- <br /> You received this message because you are subscribed to the Google Groups "cp2k" group.<br /> To unsubscribe from this group and stop receiving emails from it, send an email to <a href="mailto:cp2k+unsubscribe@googlegroups.com">cp2k+unsubscribe@googlegroups.com</a>.<br /> To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/cp2k/D03B0A9B-83F9-4F31-8855-16CD0CFFDD32%40psi.ch?utm_medium=email&utm_source=footer">https://groups.google.com/d/msgid/cp2k/D03B0A9B-83F9-4F31-8855-16CD0CFFDD32%40psi.ch</a>.<br />