<div dir="ltr"><div>Hi Prof.Hutter,</div><div><br></div><div><font face="arial, sans-serif"><span style="color:rgb(0,0,0)">The memory used CP2K (as read by TRACE from /proc/self/statm file) keeps increasing even if I run a standard OT job (the input file is attached). The TRACE helped us to establish that the memory increase happens within qs_rho_update_rho_low() subroutine (the increase is shown in the attached plot). A closer look into the TRACE printout of subroutines called by qs_rho_update_rho_low() produces confusing data. TRACE tells that memory allocation can increase not only in complex subroutines, but even in simple subroutines such as pw_zero(). How is it possible that pw_zero() increases TRACE printout? Is it because allocated memory cannot be measured precisely in Unix (via /proc/self/statm)?</span><br aria-hidden="true" style="color:rgb(0,0,0)"><br aria-hidden="true" style="color:rgb(0,0,0)"><span style="color:rgb(0,0,0)">If so, it still does not explain the SYSTEMATIC memory increase in more complex subroutines called by qs_rho_update_rho_low(). I have noticed two arrays, grids_c and npts_local, within the grid_collocate_task_list subroutine that are not explicitly deallocated. Do you know if it is by design? I know that valgrind tells that the memory is properly deallocated (somehow) in the END of the job, but the minor increases DURING a long run eventually lead to "out of memory" problem.</span><br aria-hidden="true" style="color:rgb(0,0,0)"></font><div><br></div></div><div>Best regards,</div><div>Cindy.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Oct 4, 2023 at 8:56 PM Cindy Pham <<a href="mailto:cindypham196@gmail.com">cindypham196@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi Prof. Hutter,<div> </div><div>Thank you for your suggestion!</div><div><br></div><div>Best regards,</div><div>Cindy.</div><div><br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Oct 4, 2023 at 4:15 AM Jürg Hutter <<a href="mailto:hutter@chem.uzh.ch" target="_blank">hutter@chem.uzh.ch</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi<br>
<br>
without knowing the details of your program it is impossible to point to<br>
a easy solution. As there are no memory leaks in CP2K, you must miss some<br>
routines that clean up at the end of the SCF loop.<br>
I would suggest you compile the code with memory leak detection in order<br>
to find the problematic structure and then write a routine to deallocate them.<br>
See the sdbg arch files for the gfortran options needed for leak detection.<br>
<br>
regards<br>
JH<br>
<br>
________________________________________<br>
From: <a href="mailto:cp2k@googlegroups.com" target="_blank">cp2k@googlegroups.com</a> <<a href="mailto:cp2k@googlegroups.com" target="_blank">cp2k@googlegroups.com</a>> on behalf of Cindy Pham <<a href="mailto:cindypham196@gmail.com" target="_blank">cindypham196@gmail.com</a>><br>
Sent: Tuesday, October 3, 2023 10:16 PM<br>
To: <a href="mailto:cp2k@googlegroups.com" target="_blank">cp2k@googlegroups.com</a><br>
Subject: [CP2K:19284] Deallocate memory used by Hamiltonian-related subroutines<br>
<br>
Hi CP2K forum,<br>
<br>
I am running a lengthy SCF calculation (over 10k iterations) and noticed a gradual increase in the allocated memory (I used TRACE keyword to print current allocated memory). It appears that the step-by-step increase in memory allocation happens when the Kohn-Sham Hamiltonian is re-calculated, specifically within the qs_ks_did_change function (in qs_ks_types.F).<br>
<br>
The SCF routine is my own code that relies on the CP2K built-in Hamiltonian subroutines. It functions properly, but its only problem is the ever increasing memory consumption.<br>
<br>
Since I do not need to keep previous Hamiltonians (for any kind of DIIS extrapolation), is there any way to deallocate all memory used by the Hamiltonian-related subroutines (at least once in a while, say, after 1000 SCF iterations)?<br>
<br>
Alternatively, are there any input keywords that can ensure that the Hamiltonian structures are reset once in a while?<br>
<br>
Thank you in advance for your time and your suggestions.<br>
<br>
Best regards,<br>
Cindy Pham.<br>
<br>
--<br>
You received this message because you are subscribed to the Google Groups "cp2k" group.<br>
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="mailto:cp2k%2Bunsubscribe@googlegroups.com" target="_blank">cp2k+unsubscribe@googlegroups.com</a><mailto:<a href="mailto:cp2k%2Bunsubscribe@googlegroups.com" target="_blank">cp2k+unsubscribe@googlegroups.com</a>>.<br>
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/cp2k/CAN4Jpm3f3BUuJ%2B3EDEuqpp0TbBEkaHfOjQm%2Bn9%3D6vYquYQNQvQ%40mail.gmail.com" rel="noreferrer" target="_blank">https://groups.google.com/d/msgid/cp2k/CAN4Jpm3f3BUuJ%2B3EDEuqpp0TbBEkaHfOjQm%2Bn9%3D6vYquYQNQvQ%40mail.gmail.com</a><<a href="https://groups.google.com/d/msgid/cp2k/CAN4Jpm3f3BUuJ%2B3EDEuqpp0TbBEkaHfOjQm%2Bn9%3D6vYquYQNQvQ%40mail.gmail.com?utm_medium=email&utm_source=footer" rel="noreferrer" target="_blank">https://groups.google.com/d/msgid/cp2k/CAN4Jpm3f3BUuJ%2B3EDEuqpp0TbBEkaHfOjQm%2Bn9%3D6vYquYQNQvQ%40mail.gmail.com?utm_medium=email&utm_source=footer</a>>.<br>
<br>
-- <br>
You received this message because you are subscribed to the Google Groups "cp2k" group.<br>
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="mailto:cp2k%2Bunsubscribe@googlegroups.com" target="_blank">cp2k+unsubscribe@googlegroups.com</a>.<br>
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/cp2k/ZR0P278MB07591FF04A66D61DBAA83F3A9FCBA%40ZR0P278MB0759.CHEP278.PROD.OUTLOOK.COM" rel="noreferrer" target="_blank">https://groups.google.com/d/msgid/cp2k/ZR0P278MB07591FF04A66D61DBAA83F3A9FCBA%40ZR0P278MB0759.CHEP278.PROD.OUTLOOK.COM</a>.<br>
</blockquote></div>
</blockquote></div>
<p></p>
-- <br />
You received this message because you are subscribed to the Google Groups "cp2k" group.<br />
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="mailto:cp2k+unsubscribe@googlegroups.com">cp2k+unsubscribe@googlegroups.com</a>.<br />
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/cp2k/CAN4Jpm0fQ2eh-B5VbkYRoMcphU9eYDL5f6Jki7FkvY%3Duh9h3GQ%40mail.gmail.com?utm_medium=email&utm_source=footer">https://groups.google.com/d/msgid/cp2k/CAN4Jpm0fQ2eh-B5VbkYRoMcphU9eYDL5f6Jki7FkvY%3Duh9h3GQ%40mail.gmail.com</a>.<br />