<div>Dear Léon,</div><div>Regarding 1:</div><div>Did you optimize the accuracy-relevant parameters (CUTOFF, EPS_DEFAULT) to meet your demands regarding energies and gradients? If not, there is an how to concerning this topic (https://www.cp2k.org/howto:converging_cutoff). Choose the cutoff as high as necessary and as low as possible.</div><div>TLDR: From your timing report, I recommend you to employ the MEASURE mode, see below for a more lengthy explanation.<br /></div><div>If you have optimized these parameters, check the timing report. The total CP2K runtime is given by the line with "CP2K" (ignoring warming up times of the cluster and of CP2K). Each line represents a routine within CP2K, from the name you may guess its purpose/task. Relevant to you are especially the last four columns. The last two columns indicate the average and maximum time spent in a given routine and its called routines. They give you an indication which functionality spends the most time. The fourth and third last columns indicate the the average and maximum time spent in the routine itself, i.e. without the routines called by the given routine. These two columns indicate in which compute kernel CP2K spends its most amount of time. If you check the routines with the largest self-time, you find the highest self-times of FFT-related routines, i.e. those containing 'fft' in their names, in case of fft3d_ps (10122.9 s, 3.9 % of CP2K) and
<span></span>
<span cellspacing="0" border="0">
<span width="200"></span> <span><span><span height="21" align="left">fft_wrap_pw1pw2_200 (5123.8 s, 2.0 % of CP2K). Even if you were able to reduce the timing of these routines to zero, you would gain ca. 6 % of computation time, in practice it will be much less. So, I think that a further reduction would not be able to reduce the computational time significantly.</span></span></span></span></div><div><span cellspacing="0" border="0"><span><span><span height="21" align="left">For completeness, the difference to the total runtime of the fft_wrap_pw1pw2 routine of ~50000 s is related to the respective communication routines which do not have 'fft' in their name. These are of course not affected by the FFTW planning times.</span></span></span></span></div><div><span cellspacing="0" border="0"><span><span><span height="21" align="left"><br /></span></span></span></span></div><div><span cellspacing="0" border="0"><span><span><span height="21" align="left">Regarding 2:</span></span></span></span></div><div><span cellspacing="0" border="0"><span><span><span height="21" align="left">Checking the source code of the routine fft_get_scratch in src/pw/fft_tools.F indicates that the issue might be related to an issue on a lower level routine within CP2K (probably src/fft/fftw3_lib.F) or in FFTW3 directly. I did some local tests with one of the single-node tests and did not observe any issues in these modes. So, the issue will need more thorough investigations.<br /></span></span></span></span></div><div><span cellspacing="0" border="0"><span><span><span height="21" align="left"><br /></span></span></span></span></div><div><span cellspacing="0" border="0"><span><span><span height="21" align="left">HTH</span></span></span></span></div><div><span cellspacing="0" border="0"><span><span><span height="21" align="left">Frederick<br /></span></span></span></span></div><div class="gmail_quote"><div dir="auto" class="gmail_attr">Léon Luntadila Lufungula schrieb am Freitag, 16. Februar 2024 um 10:01:04 UTC+1:<br/></div><blockquote class="gmail_quote" style="margin: 0 0 0 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">Dear Frederick,<div><br></div><div>Thanks for guiding me in solving this issue!</div><div><br></div><div>1) I have not yet checked if this is indeed the case but I'm quite new to AIMD calculations and the reference manual stated that this was recommended for long AIMD trajectories, so I assumed this would be a good option to enable. If you could show me how to check this, I would greatly appreciate the help! For the same calculation with the default FFTW_PLAN_TYPE ESTIMATE I get the following timings:</div><div><br></div><div> -------------------------------------------------------------------------------<br> - -<br> - T I M I N G -<br> - -<br> -------------------------------------------------------------------------------<br> SUBROUTINE CALLS ASD SELF TIME TOTAL TIME<br> MAXIMUM AVERAGE MAXIMUM AVERAGE MAXIMUM<br> CP2K 1 1.0 0.0 0.0 259198.3 259201.0<br> qs_mol_dyn_low 1 2.0 0.5 0.6 259198.0 259200.7<br> velocity_verlet 21600 3.0 4.7 5.3 259117.8 259120.7<br> qs_forces 21601 4.0 6.7 8.3 259054.1 259057.9<br> qs_energies 21601 5.0 1.3 2.0 216944.6 216962.9<br> scf_env_do_scf 21601 6.0 0.7 1.0 197423.7 197426.9<br> scf_env_do_scf_inner_loop 172819 7.0 2.9 3.7 147536.9 147542.8<br> rebuild_ks_matrix 194420 8.7 0.7 0.7 113213.6 113417.8<br> qs_ks_build_kohn_sham_matrix 194420 9.7 21.9 25.3 113212.9 113417.2<br> qs_ks_update_qs_env 194420 8.0 1.5 1.8 85131.3 85315.6<br><b> pw_transfer 4104421 12.9 259.1 293.8 53272.8 54657.8</b><br> sum_up_and_integrate 194420 10.7 158.9 210.4 54433.6 54518.3<br> integrate_v_rspace 194420 11.7 6.8 8.9 54274.2 54375.6<br> <b>fft_wrap_pw1pw2 3715581 14.0 41.8 46.1 52788.9 54160.2<br> fft_wrap_pw1pw2_200 2160221 15.4 5123.8 5554.3 50067.4 51727.8</b><br> init_scf_loop 21601 7.0 0.6 0.8 49795.6 49797.8<br> qs_vxc_create 194420 10.7 4.5 5.3 49235.6 49729.8<br> xc_vxc_pw_create 194420 11.7 990.7 1608.7 49231.1 49725.4<br> qs_rho_update_rho_low 194420 8.1 0.8 0.9 44519.7 44594.7<br> calculate_rho_elec 194420 9.1 109.2 184.2 44518.9 44593.9<br> fft3d_ps 3715581 16.0 10122.9 14717.9 41927.7 43177.5<br> dbcsr_multiply_generic 2981151 12.6 130.7 138.3 42145.8 42708.1<br> prepare_preconditioner 21601 8.0 0.1 0.2 38552.7 38578.0<br> grid_integrate_task_list 194420 12.7 34380.1 37401.8 34380.1 37401.8<br> xc_pw_derive 1166520 13.7 9.8 11.0 34752.0 36587.5<br> make_preconditioner 21601 9.0 0.4 0.6 36451.6 36464.5<br> qs_scf_new_mos 172819 8.0 0.9 1.4 34340.9 34469.5<br> qs_scf_loop_do_ot 172819 9.0 0.8 1.0 34340.0 34468.7<br> make_full_all 21601 10.0 4.6 5.3 34250.9 34260.8<br> ot_scf_mini 172819 10.0 4.0 4.7 31468.4 31598.1<br> multiply_cannon 2981151 13.6 192.3 222.5 29228.9 30932.4<br> mp_alltoall_z22v 3715581 18.0 27949.9 30346.5 27949.9 30346.5<br> multiply_cannon_loop 2981151 14.6 215.7 231.9 27541.2 28832.2<br> xc_pw_divergence 194420 12.7 4.0 4.7 26628.5 28220.2<br> qs_ks_update_qs_env_forces 21601 5.0 0.1 0.2 28174.2 28195.0<br> xc_rho_set_and_dset_create 194420 12.7 239.1 269.1 20595.1 28022.0<br> ot_mini 172819 11.0 1.0 1.4 23618.8 23759.5<br> grid_collocate_task_list 194420 10.1 22590.9 23442.1 22590.9 23442.1<br> rs_pw_transfer 1987402 12.3 41.9 48.7 21225.8 22117.5<br> yz_to_x 993701 17.7 742.7 1149.4 19795.7 21777.5<br> density_rs2pw 194420 10.1 8.1 8.9 20317.1 21233.4<br> mp_waitall_1 ******* -4.3 17314.7 19114.1 17314.7 19114.1<br> multiply_cannon_multrec ******* 15.6 17135.6 18128.8 17149.2 18142.9<br> cp_fm_diag_elpa 64803 11.3 0.4 0.5 18118.8 18125.4<br> cp_fm_diag_elpa_base 64803 12.3 17919.4 18033.2 18104.5 18104.8<br> mp_waitany ******* 14.1 13555.6 14684.3 13555.6 14684.3<br> xc_functional_eval 194420 13.7 2.0 2.6 8289.8 14473.2<br> pbe_lda_eval 194420 14.7 8287.8 14472.3 8287.8 14472.3<br><br></div><div><br></div><div>2) Unfortunately, no pdbg-version was compiled on my HPC center, but I did run a calculation with TRACE and TRACE_MASTER enabled. See attached files for the output.</div><div><br></div><div>Kind regards,</div><div>Léon</div><div class="gmail_quote"><div dir="auto" class="gmail_attr">On Friday 16 February 2024 at 09:36:56 UTC+1 Frederick Stein wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<p>Dear Leon,</p>
<p>I do not know the root of the error and I cannot suggest a solution.
The options themself are tested within our regtest suite and we do not
find any issues there. So, it seems to be a more complicated problem
either on FFTW site or on CP2K site.</p>
<p>I have two questions:</p>
<p>1. Did you check whether the FFT kernel actually needs an
improvement? Check the runtime of the routines pw_transfer and those
starting with fft_wrap_pw1pw2 (or similar).</p>
<p>2. Do you have a pdbg-version of CP2K available? If yes, can you run
one of the failing tests with that one? It might also help to turn on
the keywords TRACE and TRACE_MASTER in the GLOBAL section of your input
files to identify the actual routine on CP2K site where the error
occurs.</p>
<p>Regards,</p>
<p>Frederick</p>
<br><div class="gmail_quote"><div dir="auto" class="gmail_attr">Léon Luntadila Lufungula schrieb am Montag, 12. Februar 2024 um 10:42:16 UTC+1:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="direction:ltr">
<div style="direction:ltr;margin-top:0in;margin-left:0in;width:7.6041in">
<div style="direction:ltr;margin-top:0in;margin-left:0in;width:7.6041in">
<p style="margin:0in;font-family:Calibri;font-size:11pt" lang="en-US">Dear all,</p>
<p style="margin:0in;font-family:Roboto;font-size:10.5pt">I've been running
some AIMD calculations and am trying to speed up the calculations a bit by
playing with the FFTW_PLAN_TYPE option. Unfortunately, only MEASURE and the
default ESTIMATE are working. If I try to set it to PATIENT (as recommended for
long AIMD runs) or EXHAUSTIVE, the calculation crashes almost immediately with
the following error messages (see also attached files):</p>
<p style="margin:0in;font-family:Roboto;font-size:10.5pt">[PATIENT]</p>
<p style="margin:0in;font-family:Roboto;font-size:10.5pt">...</p>
<p style="margin:0in;font-family:Roboto;font-size:10.5pt">corrupted
double-linked list</p>
<p style="margin:0in;font-family:Roboto;font-size:10.5pt">corrupted
double-linked list (not small)</p>
<p style="margin:0in;font-family:Roboto;font-size:10.5pt">cp2k.popt:
malloc.c:4106: _int_malloc: Assertion `(unsigned long) (size) >= (unsigned
long) (nb)' failed.</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt" lang="en-US"> </p>
<p style="margin:0in;font-family:Roboto;font-size:10.5pt">Program received
signal SIGABRT: Process abort signal.</p>
<p style="margin:0in;font-family:Roboto;font-size:10.5pt">...</p>
<p style="margin:0in;font-family:Roboto;font-size:10.5pt">[EXHAUSTIVE]</p>
<p style="margin:0in;font-family:Roboto;font-size:10.5pt">...</p>
<p style="margin:0in;font-family:Roboto;font-size:10.5pt">malloc_consolidate():
unaligned fastbin chunk detected</p>
<p style="margin:0in;font-family:Roboto;font-size:10.5pt">malloc_consolidate():
unaligned fastbin chunk detected</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt" lang="en-US"> </p>
<p style="margin:0in;font-family:Roboto;font-size:10.5pt">Program received
signal SIGABRT: Process abort signal.</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt" lang="en-US"> </p>
<p style="margin:0in;font-family:Roboto;font-size:10.5pt">Backtrace for this
error:</p>
<p style="margin:0in;font-family:Roboto;font-size:10.5pt">corrupted
double-linked list</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt" lang="en-US"> </p>
<p style="margin:0in;font-family:Roboto;font-size:10.5pt">Program received
signal SIGABRT: Process abort signal.</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt" lang="en-US"> </p>
<p style="margin:0in;font-family:Roboto;font-size:10.5pt">Backtrace for this
error:</p>
<p style="margin:0in;font-family:Roboto;font-size:10.5pt">malloc_consolidate():
unaligned fastbin chunk detected</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt" lang="en-US"> </p>
<p style="margin:0in;font-family:Roboto;font-size:10.5pt">Program received
signal SIGABRT: Process abort signal.</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt" lang="en-US"> </p>
<p style="margin:0in;font-family:Roboto;font-size:10.5pt">Backtrace for this
error:</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt" lang="en-US"> </p>
<p style="margin:0in;font-family:Roboto;font-size:10.5pt">Program received
signal SIGABRT: Process abort signal.</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt" lang="en-US"> </p>
<p style="margin:0in;font-family:Roboto;font-size:10.5pt">Backtrace for this
error:</p>
<p style="margin:0in;font-family:Roboto;font-size:10.5pt">corrupted
double-linked list</p>
<p style="margin:0in;font-family:Roboto;font-size:10.5pt">cp2k.popt:
malloc.c:4106: _int_malloc: Assertion `(unsigned long) (size) >= (unsigned
long) (nb)' failed.</p>
<p style="margin:0in;font-family:Roboto;font-size:10.5pt">...</p>
<p style="margin:0in;font-family:Roboto;font-size:10.5pt">I'm running
CP2K/2023.2-foss-2022a as compiled with Easybuild by our HPC centre, but the
same problems appear when I try the CP2K/2022.1-foss-2022a version. However,
when I run it with the CP2K/7.1-intel-2020a version which is also available,
both EXHAUSTIVE and PATIENT seem to be working properly... Is this something that can be solved in some way or will this require a different compilation of CP2K, possibly with the intel toolchain instead of the foss toolchain? </p><p style="margin:0in;font-family:Roboto;font-size:10.5pt"><br></p><p style="margin:0in;font-family:Roboto;font-size:10.5pt">Kind regards,</p><p style="margin:0in;font-family:Roboto;font-size:10.5pt">Léon</p>
<p style="margin:0in;font-family:Calibri;font-size:11pt" lang="en-US"><br></p>
</div>
</div>
</div></blockquote></div></blockquote></div></blockquote></div>
<p></p>
-- <br />
You received this message because you are subscribed to the Google Groups "cp2k" group.<br />
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="mailto:cp2k+unsubscribe@googlegroups.com">cp2k+unsubscribe@googlegroups.com</a>.<br />
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/cp2k/70911d18-487c-47c3-8bbe-00521ee5c6cen%40googlegroups.com?utm_medium=email&utm_source=footer">https://groups.google.com/d/msgid/cp2k/70911d18-487c-47c3-8bbe-00521ee5c6cen%40googlegroups.com</a>.<br />