Yes, I've checked it. <br /><br />I'm attaching the output files I have obtained from the tests for the container (master version) and for the spack module (v2024.1) that I mentioned in the previous message so you can have all the information.<br /><br />Thanks<br /><br /><div class="gmail_quote"><div dir="auto" class="gmail_attr">On Tuesday, July 2, 2024 at 4:03:30 PM UTC+2 Krack Matthias wrote:<br/></div><blockquote class="gmail_quote" style="margin: 0 0 0 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;"> <div lang="en-CH" link="blue" vlink="purple" style="word-wrap:break-word"> <div> <p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt">Thanks for your report.<u></u><u></u></span></p> <p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt">Did you also check with the print key which ELPA kernel is actually used?<u></u><u></u></span></p> <p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt"><u></u> <u></u></span></p> <div> <div></div></div></div></div><div lang="en-CH" link="blue" vlink="purple" style="word-wrap:break-word"><div><div><div> <div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm"> <p class="MsoNormal" style="margin-right:0cm;margin-bottom:12.0pt;margin-left:36.0pt"> <b><span style="color:black">From: </span></b><span style="color:black"><a href data-email-masked rel="nofollow">cp...@googlegroups.com</a> <<a href data-email-masked rel="nofollow">cp...@googlegroups.com</a>> on behalf of Ivan Bosch <<a href data-email-masked rel="nofollow">ivan....@irbbarcelona.org</a>><br> <b>Date: </b>Tuesday, 2 July 2024 at 14:36<br> <b>To: </b>cp2k <<a href data-email-masked rel="nofollow">cp...@googlegroups.com</a>><br> <b>Subject: </b>Re: [CP2K:20400] ELPA ERROR with cp2k container<u></u><u></u></span></p> </div> <p class="MsoNormal" style="margin-right:0cm;margin-bottom:12.0pt;margin-left:36.0pt"> Thanks for your response,<u></u><u></u></p> <p style="margin-left:36.0pt">I ran a test with the container using ELPA, which is when the warning in question appears, and another with ScalaPack with an input for only 5 steps. These are the results I obtained:<u></u><u></u></p> <div> <div> <p class="MsoNormal" style="margin-left:36.0pt">[root@irblogin01 outputs_tests]# grep diag devel_<b>elpa</b>_autokernel_test.out | tail -4 <br> qs_ot_get_derivative_diag 160 12.0 0.002 0.003 3.273 4.061 <br> qs_ot_p2m_diag 161 11.0 0.026 0.104 2.773 2.863 <br> cp_fm_diag_elpa 161 13.0 0.000 0.001 2.013 2.015 <br> cp_fm_diag_elpa_base 161 14.0 0.933 1.876 0.941 1.889<br> [root@irblogin01 outputs_tests]# grep diag devel_<b>elpa</b>_avx512_block6kernel_test.out | tail -4 <br> qs_ot_get_derivative_diag 160 12.0 0.002 0.003 3.251 3.954 <br> qs_ot_p2m_diag 161 11.0 0.026 0.103 2.772 2.862 <br> cp_fm_diag_elpa 161 13.0 0.000 0.001 2.011 2.012 <br> cp_fm_diag_elpa_base 161 14.0 0.933 1.874 0.940 1.888 <br> [root@irblogin01 outputs_tests]# grep diag devel_<b>scalapack</b>_test.out | tail -4 <br> qs_ot_get_derivative_diag 160 12.0 0.002 0.003 3.319 4.190 <br> qs_ot_p2m_diag 161 11.0 0.026 0.101 2.090 2.175<u></u><u></u></p> </div> </div> <p style="margin-left:36.0pt">In this case, there is no significant acceleration when using ELPA. <br> <br> However, with version 2024.1 installed with Spack, where I have verified that ELPA works well, I get these results:<u></u><u></u></p> <div> <div> <p class="MsoNormal" style="margin-left:36.0pt">[root@irblogin01 outputs_tests]# grep diag 2024.1_<b>scalapack</b>_test.out | tail -4 <br> qs_ot_p2m_diag 161 11.0 0.010 0.017 1.828 1.841 <br> qs_ot_get_derivative_diag 160 12.0 0.002 0.002 1.410 1.421 <br> [root@irblogin01 outputs_tests]# grep diag 2024.1_<b>elpa</b>_avx512_block6kernel_test.out | tail -4 <br> qs_ot_get_derivative_diag 160 12.0 0.002 0.002 1.397 1.408 <br> qs_ot_p2m_diag 161 11.0 0.010 0.017 1.190 1.205 <br> cp_fm_diag_elpa 161 13.0 0.000 0.001 0.816 0.817 <br> cp_fm_diag_elpa_base 161 14.0 0.321 0.646 0.327 0.662<u></u><u></u></p> </div> </div> <p style="margin-left:36.0pt">In these, I do observe an improvement in the timings.<u></u><u></u></p> <p class="MsoNormal" style="margin-left:36.0pt"><u></u> <u></u></p> <div> <div> <p class="MsoNormal" style="margin-left:36.0pt">On Tuesday, July 2, 2024 at 10:56:58<span style="font-family:"Arial",sans-serif"> </span>AM UTC+2 Krack Matthias wrote:<u></u><u></u></p> </div> <blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm"> <div> <div> <p class="MsoNormal" style="margin-left:36.0pt"> <span lang="EN-US" style="font-size:11.0pt">You can use the print key <a href="https://manual.cp2k.org/trunk/CP2K_INPUT/GLOBAL/PRINT_ELPA.html" target="_blank" rel="nofollow" data-saferedirecturl="https://www.google.com/url?hl=en&q=https://manual.cp2k.org/trunk/CP2K_INPUT/GLOBAL/PRINT_ELPA.html&source=gmail&ust=1720077367965000&usg=AOvVaw0d5uqgQPrz-pmI2kWmv8f1"> https://manual.cp2k.org/trunk/CP2K_INPUT/GLOBAL/PRINT_ELPA.html</a> to get more information about the ELPA usage.</span><u></u><u></u></p> <p class="MsoNormal" style="margin-left:36.0pt"> <span lang="EN-US" style="font-size:11.0pt"> </span><u></u><u></u></p> </div> </div> <div> <div> <div> <div> <div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm"> <p class="MsoNormal" style="margin-bottom:12.0pt;margin-left:72.0pt"> <b><span style="color:black">From: </span></b><span style="color:black"><span><a href data-email-masked rel="nofollow">cp...@googlegroups.com</a></span> <<span><a href data-email-masked rel="nofollow">cp...@googlegroups.com</a></span>> on behalf of Ivan Bosch <<span><a href data-email-masked rel="nofollow">ivan....@irbbarcelona.org</a></span>><br> <b>Date: </b>Tuesday, 2 July 2024 at 10:43<br> <b>To: </b>cp2k <<span><a href data-email-masked rel="nofollow">cp...@googlegroups.com</a></span>><br> <b>Subject: </b>Re: [CP2K:20398] ELPA ERROR with cp2k container</span><u></u><u></u></p> </div> <p class="MsoNormal" style="margin-bottom:12.0pt;margin-left:72.0pt"> Sure, but does this warning mean that the ELPA library is not being used to diagonalize?<br> <br> Thanks<u></u><u></u></p> <div> <div> <p class="MsoNormal" style="margin-left:72.0pt"> On Tuesday, July 2, 2024 at 10:38:39<span style="font-family:"Arial",sans-serif"> </span>AM UTC+2 Krack Matthias wrote:<u></u><u></u></p> </div> <blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt"> <div> <div> <p class="MsoNormal" style="margin-left:72.0pt"> <span lang="DE-CH" style="font-size:11.0pt">Hi</span><u></u><u></u></p> <p class="MsoNormal" style="margin-left:72.0pt"> <span lang="DE-CH" style="font-size:11.0pt"> </span><u></u><u></u></p> <p class="MsoNormal" style="margin-left:72.0pt"> <span lang="EN-US" style="font-size:11.0pt">That’s just a warning which is not specific to the use of containerized CP2K and/or singularity. I usually ignore it.</span><u></u><u></u></p> <p class="MsoNormal" style="margin-left:72.0pt"> <span lang="EN-US" style="font-size:11.0pt"> </span><u></u><u></u></p> </div> </div> <div> <div> <div> <div> <div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm"> <p class="MsoNormal" style="margin-bottom:12.0pt;margin-left:108.0pt"> <b><span style="color:black">From: </span></b><span style="color:black"><span><a href data-email-masked rel="nofollow">cp...@googlegroups.com</a></span> <<span><a href data-email-masked rel="nofollow">cp...@googlegroups.com</a></span>> on behalf of Ivan Bosch <<span><a href data-email-masked rel="nofollow">ivan....@irbbarcelona.org</a></span>><br> <b>Date: </b>Tuesday, 2 July 2024 at 09:43<br> <b>To: </b>cp2k <<span><a href data-email-masked rel="nofollow">cp...@googlegroups.com</a></span>><br> <b>Subject: </b>[CP2K:20394] ELPA ERROR with cp2k container</span><u></u><u></u></p> </div> <div> <div> <div> <div> <div> <div> <div> <div> <div> <div> <div> <div> <div> <div> <p style="margin-left:108.0pt">Hi, I've made a build of the cp2k-container provided by <a href="https://github.com/cp2k/cp2k-containers" target="_blank" rel="nofollow" data-saferedirecturl="https://www.google.com/url?hl=en&q=https://github.com/cp2k/cp2k-containers&source=gmail&ust=1720077367965000&usg=AOvVaw3Pt1HR9IywyaJmajvBiOgO">https://github.com/cp2k/cp2k-containers</a>, specifically using master_openmpi_skylake-avx512_psmp.Dockerfile, and it gives the following error regarding ELPA:<u></u><u></u></p> <div> <div> <p class="MsoNormal" style="margin-left:108.0pt"> bash<u></u><u></u></p> <div> <p class="MsoNormal" style="margin-left:108.0pt"> Copy code<u></u><u></u></p> </div> </div> <div> <p class="MsoNormal" style="margin-left:108.0pt"> *** WARNING in fm/cp_fm_elpa.F:522 :: Setting real_kernel for ELPA failed ***. <u></u> <u></u></p> </div> </div> <p style="margin-left:108.0pt">This has happened with different cp2k containers, not just this one, even when pulling directly from Docker Hub. I am using SingularityCE v4.0, and this is the output from within the container when running cp2k -v:<u></u><u></u></p> <div> <div> <p class="MsoNormal" style="margin-left:108.0pt"> bash<u></u><u></u></p> <div> <p class="MsoNormal" style="margin-left:108.0pt"> Copy code<u></u><u></u></p> </div> </div> <div> <p class="MsoNormal" style="margin-left:108.0pt"> CP2K version 2024.1 (Development Version) Source code revision git:5b092d7 cp2kflags: omp libint fftw3 libxc libgrpp pexsi elpa parallel mpi_f08 scalapack cosma quip deepmd xsmm plumed2 spglib libdftd4 sirius libvori libbqb libtorch libvdwxc hdf5 compiler: GCC version 11.4.0 compiler options: -cpp -I /opt/cp2k/tools/toolchain/install/openmpi-4.1.6/include -I /opt/cp2k/tools/toolchain/install/openblas-0.3.27/include -I /opt/cp2k/tools/toolchain/install/fftw-3.3.10/include -I /opt/cp2k/tools/toolchain/install/libint-v2.6.0-cp2k-lmax-5/include -I /opt/cp2k/tools/toolchain/install/libxc-6.2.2/include -I /opt/cp2k/tools/toolchain/install/libgrpp-main-20231215/include -I /opt/cp2k/tools/toolchain/install/libxsmm-1.17/include -I /opt/cp2k/tools/toolchain/install/COSMA-2.6.6/include -I /opt/cp2k/tools/toolchain/install/elpa-2024.03.001/cpu/include/elpa_openmp-2024.03.001/modules -I /opt/cp2k/tools/toolchain/install/elpa-2024.03.001/cpu/include/elpa_openmp-2024.03.001/elpa -I /opt/cp2k/tools/toolchain/install/scotch-6.0.0/include -I /opt/cp2k/tools/toolchain/install/superlu_dist-6.1.0/include -I /opt/cp2k/tools/toolchain/install/pexsi-1.2.0/include -I /opt/cp2k/tools/toolchain/install/quip-0.9.10/include -I /opt/cp2k/tools/toolchain/install/gsl-2.7/include -I /opt/cp2k/tools/toolchain/install/libdeepmd_c-2.2.7/include -I /opt/cp2k/tools/toolchain/install/hdf5-1.14.2/include -I /opt/cp2k/tools/toolchain/install/libvdwxc-0.4.0/include -I /opt/cp2k/tools/toolchain/install/spglib-2.3.1/include -I /opt/cp2k/tools/toolchain/install/SpFFT-1.1.0/include -I /opt/cp2k/tools/toolchain/install/SpLA-1.6.1/include/spla -I /opt/cp2k/tools/toolchain/install/sirius-7.5.2/include -I /opt/cp2k/tools/toolchain/install/dftd4-3.6.0/include -I /opt/cp2k/tools/toolchain/install/dftd4-3.6.0/include/dftd4/GNU-11.4.0 -I /opt/cp2k/tools/toolchain/install/dftd4-3.6.0/include/mctc-lib/GNU-11.4.0 -I /opt/cp2k/src/start/ -I /opt/cp2k/obj/local/psmp/exts/dbcsr -I /opt/cp2k/tools/toolchain/install/openmpi-4.1.6/include -I /opt/cp2k/tools/toolchain/install/openmpi-4.1.6/lib -imultiarch x86_64-linux-gnu -D_REENTRANT -D __LIBXSMM -D __parallel -D __MPI_F08 -D __FFTW3 -D __LIBINT -D __LIBXC -D __LIBGRPP -D __SCALAPACK -D __COSMA -D __ELPA -D __LIBPEXSI -D __QUIP -D __GSL -D __PLUMED2 -D __DEEPMD -D __HDF5 -D __LIBVDWXC -D __SPGLIB -D __LIBVORI -D __LIBTORCH -D __SPFFT -D __SPLA -D __SIRIUS -D __DFTD4 -D __COMPILE_ARCH="local" -D __COMPILE_DATE="Mon Jul 1 20:22:13 UTC 2024" -D __COMPILE_HOST="5bc8c4d74319" -D __COMPILE_REVISION="git:5b092d7" -D __DATA_DIR="/opt/cp2k/data" -D __SHORT_FILE__="start/cp2k.F" -mtune=skylake-avx512 -march=x86-64 -g -O3 -Werror=aliasing -Werror=ampersand -Werror=c-binding-type -Werror=intrinsic-shadow -Werror=intrinsics-std -Werror=line-truncation -Werror=tabs -Werror=target-lifetime -Werror=underflow -Werror=unused-but-set-variable -Werror=unused-variable -Werror=unused-dummy-argument -Werror=unused-parameter -Werror=unused-label -Werror=conversion -Werror=zerotrip -Wno-maybe-uninitialized -Wuninitialized -Wuse-without-only -std=f2008 -fno-omit-frame-pointer -fopenmp -funroll-loops -fbacktrace -ffree-form -fimplicit-none -fallow-argument-mismatch -fpre-include=/usr/include/finclude/math-vector-fortran.h <u></u><u></u></p> </div> </div> <p style="margin-left:108.0pt">Could you help me troubleshoot and resolve this issue?<u></u><u></u></p> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> <div> <div> <div> <div> <p class="MsoNormal" style="margin-left:108.0pt"> -- <br> You received this message because you are subscribed to the Google Groups "cp2k" group.<br> To unsubscribe from this group and stop receiving emails from it, send an email to <span><a href data-email-masked rel="nofollow">cp2k+uns...@googlegroups.com</a></span>.<br> To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/cp2k/e07c8a89-84d7-49a7-8911-faada06530d1n%40googlegroups.com?utm_medium=email&utm_source=footer" target="_blank" rel="nofollow" data-saferedirecturl="https://www.google.com/url?hl=en&q=https://groups.google.com/d/msgid/cp2k/e07c8a89-84d7-49a7-8911-faada06530d1n%2540googlegroups.com?utm_medium%3Demail%26utm_source%3Dfooter&source=gmail&ust=1720077367965000&usg=AOvVaw1l-wXTrx6fP1oxlTMDD_nB"> https://groups.google.com/d/msgid/cp2k/e07c8a89-84d7-49a7-8911-faada06530d1n%40googlegroups.com</a>.<u></u><u></u></p> </div> </div> </div> </div> </blockquote> </div> </div> </div> </div> </div> <div> <div> <div> <div> <p class="MsoNormal" style="margin-left:72.0pt"> -- <br> You received this message because you are subscribed to the Google Groups "cp2k" group.<br> To unsubscribe from this group and stop receiving emails from it, send an email to <span><a href data-email-masked rel="nofollow">cp2k+uns...@googlegroups.com</a></span>.<u></u><u></u></p> </div> </div> </div> </div> <div> <div> <div> <div> <p class="MsoNormal" style="margin-left:72.0pt"> To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/cp2k/73bc7691-1ad9-493e-9641-6b5fd631ffa1n%40googlegroups.com?utm_medium=email&utm_source=footer" target="_blank" rel="nofollow" data-saferedirecturl="https://www.google.com/url?hl=en&q=https://groups.google.com/d/msgid/cp2k/73bc7691-1ad9-493e-9641-6b5fd631ffa1n%2540googlegroups.com?utm_medium%3Demail%26utm_source%3Dfooter&source=gmail&ust=1720077367965000&usg=AOvVaw0ymFNLAYnNG4AzuFhtavSD"> https://groups.google.com/d/msgid/cp2k/73bc7691-1ad9-493e-9641-6b5fd631ffa1n%40googlegroups.com</a>.<u></u><u></u></p> </div> </div> </div> </div> </blockquote> </div> </div></div></div></div><div lang="en-CH" link="blue" vlink="purple" style="word-wrap:break-word"><div><div><div><p class="MsoNormal" style="margin-left:36.0pt">-- <br> You received this message because you are subscribed to the Google Groups "cp2k" group.<br> To unsubscribe from this group and stop receiving emails from it, send an email to <a href data-email-masked rel="nofollow">cp2k+uns...@googlegroups.com</a>.<br></p></div></div></div></div><div lang="en-CH" link="blue" vlink="purple" style="word-wrap:break-word"><div><div><div><p class="MsoNormal" style="margin-left:36.0pt"> To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/cp2k/0135c2f3-5810-4ca8-ad53-eb3aa1a40839n%40googlegroups.com?utm_medium=email&utm_source=footer" target="_blank" rel="nofollow" data-saferedirecturl="https://www.google.com/url?hl=en&q=https://groups.google.com/d/msgid/cp2k/0135c2f3-5810-4ca8-ad53-eb3aa1a40839n%2540googlegroups.com?utm_medium%3Demail%26utm_source%3Dfooter&source=gmail&ust=1720077367965000&usg=AOvVaw37vwrsGqhpKr_dMsBAJNN3"> https://groups.google.com/d/msgid/cp2k/0135c2f3-5810-4ca8-ad53-eb3aa1a40839n%40googlegroups.com</a>.<u></u><u></u></p> </div> </div> </div> </div> </blockquote></div> <p></p> -- <br /> You received this message because you are subscribed to the Google Groups "cp2k" group.<br /> To unsubscribe from this group and stop receiving emails from it, send an email to <a href="mailto:cp2k+unsubscribe@googlegroups.com">cp2k+unsubscribe@googlegroups.com</a>.<br /> To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/cp2k/4831d24c-bb86-4b44-9355-f9f647295702n%40googlegroups.com?utm_medium=email&utm_source=footer">https://groups.google.com/d/msgid/cp2k/4831d24c-bb86-4b44-9355-f9f647295702n%40googlegroups.com</a>.<br />