<div dir="ltr">Nico, Juerg,<div> </div><div>Thank you. That all makes sense.</div><div> </div><div>Best regards,</div><div>Jerry On Friday, May 4, 2018 at 5:47:54 AM UTC-4, Nico Holmberg wrote:<blockquote class="gmail_quote" style="margin: 0;margin-left: 0.8ex;border-left: 1px #ccc solid;padding-left: 1ex;"><div dir="ltr">Hi Jerry, Just to expand on Juerg's answer, it appears that you are using the OT method together with the FULL_SINGLE_INVERSE <a href="https://manual.cp2k.org/trunk/CP2K_INPUT/FORCE_EVAL/DFT/SCF/OT.html#list_PRECONDITIONER" target="_blank" rel="nofollow" onmousedown="this.href='https://www.google.com/url?q\x3dhttps%3A%2F%2Fmanual.cp2k.org%2Ftrunk%2FCP2K_INPUT%2FFORCE_EVAL%2FDFT%2FSCF%2FOT.html%23list_PRECONDITIONER\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFdZ91DJVf4MKF1GWu385w5txrGlQ';return true;" onclick="this.href='https://www.google.com/url?q\x3dhttps%3A%2F%2Fmanual.cp2k.org%2Ftrunk%2FCP2K_INPUT%2FFORCE_EVAL%2FDFT%2FSCF%2FOT.html%23list_PRECONDITIONER\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFdZ91DJVf4MKF1GWu385w5txrGlQ';return true;">preconditioner</a>. ELPA won't accelerate such simulations because no direct diagonalization is performed during the SCF run. If you swap the preconditioner to FULL_ALL, you should notice a ~50 % improvement in the first step of each inner SCF loop iteration when you use ELPA instead of ScaLAPACK. Futhermore, if you switch OT to a standard diagonalization based SCF solver, each SCF iteration should be accelerated by the same factor. I would however recommend that you always use OT if the system is suitable for the method. BR, Nico torstai 3. toukokuuta 2018 19.24.34 UTC+3 jgh kirjoitti:<blockquote class="gmail_quote" style="margin:0;margin-left:0.8ex;border-left:1px #ccc solid;padding-left:1ex">Hi how much time is used for Diagonalization in these runs? I would guess it is a rather small fraction. We developed the OT method especially to avoid diagonalization as much as possible. If this is in fact the case, no speedup can be expected by using ELPA. best regards Juerg -------------------------------------------------------------- Juerg Hutter Phone : ++41 44 635 4491 Institut für Chemie C FAX : ++41 44 635 6838 Universität Zürich E-mail: <a rel="nofollow">hut...@chem.uzh.ch</a> Winterthurerstrasse 190 CH-8057 Zürich, Switzerland --------------------------------------------------------------- -----<a rel="nofollow">cp...@googlegroups.com</a> wrote: ----- To: cp2k <<a rel="nofollow">cp...@googlegroups.com</a>> From: Jerry Tanoury Sent by: <a rel="nofollow">cp...@googlegroups.com</a> Date: 05/03/2018 06:15PM Subject: [CP2K:10266] Re: ELPA speed-up with Intel-compiled code Dear Alfio, My apologies for not attaching these at the beginning. The input and output files are now attached. Please note that the file names (coordinate files, etc.) have been changed in the input and output files for proprietary reasons. Also, the ELPA run was killed rather soon after it began because no speed-up was observed. Best regards, Jerry On Wednesday, May 2, 2018 at 2:57:12 PM UTC-4, Jerry Tanoury wrote: Dear forum, I am running CP2K version 5.1 compiled with Intel 2017 update 5 compilers and corresponding MKL libs. The arch file is attached. Everything runs as expected. I then build ELPA-2016.05.004 as shown below and built an ELPA-enable cp2k version according to the attached arch file: ../configure --prefix=/cluster/home/tanoury/CP2K/intelbuilt_packages/2017u5/elpa-2016.05.004 FC=mpiifort FCFLAGS=-O2 -xHost CC=mpiicc CFLAGS=-O2 -xHost --enable-option-checking=fatal --enable-static=yes --enable-avx2=no --enable-avx=no SCALAPACK_LDFLAGS=-L/cluster/home/tanoury/intel/2017u5/compilers_and_libraries_2017.5.239/linux/mkl/lib/intel64 -lmkl_scalapack_lp64 -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lmkl_blacs_intelmpi_lp64 -lpthread -Wl,-rpath,/cluster/home/tanoury/intel/2017u5/compilers_and_libraries_2017.5.239/linux/mkl/lib/intel64 SCALAPACK_FCFLAGS=-L/cluster/home/tanoury/intel/2017u5/compilers_and_libraries_2017.5.239/linux/mkl/lib/intel64 -lmkl_scalapack_lp64 -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lmkl_blacs_intelmpi_lp64 -lpthread -I/cluster/home/tanoury/intel/2017u5/compilers_and_libraries_2017.5.239/linux/mkl/include/intel64/lp64 When doing a speed test on 80 cores, I saw no speed-up from ELPA. Is this unexpected? Did I build ELPA correctly? Perhaps I need to run on 100's of cores. Thank you for the help, Jerry -- You received this message because you are subscribed to the Google Groups "cp2k" group. To unsubscribe from this group and stop receiving emails from it, send an email to <a rel="nofollow">cp2k+...@googlegroups.com</a>. To post to this group, send email to <a rel="nofollow">cp...@googlegroups.com</a>. Visit this group at <a href="https://groups.google.com/group/cp2k" rel="nofollow" target="_blank" onmousedown="this.href='https://groups.google.com/group/cp2k';return true;" onclick="this.href='https://groups.google.com/group/cp2k';return true;">https://groups.google.com/group/cp2k</a>. For more options, visit <a href="https://groups.google.com/d/optout" rel="nofollow" target="_blank" onmousedown="this.href='https://groups.google.com/d/optout';return true;" onclick="this.href='https://groups.google.com/d/optout';return true;">https://groups.google.com/d/optout</a>. [attachment "CP2K_Test.inp" removed by Jürg Hutter/at/UZH] [attachment "CP2K_Test1-ELPA.inp" removed by Jürg Hutter/at/UZH] [attachment "CP2K_Test1-ELPA.output" removed by Jürg Hutter/at/UZH] </blockquote></div></blockquote></div></div>