<div>Hello Tiziano,</div><div><br></div><div>Thank you again for your help! After compiling psmp version of cp2k (openMPI based ), both of them, SIRIUS and ELPA are included. After the compilation I have benchmarked both of them and I got some results that I do not understand (ssmp and psmp were compiled with CUDA enabled). For testing, I was using my desktop pc, Ubuntu 20.04, Intel Core i7-4790, (4 cores, 2 threads), Gtx 1660 super and 16 gb of RAM. As benchmark I was using H2O-64.inp. Please find below what I got:</div><div><br></div><div>1) time cp2k.ssmp -o H2O-64.out H2O-64.inp<br><br>real 8m7,011s<br>user 27m34,937s<br>sys 1m2,519s</div><div><br></div><div>2) time mpirun -np 4 cp2k.psmp -o H2O-64.out H2O-64.inp<br><br>no load on GPU, only video memory used<br>real 49m20,818s<br>user 361m47,512s<br>sys 2m30,998s</div><div><br></div><div>Why do I observe such huge difference? I run it twice and I got the same result. After that I started to play little bit.</div><div><br></div><div>3) time mpirun -np 4 -x OMP_NUM_THREADS=2 cp2k.psmp -o H2O-64.out H2O-64.inp<br><br>real 7m59,329s<br>user 41m36,361s<br>sys 0m47,529s</div><div><br></div><div>4) time mpirun -np 4 --map-by slot:PE=1 -x OMP_PLACES=threads -x OMP_PROC_BIND=SPREAD -x OMP_NUM_THREADS=2 cp2k.psmp -o H2O-64.out H2O-64.inp <br><br>real 7m50,873s<br>user 41m25,381s<br>sys 0m46,038s</div><div><br></div><div>Tiziano, do you know why such huge discrepancies are observed? Thank you in advance for you comments!</div><div><br></div><div>Best regards,</div><div>Krzysztof</div><div class="gmail_quote"><div dir="auto" class="gmail_attr">On Wednesday, December 9, 2020 at 10:25:01 PM UTC+1 tiz...@chem.uzh.ch wrote:<br/></div><blockquote class="gmail_quote" style="margin: 0 0 0 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">Hi Krzysztof,
<br>
<br>I assume you used the `ssmp` (or the deprecated `sopt`) VERSION?
<br>SIRIUS and ELPA are only available when built with MPI support.
<br>So, you have to run:
<br>
<br> make ARCH=local VERSION=psmp
<br>
<br>Best,
<br>Tiziano
<br>
<br>On 12/9/20 7:19 PM, Krzysztof Kolman wrote:
<br>> Hi Tiziano,
<br>>
<br>> Thank you for your help! This time it worked :) I have one more
<br>> question. After the compilation I performed the regression testing. In
<br>> the beginning I got following message:
<br>>
<br>> CP2K supports: cp2kflags: omp libint fftw3 libxc acc pw_cuda xsmm
<br>> dbcsr_acc spglib libvori libbqb grid_cuda
<br>> Skipping QS/regtest-cdft-hirshfeld-2 : missing required feature : parallel
<br>> Skipping QS/regtest-cdft-hirshfeld-2 : missing required feature : mpiranks>1
<br>> Skipping SIRIUS/regtest-1 : missing required feature : sirius
<br>> Skipping QS/regtest-elpa-2 : missing required feature : elpa
<br>> Skipping QMMM/QS/regtest-cdft : missing required feature : parallel
<br>> Skipping QMMM/QS/regtest-cdft : missing required feature :
<br>> mpiranks==2||mpiranks==4||mpiranks==6||mpiranks==16||mpiranks==24
<br>> Skipping QS/regtest-rma : missing required feature : mpi3
<br>> Skipping QS/regtest-rma : missing required feature : mpiranks==4
<br>> Skipping QS/regtest-cdft-3 : missing required feature : parallel
<br>> Skipping QS/regtest-cdft-3 : missing required feature :
<br>> mpiranks==2||mpiranks==4||mpiranks==6||mpiranks==16||mpiranks==24
<br>> Skipping QS/regtest-pexsi : missing required feature : pexsi
<br>> Skipping QS/regtest-rma-3D : missing required feature : mpi3
<br>> Skipping QS/regtest-rma-3D : missing required feature : mpiranks==8
<br>> Skipping TMC/regtest_ana_on_the_fly : missing required feature : parallel
<br>> Skipping TMC/regtest_ana_on_the_fly : missing required feature : mpiranks>2
<br>> Skipping TMC/regtest : missing required feature : parallel
<br>> Skipping TMC/regtest : missing required feature : mpiranks>1
<br>> Skipping QS/regtest-elpa-qr : missing required feature : elpa
<br>> Skipping TMC/regtest_ana_post_proc : missing required feature : parallel
<br>> Skipping TMC/regtest_ana_post_proc : missing required feature : mpiranks>1
<br>> Skipping QS/regtest-elpa : missing required feature : elpa
<br>> Skipping Fist/regtest-quip : missing required feature : quip
<br>> Skipping Fist/regtest-plumed2 : missing required feature : plumed2
<br>>
<br>> Here it claims that ELPA and SIRIUS are missing. Do you know maybe why?
<br>>
<br>> Best regards,
<br>> Krzysztof
<br>>
<br>> On Wednesday, December 9, 2020 at 4:14:07 PM UTC+1
<br>> <a href data-email-masked rel="nofollow">tiz...@chem.uzh.ch</a> wrote:
<br>>
<br>> Hi Krzysztof,
<br>>
<br>> please use the latest git-master of CP2K if you want to try out a
<br>> CUDA-enabled ELPA. In that version you should not only be able to build
<br>> ELPA with support for a CUDA-kernel, but CP2K should actually be
<br>> able to
<br>> use it.
<br>>
<br>> Best regards,
<br>> Tiziano
<br>>
<br>>
<br>> On 12/9/20 12:15 PM, Krzysztof Kolman wrote:
<br>> > Hello,
<br>> >
<br>> > I have encounter a problem while trying to compile cp2k at Ubuntu
<br>> 20.04
<br>> > with CUDA support. When I use following toolchain command:
<br>> > ./install_cp2k_toolchain.sh --with-libxsmm=install
<br>> > --with-openblas=install --with-fftw=install --with-reflapack=no
<br>> > --enable-cuda --enable-omp --gpu-ver=V100 --with-cmake=system
<br>> > --with-openmpi=install
<br>> >
<br>> > I get such error:
<br>> > ==================== Installing ELPA ====================
<br>> > elpa-2019.05.001.tar.gz is found
<br>> > Installing from scratch into
<br>> > /cp2k-7.1/tools/toolchain/install/elpa-2019.05.001
<br>> > ERROR: (./scripts/install_elpa.sh, line 103) Non-zero exit code
<br>> detected.
<br>> >
<br>> > From the make.log located at /build/elpa-2019.05.00/obj_no_thread
<br>> I can
<br>> > see that nvcc crashes:
<br>> > nvcc fatal : Unknown option 'Wl,-rpath'
<br>> >
<br>> > Please find attached the make. log file as well as config.log and
<br>> > configure.log located at /build/elpa-2019.05.00/obj_no_threa.
<br>> >
<br>> > I think it is also worth to mention that I manage to compile cp2k
<br>> with
<br>> > flag --with-elpa=no.
<br>> >
<br>> > Thank you in advance for your help!
<br>> >
<br>> > Best regards,
<br>> > Krzysztof
<br>> >
<br>> > --
<br>> > You received this message because you are subscribed to the Google
<br>> > Groups "cp2k" group.
<br>> > To unsubscribe from this group and stop receiving emails from it,
<br>> send
<br>> > an email to <a href data-email-masked rel="nofollow">cp...@googlegroups.com</a>
<br>> > <mailto:<a href data-email-masked rel="nofollow">cp...@googlegroups.com</a>>.
<br>> > To view this discussion on the web visit
<br>> >
<br>> <a href="https://groups.google.com/d/msgid/cp2k/60525719-7909-44a5-b729-4314898ca143n%40googlegroups.com" target="_blank" rel="nofollow" data-saferedirecturl="https://www.google.com/url?hl=en&q=https://groups.google.com/d/msgid/cp2k/60525719-7909-44a5-b729-4314898ca143n%2540googlegroups.com&source=gmail&ust=1607803149096000&usg=AFQjCNFs34J4NTaYKpZmH6SEa0XabwtBag">https://groups.google.com/d/msgid/cp2k/60525719-7909-44a5-b729-4314898ca143n%40googlegroups.com</a>
<br>> <<a href="https://groups.google.com/d/msgid/cp2k/60525719-7909-44a5-b729-4314898ca143n%40googlegroups.com" target="_blank" rel="nofollow" data-saferedirecturl="https://www.google.com/url?hl=en&q=https://groups.google.com/d/msgid/cp2k/60525719-7909-44a5-b729-4314898ca143n%2540googlegroups.com&source=gmail&ust=1607803149096000&usg=AFQjCNFs34J4NTaYKpZmH6SEa0XabwtBag">https://groups.google.com/d/msgid/cp2k/60525719-7909-44a5-b729-4314898ca143n%40googlegroups.com</a>>
<br>>
<br>> >
<br>> <<a href="https://groups.google.com/d/msgid/cp2k/60525719-7909-44a5-b729-4314898ca143n%40googlegroups.com?utm_medium=email&utm_source=footer" target="_blank" rel="nofollow" data-saferedirecturl="https://www.google.com/url?hl=en&q=https://groups.google.com/d/msgid/cp2k/60525719-7909-44a5-b729-4314898ca143n%2540googlegroups.com?utm_medium%3Demail%26utm_source%3Dfooter&source=gmail&ust=1607803149096000&usg=AFQjCNHbiTrPfnDkaoR3rMszcsfsIGuDiQ">https://groups.google.com/d/msgid/cp2k/60525719-7909-44a5-b729-4314898ca143n%40googlegroups.com?utm_medium=email&utm_source=footer</a>
<br>> <<a href="https://groups.google.com/d/msgid/cp2k/60525719-7909-44a5-b729-4314898ca143n%40googlegroups.com?utm_medium=email&utm_source=footer" target="_blank" rel="nofollow" data-saferedirecturl="https://www.google.com/url?hl=en&q=https://groups.google.com/d/msgid/cp2k/60525719-7909-44a5-b729-4314898ca143n%2540googlegroups.com?utm_medium%3Demail%26utm_source%3Dfooter&source=gmail&ust=1607803149096000&usg=AFQjCNHbiTrPfnDkaoR3rMszcsfsIGuDiQ">https://groups.google.com/d/msgid/cp2k/60525719-7909-44a5-b729-4314898ca143n%40googlegroups.com?utm_medium=email&utm_source=footer</a>>>.
<br>>
<br>>
<br>> --
<br>> Tiziano Müller
<br>> University of Zurich
<br>> Department of Chemistry
<br>> Winterthurerstrasse 190
<br>> CH-8057 Zürich
<br>>
<br>> Tel: +41 44 63 54234
<br>> <a href="http://www.chem.uzh.ch" target="_blank" rel="nofollow" data-saferedirecturl="https://www.google.com/url?hl=en&q=http://www.chem.uzh.ch&source=gmail&ust=1607803149096000&usg=AFQjCNF9QZGfHb8d-UtLa9Nv3zXvepf2Xg">www.chem.uzh.ch</a> <<a href="http://www.chem.uzh.ch" target="_blank" rel="nofollow" data-saferedirecturl="https://www.google.com/url?hl=en&q=http://www.chem.uzh.ch&source=gmail&ust=1607803149096000&usg=AFQjCNF9QZGfHb8d-UtLa9Nv3zXvepf2Xg">http://www.chem.uzh.ch</a>>
<br>> <a href data-email-masked rel="nofollow">tiz...@chem.uzh.ch</a>
<br>>
<br>> --
<br>> You received this message because you are subscribed to the Google
<br>> Groups "cp2k" group.
<br>> To unsubscribe from this group and stop receiving emails from it, send
<br>> an email to <a href data-email-masked rel="nofollow">cp...@googlegroups.com</a>
<br>> <mailto:<a href data-email-masked rel="nofollow">cp...@googlegroups.com</a>>.
<br>> To view this discussion on the web visit
<br>> <a href="https://groups.google.com/d/msgid/cp2k/c8f0d53a-bef6-44ae-ba86-e8752d0bc79dn%40googlegroups.com" target="_blank" rel="nofollow" data-saferedirecturl="https://www.google.com/url?hl=en&q=https://groups.google.com/d/msgid/cp2k/c8f0d53a-bef6-44ae-ba86-e8752d0bc79dn%2540googlegroups.com&source=gmail&ust=1607803149096000&usg=AFQjCNGoHzP2Knir27Y6APxOy2nSFm_MKw">https://groups.google.com/d/msgid/cp2k/c8f0d53a-bef6-44ae-ba86-e8752d0bc79dn%40googlegroups.com</a>
<br>> <<a href="https://groups.google.com/d/msgid/cp2k/c8f0d53a-bef6-44ae-ba86-e8752d0bc79dn%40googlegroups.com?utm_medium=email&utm_source=footer" target="_blank" rel="nofollow" data-saferedirecturl="https://www.google.com/url?hl=en&q=https://groups.google.com/d/msgid/cp2k/c8f0d53a-bef6-44ae-ba86-e8752d0bc79dn%2540googlegroups.com?utm_medium%3Demail%26utm_source%3Dfooter&source=gmail&ust=1607803149097000&usg=AFQjCNHPXXlW388Ucqx-OgVShCCV3JDtEA">https://groups.google.com/d/msgid/cp2k/c8f0d53a-bef6-44ae-ba86-e8752d0bc79dn%40googlegroups.com?utm_medium=email&utm_source=footer</a>>.
<br>
<br>--
<br>Tiziano Müller
<br>University of Zurich
<br>Department of Chemistry
<br>Winterthurerstrasse 190
<br>CH-8057 Zürich
<br>
<br>Tel: +41 44 63 54234
<br><a href="http://www.chem.uzh.ch" target="_blank" rel="nofollow" data-saferedirecturl="https://www.google.com/url?hl=en&q=http://www.chem.uzh.ch&source=gmail&ust=1607803149097000&usg=AFQjCNHI9N9nS1Qwhuh-A397hrCzG0G0GA">www.chem.uzh.ch</a>
<br><a href data-email-masked rel="nofollow">tiz...@chem.uzh.ch</a>
<br></blockquote></div>