<div>I found I was partially wrong. In the arch file it does provide std=c11 for CFLAGS, however, c11 will make code with clock_gettime fail (see https://stackoverflow.com/a/44812655). Changed to std=gnu11 and added qopenmp made it work.</div><div><br></div><div>I compiled the libraries manually, then use the toolchain script to generate the arch files, then correct the arch files manually to succeed.</div><div><br></div><div>I have found that the libraries compiled by the toolchain script is not optimal (though the performance impact might be negligible), mostly because not specifying '-xCORE-AVX512'.</div><div><br></div><div>There is a small issue for libint, v2.6.0 install will fail and no fortran module is installed. Had to try v2.7.</div><div>ELPA is slightly stranger, I don't know why the toolchain script wants to use libmkl_gf, even though I am using intel compilers. Also the strange thing is that the link args provided by MKL advisor does not work (which contains -Wl,--start-group and end-group to work around circular dependence), libmkl_core has a lot of undefined references. I learned from outputs of the toolchain script to make it work.</div><div><br></div><div>Here are my configure and make commands:</div><div>suppose the place I install the libraries is $HOME/cp2k_install/lib</div><div>libraries are all from cp2k.org links provided in the toolchain script</div><div><br></div><div>libint (v2.6.0 fortran module compilation is so broken, had to turn to v2.7.0 beta 5):</div><div><br></div><div>mkdir build</div><div>cd build</div><div>cmake -DCMAKE_INSTALL_PREFIX=$HOME/cp2k_install -DCMAKE_C_COMPILER=icc -DCMAKE_C_FLAGS="-qopenmp -O2 -xCORE-AVX512" -DCMAKE_CXX_COMPILER=icpc -DCMAKE_CXX_FLAGS="-qopenmp -O2 -xCORE-AVX512" -DCMAKE_Fortran_COMPILER=ifort -DCMAKE_Fortran_FLAGS="-qopenmp -O2 -xCORE-AVX512" -DLIBINT2_INSTALL_LIBDIR=$HOME/cp2k_install/lib -DREQUIRE_CXX_API=OFF -DENABLE_FORTRAN=ON ..</div><div># note that the toolchain script incorrectly uses -DCXXFLAGS=, and did not specify Fortran compilers.<br></div><div>CMAKE_BUILD_PARALLEL_LEVEL=8 cmake --build .</div><div>cmake --build . --target install<br></div><div><br></div><div># v2.6.0 configure command. Failed when make install tried to build the fortran example, thus refusing to install the fortran module</div><div># the trick in the toolchain script that replaces CXX CXXFLAGS by FC FCFLAGS in fortran/Makefile did not work for me, as either way will fail, for different reasons.<br></div><div># ./configure --prefix=$HOME/cp2k_install --libdir=$HOME/cp2k_install/lib --enable-fortran --with-cxx=icpc --with-cxx-optflags="-qopenmp -O2 -xCORE-AVX512" --with-fc=ifort --with-boost-libdir=$HOME/software/boost_1_76_0/stage/lib FCFLAGS="-qopenmp -O2 -xCORE-AVX512"</div><div># make -j8 && make install</div><div><br></div><div><br></div><div>libxc (needs to specify gnu99 explicitly, configure won't test and add it):</div><div><br></div><div>./configure --prefix=$HOME/cp2k_install --libdir=$HOME/cp2k_install/lib --disable-lxc CC=icc CFLAGS="-qopenmp -O2 -xCORE-AVX512 -std=gnu99" FC=ifort FCFLAGS="-qopenmp -O2 -xCORE-AVX512"</div><div> make -j8 && make install <br></div><div><br></div><div><br></div><div>libxsmm (toolchain does not specify AVX=3):</div><div><br></div><div>make -j 8 CXX=icpc CC=icc FC=ifort INTRINSICS=1 AVX=3 PREFIX=$HOME/cp2k_install</div><div>make -j 8 CXX=icpc CC=icc FC=ifort INTRINSICS=1 AVX=3 PREFIX=$HOME/cp2k_install install<br></div><div><br></div><div><br></div><div>elpa (pretty strange that I had to delete -Wl,--start-group and end-group):</div><div><br></div><div>mkdir build</div><div>cd build</div><div>../configure --prefix=$HOME/cp2k_install --libdir=$HOME/cp2k_install/lib --enable-openmp=yes --enable-shared=no --enable-static=yes --enable-avx=yes --enable-avx2=yes --enable-avx512=yes FC=mpifort CC=mpicc CXX=mpicxx FCFLAGS="-O2 -xCORE-AVX512 -I${MKLROOT}/include/intel64/lp64 -I${MKLROOT}/include" CFLAGS="-O2 -xCORE-AVX512 -I${MKLROOT}/include" CXXFLAGS="-O2 -xCORE-AVX512 -I${MKLROOT}/include" LIBS="-L${MKLROOT}/lib/intel64 -lmkl_scalapack_lp64 -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lmkl_blacs_openmpi_lp64 -lpthread -lm -ldl"</div><div>make -j1 && make install</div><div># note that cp2k requires sequential version of the MKL<br></div><div><br></div><div><br></div><div>spglib:</div><div><br></div><div>mkdir build</div><div>cd build<br></div><div>cmake -DCMAKE_C_COMPILER=icc -DCMAKE_INSTALL_PREFIX=$HOME/cp2k_install -DCMAKE_INSTALL_LIBDIR=lib -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=NO -DCMAKE_VERBOSE_MAKEFILE=ON ..</div><div>make -j8 symspg && make install</div><div># then manually delete those *.so* files in $HOME/cp2k_instal/lib<br></div><div><br></div><div><br></div><div>libvori:</div><div><br></div><div>mkdir build</div><div>cd build<br></div><div>cmake -DCMAKE_CXX_COMPILER=icpc -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$HOME/cp2k_install -DCMAKE_INSTALL_LIBDIR=lib -DCMAKE_INSTALL_RPATH_USE_LINK_PATH=ON -DCMAKE_VERBOSE_MAKEFILE=ON ..</div><div>CMAKE_BUILD_PARALLEL_LEVEL=8 cmake --build .<br>CMAKE_BUILD_PARALLEL_LEVEL=8 cmake --build . --target test<br>CMAKE_BUILD_PARALLEL_LEVEL=8 cmake --build . --target install<br></div><div class="gmail_quote"><br></div><div class="gmail_quote"><br></div><div class="gmail_quote">The environment variables needed (put in a file then source it):</div><div class="gmail_quote">export PATH=$HOME/cp2k_install/bin:$PATH # required for the toolchain script to find your libxsmm<br>export CPATH=$HOME/cp2k_install/include:$CPATH # required for the toolchain script to find your include dir<br>export LIBRARY_PATH=$HOME/cp2k_install/lib:/usr/lib64:$LIBRARY_PATH<br>export LD_LIBRARY_PATH=$HOME/cp2k_install/lib:/usr/lib64:$LD_LIBRARY_PATH # my case is strange, libstdc++ is in /usr/lib64 and it was not in the path<br>export LD_RUN_PATH=$HOME/cp2k_install/lib:/usr/lib64:$LD_RUN_PATH</div><div class="gmail_quote"><br></div><div class="gmail_quote"><br></div><div class="gmail_quote">Then run the script to generate the arch files. A small issue exists in cp2k/tools/toolchain/scripts/stage4/install_libxsmm.sh, change <span>libxsmm_generator</span> to <span>libxsmm_gemm_generator.</span></div><div class="gmail_quote"><span>Notice that this also uses gf version of the MKL, needs to change a bit.<br></span></div><div class="gmail_quote"><span><br></span></div><div class="gmail_quote"><span>./install_cp2k_toolchain.sh -j 8 --no-check-certificate --mpi-mode=openmpi --math-mode=mkl --with-gcc=no --with-intel=system --with-cmake=system --with-openmpi=system --with-mpich=no --with-intelmpi=no --with-libxc=system --with-libint=system --with-fftw=no --with-acml=no --with-mkl=system --with-openblas=no --with-scalapack=no --with-libxsmm=system --with-elpa=system --with-cosma=no --with-sirius=no --with-spglib=system --with-libvori=system</span> </div><div class="gmail_quote"><br></div><div class="gmail_quote"><br></div><div class="gmail_quote">For the arch files generated, for example, local_static.psmp</div><div class="gmail_quote">in CFLAGS, change -std=c11 to -std=gnu11, and add -qopenmp -xCORE-AVX512<br></div><div class="gmail_quote">in FCFLAGS, add -qopenmp -nofor-main -xCORE-AVX512<br></div><div class="gmail_quote">CXXFLAGS somehow already has -fopenmp. Change -march=native -mtune=native to -xCORE-AVX512<br></div><div class="gmail_quote">in LDFLAGS, remove those rpath, as ifort does not recognize them. Change to -static-intel if some libs (in my case openmpi) have only shared libs.<br></div><div class="gmail_quote">in LIBS, change -lmkl_gf_lp64 to -lmkl_intel_lp64. Remove some duplicated libs to keep only the rightmost one. I also noticed that in my /usr/lib64 there is an ancient version of libxc as well. Explicitly change -lxcf03 -lxc to absolute paths of those *.a files to avoid possible confusions. I also added -L/usr/lib64 after mkl and before -lpthread -lm -ldl -lstdc++, for those standard libraries (may be weird but I have to do so). In addition, even with -static-intel, mkl is still dynamic. Maybe should just use absolute paths as well (i.e., change to what MKL link advisor gives for static linking). </div><div class="gmail_quote"><br></div><div class="gmail_quote">make -j 8 ARCH=local_static VERSION=psmp</div><div class="gmail_quote">Done!<br></div><div class="gmail_quote"><br></div> <p></p> -- <br /> You received this message because you are subscribed to the Google Groups "cp2k" group.<br /> To unsubscribe from this group and stop receiving emails from it, send an email to <a href="mailto:cp2k+unsubscribe@googlegroups.com">cp2k+unsubscribe@googlegroups.com</a>.<br /> To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/cp2k/621e2a0d-4d7e-4abf-84da-5ac1dd7b0410n%40googlegroups.com?utm_medium=email&utm_source=footer">https://groups.google.com/d/msgid/cp2k/621e2a0d-4d7e-4abf-84da-5ac1dd7b0410n%40googlegroups.com</a>.<br />