[CP2K-user] [CP2K:16792] Re: Intel compiler compilation help (missing std=gnu99, qopenmp)

[zc62]

I found I was partially wrong. In the arch file it does provide std=c11 for 
CFLAGS, however, c11 will make code with clock_gettime fail (see 
https://stackoverflow.com/a/44812655). Changed to std=gnu11 and added 
qopenmp made it work.

I compiled the libraries manually, then use the toolchain script to 
generate the arch files, then correct the arch files manually to succeed.

I have found that the libraries compiled by the toolchain script is not 
optimal (though the performance impact might be negligible), mostly because 
not specifying '-xCORE-AVX512'.

There is a small issue for libint, v2.6.0 install will fail and no fortran 
module is installed. Had to try v2.7.
ELPA is slightly stranger, I don't know why the toolchain script wants to 
use libmkl_gf, even though I am using intel compilers. Also the strange 
thing is that the link args provided by MKL advisor does not work (which 
contains -Wl,--start-group and end-group to work around circular 
dependence), libmkl_core has a lot of undefined references. I learned from 
outputs of the toolchain script to make it work.

Here are my configure and make commands:
suppose the place I install the libraries is $HOME/cp2k_install/lib
libraries are all from cp2k.org links provided in the toolchain script

libint (v2.6.0 fortran module compilation is so broken, had to turn to 
v2.7.0 beta 5):

mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=$HOME/cp2k_install -DCMAKE_C_COMPILER=icc 
-DCMAKE_C_FLAGS="-qopenmp -O2 -xCORE-AVX512" -DCMAKE_CXX_COMPILER=icpc 
-DCMAKE_CXX_FLAGS="-qopenmp -O2 -xCORE-AVX512" 
-DCMAKE_Fortran_COMPILER=ifort -DCMAKE_Fortran_FLAGS="-qopenmp -O2 
-xCORE-AVX512" -DLIBINT2_INSTALL_LIBDIR=$HOME/cp2k_install/lib 
-DREQUIRE_CXX_API=OFF -DENABLE_FORTRAN=ON ..
# note that the toolchain script incorrectly uses -DCXXFLAGS=, and did not 
specify Fortran compilers.
CMAKE_BUILD_PARALLEL_LEVEL=8 cmake --build .
cmake --build . --target install

# v2.6.0 configure command. Failed when make install tried to build the 
fortran example, thus refusing to install the fortran module
# the trick in the toolchain script that replaces CXX CXXFLAGS by FC 
FCFLAGS in fortran/Makefile did not work for me, as either way will fail, 
for different reasons.
# ./configure --prefix=$HOME/cp2k_install --libdir=$HOME/cp2k_install/lib 
--enable-fortran --with-cxx=icpc --with-cxx-optflags="-qopenmp -O2 
-xCORE-AVX512" --with-fc=ifort 
--with-boost-libdir=$HOME/software/boost_1_76_0/stage/lib FCFLAGS="-qopenmp 
-O2 -xCORE-AVX512"
# make -j8 && make install


libxc (needs to specify gnu99 explicitly, configure won't test and add it):

./configure --prefix=$HOME/cp2k_install --libdir=$HOME/cp2k_install/lib 
--disable-lxc CC=icc CFLAGS="-qopenmp -O2 -xCORE-AVX512 -std=gnu99" 
FC=ifort FCFLAGS="-qopenmp -O2 -xCORE-AVX512"
make -j8 && make install 


libxsmm (toolchain does not specify AVX=3):

make -j 8 CXX=icpc CC=icc FC=ifort INTRINSICS=1 AVX=3 
PREFIX=$HOME/cp2k_install
make -j 8 CXX=icpc CC=icc FC=ifort INTRINSICS=1 AVX=3 
PREFIX=$HOME/cp2k_install install


elpa (pretty strange that I had to delete -Wl,--start-group and end-group):

mkdir build
cd build
../configure --prefix=$HOME/cp2k_install --libdir=$HOME/cp2k_install/lib 
--enable-openmp=yes --enable-shared=no --enable-static=yes --enable-avx=yes 
--enable-avx2=yes --enable-avx512=yes FC=mpifort CC=mpicc CXX=mpicxx 
FCFLAGS="-O2 -xCORE-AVX512 -I${MKLROOT}/include/intel64/lp64 
-I${MKLROOT}/include" CFLAGS="-O2 -xCORE-AVX512 -I${MKLROOT}/include" 
CXXFLAGS="-O2 -xCORE-AVX512 -I${MKLROOT}/include" 
LIBS="-L${MKLROOT}/lib/intel64 -lmkl_scalapack_lp64 -lmkl_intel_lp64 
-lmkl_sequential -lmkl_core -lmkl_blacs_openmpi_lp64 -lpthread -lm -ldl"
make -j1 && make install
# note that cp2k requires sequential version of the MKL


spglib:

mkdir build
cd build
cmake -DCMAKE_C_COMPILER=icc -DCMAKE_INSTALL_PREFIX=$HOME/cp2k_install 
-DCMAKE_INSTALL_LIBDIR=lib -DCMAKE_BUILD_TYPE=Release 
-DBUILD_SHARED_LIBS=NO -DCMAKE_VERBOSE_MAKEFILE=ON ..
make -j8 symspg && make install
# then manually delete those *.so* files in  $HOME/cp2k_instal/lib


libvori:

mkdir build
cd build
cmake -DCMAKE_CXX_COMPILER=icpc -DCMAKE_BUILD_TYPE=Release 
-DCMAKE_INSTALL_PREFIX=$HOME/cp2k_install -DCMAKE_INSTALL_LIBDIR=lib 
-DCMAKE_INSTALL_RPATH_USE_LINK_PATH=ON -DCMAKE_VERBOSE_MAKEFILE=ON ..
CMAKE_BUILD_PARALLEL_LEVEL=8 cmake --build .
CMAKE_BUILD_PARALLEL_LEVEL=8 cmake --build . --target test
CMAKE_BUILD_PARALLEL_LEVEL=8 cmake --build . --target install


The environment variables needed (put in a file then source it):
export PATH=$HOME/cp2k_install/bin:$PATH  # required for the toolchain 
script to find your libxsmm
export CPATH=$HOME/cp2k_install/include:$CPATH  # required for the 
toolchain script to find your include dir
export LIBRARY_PATH=$HOME/cp2k_install/lib:/usr/lib64:$LIBRARY_PATH
export LD_LIBRARY_PATH=$HOME/cp2k_install/lib:/usr/lib64:$LD_LIBRARY_PATH  
# my case is strange, libstdc++ is in /usr/lib64 and it was not in the path
export LD_RUN_PATH=$HOME/cp2k_install/lib:/usr/lib64:$LD_RUN_PATH


Then run the script to generate the arch files. A small issue exists in 
cp2k/tools/toolchain/scripts/stage4/install_libxsmm.sh, change  
libxsmm_generator to  libxsmm_gemm_generator.
Notice that this also uses gf version of the MKL, needs to change a bit.

./install_cp2k_toolchain.sh -j 8 --no-check-certificate --mpi-mode=openmpi 
--math-mode=mkl --with-gcc=no --with-intel=system --with-cmake=system 
--with-openmpi=system --with-mpich=no --with-intelmpi=no 
--with-libxc=system --with-libint=system --with-fftw=no --with-acml=no 
--with-mkl=system --with-openblas=no --with-scalapack=no 
--with-libxsmm=system --with-elpa=system --with-cosma=no --with-sirius=no 
--with-spglib=system --with-libvori=system 


For the arch files generated, for example, local_static.psmp
in CFLAGS, change -std=c11 to -std=gnu11, and add -qopenmp -xCORE-AVX512
in FCFLAGS, add -qopenmp -nofor-main -xCORE-AVX512
CXXFLAGS somehow already has -fopenmp. Change -march=native -mtune=native 
to -xCORE-AVX512
in LDFLAGS, remove those rpath, as ifort does not recognize them. Change to 
-static-intel if some libs (in my case openmpi) have only shared libs.
in LIBS, change -lmkl_gf_lp64 to -lmkl_intel_lp64. Remove some duplicated 
libs to keep only the rightmost one. I also noticed that in my /usr/lib64 
there is an ancient version of libxc as well. Explicitly change -lxcf03 
-lxc to absolute paths of those *.a files to avoid possible confusions. I 
also added -L/usr/lib64 after mkl and before -lpthread -lm -ldl -lstdc++, 
for those standard libraries (may be weird but I have to do so). In 
addition, even with -static-intel, mkl is still dynamic. Maybe should just 
use absolute paths as well (i.e., change to what MKL link advisor gives for 
static linking). 

make -j 8 ARCH=local_static VERSION=psmp
Done!

-- 
You received this message because you are subscribed to the Google Groups "cp2k" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cp2k+unsubscribe at googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cp2k/621e2a0d-4d7e-4abf-84da-5ac1dd7b0410n%40googlegroups.com.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.cp2k.org/archives/cp2k-user/attachments/20220330/8fdfb3c4/attachment-0001.htm>