[CP2K-user] [CP2K:16792] Re: Intel compiler compilation help (missing std=gnu99, qopenmp)

zc62 chenzh1993 at gmail.com
Thu Mar 31 05:11:36 UTC 2022

I found I was partially wrong. In the arch file it does provide std=c11 for 
CFLAGS, however, c11 will make code with clock_gettime fail (see 
https://stackoverflow.com/a/44812655). Changed to std=gnu11 and added 
qopenmp made it work.

I compiled the libraries manually, then use the toolchain script to 
generate the arch files, then correct the arch files manually to succeed.

I have found that the libraries compiled by the toolchain script is not 
optimal (though the performance impact might be negligible), mostly because 
not specifying '-xCORE-AVX512'.

There is a small issue for libint, v2.6.0 install will fail and no fortran 
module is installed. Had to try v2.7.
ELPA is slightly stranger, I don't know why the toolchain script wants to 
use libmkl_gf, even though I am using intel compilers. Also the strange 
thing is that the link args provided by MKL advisor does not work (which 
contains -Wl,--start-group and end-group to work around circular 
dependence), libmkl_core has a lot of undefined references. I learned from 
outputs of the toolchain script to make it work.

Here are my configure and make commands:
suppose the place I install the libraries is $HOME/cp2k_install/lib
libraries are all from cp2k.org links provided in the toolchain script

libint (v2.6.0 fortran module compilation is so broken, had to turn to 
v2.7.0 beta 5):

mkdir build
cd build
-DCMAKE_CXX_FLAGS="-qopenmp -O2 -xCORE-AVX512" 
-DCMAKE_Fortran_COMPILER=ifort -DCMAKE_Fortran_FLAGS="-qopenmp -O2 
-xCORE-AVX512" -DLIBINT2_INSTALL_LIBDIR=$HOME/cp2k_install/lib 
# note that the toolchain script incorrectly uses -DCXXFLAGS=, and did not 
specify Fortran compilers.
cmake --build . --target install

# v2.6.0 configure command. Failed when make install tried to build the 
fortran example, thus refusing to install the fortran module
# the trick in the toolchain script that replaces CXX CXXFLAGS by FC 
FCFLAGS in fortran/Makefile did not work for me, as either way will fail, 
for different reasons.
# ./configure --prefix=$HOME/cp2k_install --libdir=$HOME/cp2k_install/lib 
--enable-fortran --with-cxx=icpc --with-cxx-optflags="-qopenmp -O2 
-xCORE-AVX512" --with-fc=ifort 
--with-boost-libdir=$HOME/software/boost_1_76_0/stage/lib FCFLAGS="-qopenmp 
-O2 -xCORE-AVX512"
# make -j8 && make install

libxc (needs to specify gnu99 explicitly, configure won't test and add it):

./configure --prefix=$HOME/cp2k_install --libdir=$HOME/cp2k_install/lib 
--disable-lxc CC=icc CFLAGS="-qopenmp -O2 -xCORE-AVX512 -std=gnu99" 
FC=ifort FCFLAGS="-qopenmp -O2 -xCORE-AVX512"
make -j8 && make install 

libxsmm (toolchain does not specify AVX=3):

make -j 8 CXX=icpc CC=icc FC=ifort INTRINSICS=1 AVX=3 
make -j 8 CXX=icpc CC=icc FC=ifort INTRINSICS=1 AVX=3 
PREFIX=$HOME/cp2k_install install

elpa (pretty strange that I had to delete -Wl,--start-group and end-group):

mkdir build
cd build
../configure --prefix=$HOME/cp2k_install --libdir=$HOME/cp2k_install/lib 
--enable-openmp=yes --enable-shared=no --enable-static=yes --enable-avx=yes 
--enable-avx2=yes --enable-avx512=yes FC=mpifort CC=mpicc CXX=mpicxx 
FCFLAGS="-O2 -xCORE-AVX512 -I${MKLROOT}/include/intel64/lp64 
-I${MKLROOT}/include" CFLAGS="-O2 -xCORE-AVX512 -I${MKLROOT}/include" 
CXXFLAGS="-O2 -xCORE-AVX512 -I${MKLROOT}/include" 
LIBS="-L${MKLROOT}/lib/intel64 -lmkl_scalapack_lp64 -lmkl_intel_lp64 
-lmkl_sequential -lmkl_core -lmkl_blacs_openmpi_lp64 -lpthread -lm -ldl"
make -j1 && make install
# note that cp2k requires sequential version of the MKL


mkdir build
cd build
make -j8 symspg && make install
# then manually delete those *.so* files in  $HOME/cp2k_instal/lib


mkdir build
cd build
CMAKE_BUILD_PARALLEL_LEVEL=8 cmake --build . --target test
CMAKE_BUILD_PARALLEL_LEVEL=8 cmake --build . --target install

The environment variables needed (put in a file then source it):
export PATH=$HOME/cp2k_install/bin:$PATH  # required for the toolchain 
script to find your libxsmm
export CPATH=$HOME/cp2k_install/include:$CPATH  # required for the 
toolchain script to find your include dir
export LIBRARY_PATH=$HOME/cp2k_install/lib:/usr/lib64:$LIBRARY_PATH
export LD_LIBRARY_PATH=$HOME/cp2k_install/lib:/usr/lib64:$LD_LIBRARY_PATH  
# my case is strange, libstdc++ is in /usr/lib64 and it was not in the path
export LD_RUN_PATH=$HOME/cp2k_install/lib:/usr/lib64:$LD_RUN_PATH

Then run the script to generate the arch files. A small issue exists in 
cp2k/tools/toolchain/scripts/stage4/install_libxsmm.sh, change  
libxsmm_generator to  libxsmm_gemm_generator.
Notice that this also uses gf version of the MKL, needs to change a bit.

./install_cp2k_toolchain.sh -j 8 --no-check-certificate --mpi-mode=openmpi 
--math-mode=mkl --with-gcc=no --with-intel=system --with-cmake=system 
--with-openmpi=system --with-mpich=no --with-intelmpi=no 
--with-libxc=system --with-libint=system --with-fftw=no --with-acml=no 
--with-mkl=system --with-openblas=no --with-scalapack=no 
--with-libxsmm=system --with-elpa=system --with-cosma=no --with-sirius=no 
--with-spglib=system --with-libvori=system 

For the arch files generated, for example, local_static.psmp
in CFLAGS, change -std=c11 to -std=gnu11, and add -qopenmp -xCORE-AVX512
in FCFLAGS, add -qopenmp -nofor-main -xCORE-AVX512
CXXFLAGS somehow already has -fopenmp. Change -march=native -mtune=native 
to -xCORE-AVX512
in LDFLAGS, remove those rpath, as ifort does not recognize them. Change to 
-static-intel if some libs (in my case openmpi) have only shared libs.
in LIBS, change -lmkl_gf_lp64 to -lmkl_intel_lp64. Remove some duplicated 
libs to keep only the rightmost one. I also noticed that in my /usr/lib64 
there is an ancient version of libxc as well. Explicitly change -lxcf03 
-lxc to absolute paths of those *.a files to avoid possible confusions. I 
also added -L/usr/lib64 after mkl and before -lpthread -lm -ldl -lstdc++, 
for those standard libraries (may be weird but I have to do so). In 
addition, even with -static-intel, mkl is still dynamic. Maybe should just 
use absolute paths as well (i.e., change to what MKL link advisor gives for 
static linking). 

make -j 8 ARCH=local_static VERSION=psmp

You received this message because you are subscribed to the Google Groups "cp2k" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cp2k+unsubscribe at googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cp2k/621e2a0d-4d7e-4abf-84da-5ac1dd7b0410n%40googlegroups.com.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.cp2k.org/archives/cp2k-user/attachments/20220330/8fdfb3c4/attachment-0001.htm>

More information about the CP2K-user mailing list