[CP2K:8330] Problem with compilation with multi-threading + mkl + fftw3 (and gnu compiler)

[Iain Bethune]

Hi Daniele,

The problem here lies in fact that Intel only ship an mkl_version.h header with the MKL version using intel’s proprietary preprocessor syntax.  We need this in order to check we have an MKL version where the multi-threaded FFT works correctly (older than 11.1.0 gave wrong results).  However, with if compiling with gfortran and using MKL, we can’t make this check so have to fail safe with the error message you found.

You should be able to workaround this by building FFTW3 with threading enabled.  What I think you missed is that you need to link the fftw3_omp library before the MKL stuff in your LIBS variable, so that the genuine FFTW code gets linked, rather than the MKL-FFTW interface.  You should also be able to continue using threaded MKL ( libmkl_gnu_thread.a) for other parts of MKL (LAPACK etc.)

So try putting -lfftw3 -lfftw3_omp before the MKL libs and rebuild.

Cheers

- Iain

--

Iain Bethune
Project Manager, EPCC

Email: ibet... at epcc.ed.ac.uk
Twitter: @IainBethune @PrimeGrid @CP2Kproject
Web: http://www2.epcc.ed.ac.uk/~ibethune
Tel/Fax: +44 (0)131 650 5201/6555
Mob: +44 (0)7598317015
Addr: 2404 JCMB, The King's Buildings, Peter Guthrie Tait Road, Edinburgh, EH9 3FD

> On 1 Nov 2016, at 18:39, Daniele Fontanari <danielef... at gmail.com> wrote:
> 
> Small update:
> 
> Looking at old threads I tried to modify the arch file (be careful, last line begin with a tab, not several spaces):
> 
> ## Begin test.ssmp ##
> MKLROOT = /opt/intel/compilers_and_libraries_2016.0.109/linux/mkl
> MKLLIB = $(MKLROOT)/lib/intel64
> FFTW3ROOT = $(HOME)/cp2k_build/src/fftw3_vanilla/fftw-3.3.5/installazione
> 
> CC = gcc
> CPP =
> FC = mpif90
> LD = mpif90
> AR = ar -r
> ALLFLAGS = -D__MKL -D__FFTW3 -O2 -I$(MKLROOT)/include -I$(FFTW3ROOT)/include
> FCFLAGS = $(ALLFLAGS) -ffree-form -fopenmp
> FCFLAGS2 = $(ALLFLAGS) -ffree-form
> CFLAGS = $(ALLFLAGS) -fopenmp
> LDFLAGS = -L$(FFTW3ROOT)/lib -fopenmp
> LIBS = -Wl,--start-group $(MKLLIB)/libmkl_gf_lp64.a $(MKLLIB)/libmkl_sequential.a $(MKLLIB)/libmkl_core.a -Wl,--end-group -lpthread -lm -ldl -lstdc++ -lfftw3
> 
> fftw3_lib.o: pw/fft/fftw3_lib.F
>                  $(FC) -c $(FCFLAGS2) $<dfontanari at orval02:~/cp2k/cp2k/makefiles$ 
> ## End test.ssmp ##
> 
> and regtest is working. Am I losing a lot of performance with this workaround (fftw3 not multithreaded)? Moreover It was suggested to use libmkl_sequential.a instead of libmkl_gnu_thread.a care to comment on this?
> 
> However I'd appreciate is someone can point out possible reasons for my previous error
> 
> Thanks!
> 
> Daniele
> 
> 
> Il giorno martedì 1 novembre 2016 18:31:18 UTC+1, Daniele Fontanari ha scritto:
> Hello everyone
> 
> I have now a cp2k version which is running smoothly if I compile the *opt versions, however I have problems (in term of runtime failures) with the *smp version.
> 
> I can access the linear algebra routines through a MKL library, which provide also a wrapper for FFTW3 (but I DON'T have access to the intel compilers, I use the GNU ones)
> The runtime error for the smp version is the following:
> 
> STOP Intel's FFTW3 interface to MKL is not thread-safe prior to MKL 11.1.0!  Please rebuild CP2K, linking against FFTW 3 from www.fftw.org or a newer version of MKL. Now exiting...
> 
> This is quite strange since my version of MKL should be newer than 11.1.0 (I am not the sysadmin, the intel folder is referred as "2016.0.109", a quick search suggests that the 11.3 version should be installed, but I'd like to know a way to check this for sure)
> 
> However if I try to install the "vanilla" version of fftw3 and try to install with the configure option "--enable-openmp" and thus linking the libraries "libfftw3.a" and "libfftw3_omp.a" I get the same error.
> the arch file I am using is the following:
> (the fftw libraries are in $(FFTW3ROOT)/lib )
> 
> 
> ## Begin test.ssmp ##
> MKLROOT    = /opt/intel/compilers_and_libraries_2016.0.109/linux/mkl
> MKLLIB     = $(MKLROOT)/lib/intel64
> FFTW3ROOT  = $(HOME)/cp2k_build/src/fftw3_vanilla/fftw-3.3.5/installazione
> 
> CC         = gcc
> CPP        =
> FC         = mpif90
> LD         = mpif90
> AR         = ar -r
> ALLFLAGS   = -D__MKL -D__FFTW3 -O2 -fopenmp -I$(MKLROOT)/include -I$(FFTW3ROOT)/include
> FCFLAGS    = $(ALLFLAGS) -ffree-form
> CFLAGS     = $(ALLFLAGS)
> LDFLAGS    = -L$(FFTW3ROOT)/lib -fopenmp
> LIBS       = -Wl,--start-group $(MKLLIB)/libmkl_gf_lp64.a $(MKLLIB)/libmkl_gnu_thread.a $(MKLLIB)/libmkl_core.a -Wl,--end-group -lpthread -lm -ldl -lstdc++ -lfftw3 -lfftw3_omp
> ## End test.ssmp ##
> 
> 
> Just recently I noticed that the installation wiki on the website might be saying that if I compile with GNU+MKL I cannot have openmp? Is this the case?
> 
> Thanks
> 
> Daniele
> 
> -- 
> You received this message because you are subscribed to the Google Groups "cp2k" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to cp2k+uns... at googlegroups.com.
> To post to this group, send email to cp... at googlegroups.com.
> Visit this group at https://groups.google.com/group/cp2k.
> For more options, visit https://groups.google.com/d/optout.


-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.