<div>Hi Luis,<br></div><div><br></div>> Could the order of the CUDA DFLAGS elements be relevant?<div><br></div><div>No, the order of the DFLAGS does not matter. </div><div><br></div><div>> /home/cp2k-2022.2/src/pw/gpu/pw_gpu_kernels.h(22): error: identifier "offloadStream_t" is undefined</div><div><br></div><div>This is really strange!</div><div><br></div><div>The "offloadStream_t" is defined here: https://github.com/cp2k/cp2k/blob/support/v2022.2/src/offload/offload_runtime.h#L27</div><div>... which "pw_gpu_kernels.h" includes here: https://github.com/cp2k/cp2k/blob/support/v2022.2/src/pw/gpu/pw_gpu_kernels.h#L11</div><div><br></div><div>I'm afraid you'll have to do some digging on your local machine to get to the bottom of this.</div><div><br></div><div>-Ole</div><div><br></div><div class="gmail_quote"><div dir="auto" class="gmail_attr">On Wednesday, October 26, 2022 at 3:23:56 PM UTC+2 LC wrote:<br/></div><blockquote class="gmail_quote" style="margin: 0 0 0 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;"> <div> Thanks Ole,<br> <br> I've replaced my DFLAGS with the new ones:<br> <br> -D__MKL -D__FFTW3 -D__LIBXSMM -D__LIBINT -D__parallel -D__SCALAPACK -D__HAS_ISO_C_BINDING -D__DBCSR_ACC -D__OFFLOAD_CUDA -D__OFFLOAD_PROFILING -D__HAS_PATCHED_CUFFT_70 -D__COSMA -D__ELPA_NVIDIA_GPU -D__LIBVORI -D__SPGLIB<br> <br> Unfortunately, I am still getting the same error. Could the order of the CUDA DFLAGS elements be relevant?<br> <br> Regards</div><div><br> <br> <div>On 26/10/2022 14:03, Ole Schütt wrote:<br> </div> </div><div><blockquote type="cite"> <div>Hi Luis,</div> <div><br> </div> <div>the flags for GPU support have recently changed a bit:</div> <div><br> </div> <div> <a href="https://github.com/cp2k/cp2k/blob/master/INSTALL.md#2i-cuda-optional-improved-performance-on-gpu-systems" target="_blank" rel="nofollow" data-saferedirecturl="https://www.google.com/url?hl=en&q=https://github.com/cp2k/cp2k/blob/master/INSTALL.md%232i-cuda-optional-improved-performance-on-gpu-systems&source=gmail&ust=1666962324376000&usg=AOvVaw0EqJt1K8uM0F_N4TYRu5wV">https://github.com/cp2k/cp2k/blob/master/INSTALL.md#2i-cuda-optional-improved-performance-on-gpu-systems</a></div> <div><br> </div> <div> Try adding -D__OFFLOAD_CUDA<br> </div> <div><br> </div> <div>-Ole</div> <br> <div class="gmail_quote"> <div dir="auto" class="gmail_attr">On Wednesday, October 26, 2022 at 2:35:08 PM UTC+2 LC wrote:<br> </div> <blockquote class="gmail_quote" style="margin:0 0 0 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div dir="ltr"> <div dir="ltr"> <div>Hello,</div> <div><br> </div> <div>I am trying to build CP2K 2022.2 for NVIDIA GPUs. I've built ELPA with GPU support (no OpenMP, yes MPI), and COSMA 2.5 with GPU support too. The error I am getting is as follows:</div> <div><br> </div> <div>mpifort -c -D__MKL -D__FFTW3 -D__LIBXSMM -D__LIBINT -D__parallel -D__SCALAPACK -D__HAS_ISO_C_BINDING -D__ACC -D__DBCSR_ACC -D__PW_CUDA -D__GRID_CUDA -D__OFFLOAD_PROFILING -D__HAS_PATCHED_CUFFT_70 -D__COSMA -D__ELPA_NVIDIA_GPU -D__LIBVORI -D__SPGLIB -O3 -fopenmp -ffree-form -ffree-line-length-none -ftree-vectorize -funroll-loops -std=f2008 -I/opt/intel/compilers_and_libraries_2019.5.281/linux/mkl/include -I/home/COSMA/include -I/home/libxsmm-1.17.0/include -I/home/elpa-2021.11.002/include/elpa-2021.11.002/modules -I/home/elpa-2021.11.002/include/elpa-2021.11.002/elpa -I/home/elpa-2021.11.002/include/elpa-2021.11.002/src/helpers -I/home/fftw-3.3.10/include -I/home/libint-v2.6.0-cp2k-lmax-4/include -I/cuda/cuda-11.2/include -m64 -I/opt/intel/compilers_and_libraries_2019.5.281/linux/mkl/include -D__COMPILE_ARCH="\"nvidiaGPU\"" -D__COMPILE_REVISION="\"git:a95ec40\"" -D__DATA_DIR="\"/home/cp2k-2022.2/data\"" -D__SHORT_FILE__="\"xc/xc_input_constants.F\"" -I'/home/cp2k-2022.2/src/xc/' -I'/home/cp2k-2022.2/obj/nvidiaGPU/psmp/exts/dbcsr' xc_input_constants.F90 <br> </div> <div><br> </div> <div>/home/cp2k-2022.2/tools/build_utils/fypp -n /home/cp2k-2022.2/src/shg_int/constants_operator.F constants_operator.F90<br> cmp: /home/cp2k-2022.2/obj/git-ref: No such file or directory<br> /home/cp2k-2022.2/src/pw/gpu/pw_gpu_kernels.h(22): error: identifier "offloadStream_t" is undefined</div> <div><br> </div> <div>and here you can see my arch file:</div> <div><br> </div> <div><br> </div> <div>NVCC = ${CUDA_HOME}/bin/nvcc</div> GPUVER = V100<br> <br> CXX = mpicxx<br> CC = mpicc<br> FC = mpifort<br> LD = mpifort<br> AR = ar -r<br> <br> OFFLOAD_CC = nvcc<br> OFFLOAD_FLAGS = -O3 -g -w --std=c++11<br> <br> OPTFLAGS += -fopenmp -O3 -march=native<br> <br> DFLAGS += -D__MKL -D__FFTW3 -D__LIBXSMM -D__LIBINT \<br> -D__parallel -D__SCALAPACK -D__HAS_ISO_C_BINDING<br> <br> DFLAGS += -D__ACC -D__DBCSR_ACC -D__PW_CUDA -D__GRID_CUDA -D__OFFLOAD_PROFILING<br> DFLAGS += -D__HAS_PATCHED_CUFFT_70 -D__COSMA -D__ELPA_NVIDIA_GPU <br> <br> DFLAGS+= -D__LIBVORI -D__SPGLIB<br> <br> # C++ flags<br> CXXFLAGS = -O3 $(OPTFLAGS) -g -I${CUDA_HOME}/include -std=c++11<br> <br> # NVIDIA <br> NVFLAGS = $(DFLAGS) $(OPTFLAGS) -g -O3 -arch sm_70 -Xcompiler='-fopenmp' --std=c++11<br> <br> # Fortran flags<br> FCFLAGS = $(DFLAGS) -O3 -fopenmp -ffree-form -ffree-line-length-none<br> FCFLAGS += -ftree-vectorize -funroll-loops -std=f2008<br> INC_FLAGS = -I${MKLROOT}/include <br> INC_FLAGS += -I$(COSMA)/include<br> INC_FLAGS += -I$(LIBXSMM)/include <br> INC_FLAGS += -I$(INCELPA)/modules -I$(INCELPA)/elpa -I$(INCELPA)/src/helpers<br> INC_FLAGS += -I$(LIBFFTW3)/include<br> INC_FLAGS += -I$(LIBINT)/include<br> INC_FLAGS += -I${CUDA_HOME}/include<br> INC_FLAGS += -m64 -I${MKLROOT}/include<br> <br> # C++ flags<br> CXXFLAGS += $(INC_FLAGS)<br> <br> FCFLAGS += $(INC_FLAGS)<br> <br> # C flags<br> CFLAGS = $(DFLAGS) $(OPTFLAGS) $(INC_FLAGS)<br> <br> # Libraries<br> LDFLAGS = $(FCFLAGS) <br> LDFLAGS_C = $(FCFLAGS)<br> <br> # ELPA libs<br> LIBS = $(LIBELPA)/lib/libelpa.a<br> # Libvori lib<br> LIBS += $(LIBVORI)/lib/libvori.a<br> # LIBXSMM libs<br> LIBS += -L$(LIBXSMM)/lib -lxsmmf -lxsmm -lxsmmext <br> # LIBINT libs<br> LIBS += -L$(LIBINT)/lib -lint2 -lstdc++<br> # FFTW libs<br> LIBS += -L$(LIBFFTW3)/lib -lfftw3 -lfftw3_threads<br> #SPGLIB<br> LIBS += -L$(LIBSPG)/lib64 -lsymspg<br> #LIBXC<br> LIBS += -L$(LIBXC)/lib -lxcf03 -lxc<br> </div> <div dir="ltr"><br> </div> <div dir="ltr"># Scalapack libs<br> LIBS += -L/usr/lib64<br> <br> LIBS += -L$(MKLROOT)/lib/intel64 -Wl,--start-group -lmkl_scalapack_lp64 -lmkl_gf_lp64 -lmkl_sequential -lmkl_core -l<br> mkl_blacs_openmpi_lp64 -Wl,--end-group -lpthread -lz -ldl<br> <br> #COSMA<br> LIBS += -L$(COSMA_LIB)/ -lcosma_prefixed_pxgemm -lcosma -lcosta_prefixed_scalapack -lcosta<br> <br> #CUDA<br> LIBS += -L${CUDA_HOME}/lib64 -lstdc++ -lcudart -lnvrtc -lcuda -lcufft -lcublas -lnvToolsExt<br> <br> # Required due to memory leak that occurs if high optimisations are used<br> mp2_optimize_ri_basis.o: mp2_optimize_ri_basis.F<br> $(FC) -c $(subst O2,O0,$(FCFLAGS)) $< <div><br> </div> <div>Could anyone help me with this please?</div> <div><br> </div> <div>Regards</div> <div>LC<br> </div> </div> </div> </blockquote> </div></blockquote></div><div><blockquote type="cite"> -- <br> You received this message because you are subscribed to the Google Groups "cp2k" group.<br> To unsubscribe from this group and stop receiving emails from it, send an email to <a href data-email-masked rel="nofollow">cp2k+uns...@googlegroups.com</a>.<br> To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/cp2k/359554dc-fd75-466e-b155-2b14606e9ca9n%40googlegroups.com?utm_medium=email&utm_source=footer" target="_blank" rel="nofollow" data-saferedirecturl="https://www.google.com/url?hl=en&q=https://groups.google.com/d/msgid/cp2k/359554dc-fd75-466e-b155-2b14606e9ca9n%2540googlegroups.com?utm_medium%3Demail%26utm_source%3Dfooter&source=gmail&ust=1666962324376000&usg=AOvVaw3wnseesVQTMh9hpjvN2fp9">https://groups.google.com/d/msgid/cp2k/359554dc-fd75-466e-b155-2b14606e9ca9n%40googlegroups.com</a>.<br> </blockquote> <br> </div> </blockquote></div> <p></p> -- <br /> You received this message because you are subscribed to the Google Groups "cp2k" group.<br /> To unsubscribe from this group and stop receiving emails from it, send an email to <a href="mailto:cp2k+unsubscribe@googlegroups.com">cp2k+unsubscribe@googlegroups.com</a>.<br /> To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/cp2k/b3fab3a3-9441-41d3-99ff-a006c4c6979dn%40googlegroups.com?utm_medium=email&utm_source=footer">https://groups.google.com/d/msgid/cp2k/b3fab3a3-9441-41d3-99ff-a006c4c6979dn%40googlegroups.com</a>.<br />