<div><span>Hi Amit,</span></div><div><span><br></span></div><div><span>if you are getting <span>cudaErrorStubLibrary then the app failed to load the real driver. Presumably your local admin can help with this. See also </span><a href="https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__TYPES.html#group__CUDART__TYPES_1g3f51e3575c2178246db0a94a430e0038">the CUDA docs</a>:<br></span></div><div><br></div><div><span><span> cudaErrorStubLibrary = <span>34</span></span></span>
<span>This indicates that the CUDA driver that the application has loaded is a stub library. Applications that run with the stub
rather than a real driver loaded will result in CUDA API returning this error. <br></span></div><div><span><br></span></div><div><span>-Ole<br></span></div><div><span><br></span></div><div class="gmail_quote"><div dir="auto" class="gmail_attr">On Sunday, October 30, 2022 at 9:59:11 PM UTC+1 Amit Gupta wrote:<br/></div><blockquote class="gmail_quote" style="margin: 0 0 0 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;"><div dir="ltr"><div>Hi,</div><div>I compiled a CUDA compatible version of CP2K, but when I submit my jobs, I get the following error:</div><div>--------------------------------------------------------------------------<br>ERROR: cudaErrorStubLibrary /scratch/ag9288/softwares/cp2k-2022.2/src/offload/offload_library.c 49<br>forrtl: error (76): Abort trap signal<br>Image PC Routine Line Source <br>cp2k.psmp 0000000006B83CAB Unknown Unknown Unknown<br>libpthread-2.28.s 000014D73245EB20 Unknown Unknown Unknown<br><a href="http://libc-2.28.so" target="_blank" rel="nofollow" data-saferedirecturl="https://www.google.com/url?hl=en&q=http://libc-2.28.so&source=gmail&ust=1667292539246000&usg=AOvVaw2hVJhC_u36YThrue0YUUv7">libc-2.28.so</a> 000014D712D4037F gsignal Unknown Unknown<br><a href="http://libc-2.28.so" target="_blank" rel="nofollow" data-saferedirecturl="https://www.google.com/url?hl=en&q=http://libc-2.28.so&source=gmail&ust=1667292539246000&usg=AOvVaw2hVJhC_u36YThrue0YUUv7">libc-2.28.so</a> 000014D712D2ADB5 abort Unknown Unknown<br>cp2k.psmp 00000000031F8C29 offload_get_devic 49 offload_library.c<br>cp2k.psmp 0000000001387F46 f77_interface_mp_ 276 f77_interface.F<br>cp2k.psmp 000000000042281D MAIN__ 284 cp2k.F<br>cp2k.psmp 0000000006BED096 Unknown Unknown Unknown<br><a href="http://libc-2.28.so" target="_blank" rel="nofollow" data-saferedirecturl="https://www.google.com/url?hl=en&q=http://libc-2.28.so&source=gmail&ust=1667292539246000&usg=AOvVaw2hVJhC_u36YThrue0YUUv7">libc-2.28.so</a> 000014D712D2C493 __libc_start_main Unknown Unknown<br>cp2k.psmp 000000000042195E Unknown Unknown Unknown<br>--------------------------------------------------------------------------</div><div><br></div><div>It looks like the executable is unable to find the card, while all the libraries are correctly loaded.</div><div><br></div><div>Version: <br></div><div>Intel fortran: 19.1</div><div>cuda : 11.3</div><div>openmpi 4.1.1</div><div>CP2K: 2022.2</div><div><br></div><div>Steps I took to build the binary:</div><div>1. Install the toolchain:</div><div>./install_cp2k_toolchain.sh
--with-intel=system --enable-cuda --gpu-ver=V100 --with-openmpi=system
--with-libxc=install --with-libint=install --with-fftw=install
--with-mkl=system --with-libxsmm=install --with-elpa=install
--with-ptscotch=no --with-superlu=no --with-pexsi=no
--with-quip=no --with-plumed=no --with-sirius=no
--with-gsl=no --with-libvdwxc=no --with-spglib=no
--with-hdf5=no --with-spfft=no --with-spla=install
--with-cosma=install --with-libvori=no --with-openblas=no -j4</div><div><br></div><div>2. Manualy build and copy Libint, for the fortran makefile bug:</div><div><a href="https://github.com/cp2k/libint-cp2k/releases" target="_blank" rel="nofollow" data-saferedirecturl="https://www.google.com/url?hl=en&q=https://github.com/cp2k/libint-cp2k/releases&source=gmail&ust=1667292539246000&usg=AOvVaw3P4cNncXW1gE8-w8Q_LuXp">https://github.com/cp2k/libint-cp2k/releases</a><br>cmake
-DCMAKE_C_COMPILER=icc -DCMAKE_C_FLAGS="-qopenmp -O2"
-DCMAKE_CXX_COMPILER=icpc -DCMAKE_CXX_FLAGS="-qopenmp -O2"
-DCMAKE_Fortran_COMPILER=ifort -DCMAKE_Fortran_FLAGS="-qopenmp -O2"
-DREQUIRE_CXX_API=OFF -DENABLE_FORTRAN=ON ..</div><div><br></div><div>3. Apply required CUDA patch to CP2K and compile:</div><div><a href="https://github.com/cp2k/cp2k/commit/ee6c3aa.patch" target="_blank" rel="nofollow" data-saferedirecturl="https://www.google.com/url?hl=en&q=https://github.com/cp2k/cp2k/commit/ee6c3aa.patch&source=gmail&ust=1667292539246000&usg=AOvVaw1aMrI-afBEq1vr7XnqIwEJ">https://github.com/cp2k/cp2k/commit/ee6c3aa.patch</a></div><div> make -j 4 ARCH=local_cuda VERSION="ssmp sdbg psmp pdbg"</div><div><br></div><div>PS:
The job-node is different from the compile node, could that might be an
issue? Also while the compilation went fine, there was an error in
symlinking the binaries.<br></div></div>
</blockquote></div>
<p></p>
-- <br />
You received this message because you are subscribed to the Google Groups "cp2k" group.<br />
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="mailto:cp2k+unsubscribe@googlegroups.com">cp2k+unsubscribe@googlegroups.com</a>.<br />
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/cp2k/49ebddbe-ec34-4f10-b1a9-d91514836765n%40googlegroups.com?utm_medium=email&utm_source=footer">https://groups.google.com/d/msgid/cp2k/49ebddbe-ec34-4f10-b1a9-d91514836765n%40googlegroups.com</a>.<br />