<div>I resolved the above error, by symlinking to libcudart as libcuds.so.1. So I guess this issue is closed. <br></div><div>I ran in secondary issue of `cudaErrorInsufficientDriver` so I guess there is something wrong with my loaded CUDA libraries, So I will probably recompile it again with more carefully loaded latest driver. <br></div><div>Thank you<br></div><br><div class="gmail_quote"><div dir="auto" class="gmail_attr">On Monday, October 31, 2022 at 10:14:26 AM UTC-5 Amit Gupta wrote:<br/></div><blockquote class="gmail_quote" style="margin: 0 0 0 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">Could it be a compile time issue on my end? I remember toolchain requiring libcuda for linking (cosma I believe), and I found issues on different repositories on github, mentioning that libcuda resides in lib/stubs/libcuda.so. So I linked against it, provide its location at runtime in LD_LIBRARY_PATH. <br><br><div class="gmail_quote"><div dir="auto" class="gmail_attr">On Monday, October 31, 2022 at 3:55:13 AM UTC-5 Ole Schütt wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><span>Hi Amit,</span></div><div><span><br></span></div><div><span>if you are getting <span>cudaErrorStubLibrary then the app failed to load the real driver. Presumably your local admin can help with this. See also </span><a href="https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__TYPES.html#group__CUDART__TYPES_1g3f51e3575c2178246db0a94a430e0038" rel="nofollow" target="_blank" data-saferedirecturl="https://www.google.com/url?hl=en&q=https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__TYPES.html%23group__CUDART__TYPES_1g3f51e3575c2178246db0a94a430e0038&source=gmail&ust=1667316288466000&usg=AOvVaw0jL2wedVQUEoK8eewGRKAh">the CUDA docs</a>:<br></span></div><div><br></div><div><span><span> cudaErrorStubLibrary = <span>34</span></span></span>
<span>This indicates that the CUDA driver that the application has loaded is a stub library. Applications that run with the stub
rather than a real driver loaded will result in CUDA API returning this error. <br></span></div><div><span><br></span></div><div><span>-Ole<br></span></div><div><span><br></span></div><div class="gmail_quote"><div dir="auto" class="gmail_attr">On Sunday, October 30, 2022 at 9:59:11 PM UTC+1 Amit Gupta wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Hi,</div><div>I compiled a CUDA compatible version of CP2K, but when I submit my jobs, I get the following error:</div><div>--------------------------------------------------------------------------<br>ERROR: cudaErrorStubLibrary /scratch/ag9288/softwares/cp2k-2022.2/src/offload/offload_library.c 49<br>forrtl: error (76): Abort trap signal<br>Image PC Routine Line Source <br>cp2k.psmp 0000000006B83CAB Unknown Unknown Unknown<br>libpthread-2.28.s 000014D73245EB20 Unknown Unknown Unknown<br><a href="http://libc-2.28.so" rel="nofollow" target="_blank" data-saferedirecturl="https://www.google.com/url?hl=en&q=http://libc-2.28.so&source=gmail&ust=1667316288466000&usg=AOvVaw0Rms4BiUUpZ6ax97iWz42o">libc-2.28.so</a> 000014D712D4037F gsignal Unknown Unknown<br><a href="http://libc-2.28.so" rel="nofollow" target="_blank" data-saferedirecturl="https://www.google.com/url?hl=en&q=http://libc-2.28.so&source=gmail&ust=1667316288466000&usg=AOvVaw0Rms4BiUUpZ6ax97iWz42o">libc-2.28.so</a> 000014D712D2ADB5 abort Unknown Unknown<br>cp2k.psmp 00000000031F8C29 offload_get_devic 49 offload_library.c<br>cp2k.psmp 0000000001387F46 f77_interface_mp_ 276 f77_interface.F<br>cp2k.psmp 000000000042281D MAIN__ 284 cp2k.F<br>cp2k.psmp 0000000006BED096 Unknown Unknown Unknown<br><a href="http://libc-2.28.so" rel="nofollow" target="_blank" data-saferedirecturl="https://www.google.com/url?hl=en&q=http://libc-2.28.so&source=gmail&ust=1667316288466000&usg=AOvVaw0Rms4BiUUpZ6ax97iWz42o">libc-2.28.so</a> 000014D712D2C493 __libc_start_main Unknown Unknown<br>cp2k.psmp 000000000042195E Unknown Unknown Unknown<br>--------------------------------------------------------------------------</div><div><br></div><div>It looks like the executable is unable to find the card, while all the libraries are correctly loaded.</div><div><br></div><div>Version: <br></div><div>Intel fortran: 19.1</div><div>cuda : 11.3</div><div>openmpi 4.1.1</div><div>CP2K: 2022.2</div><div><br></div><div>Steps I took to build the binary:</div><div>1. Install the toolchain:</div><div>./install_cp2k_toolchain.sh
--with-intel=system --enable-cuda --gpu-ver=V100 --with-openmpi=system
--with-libxc=install --with-libint=install --with-fftw=install
--with-mkl=system --with-libxsmm=install --with-elpa=install
--with-ptscotch=no --with-superlu=no --with-pexsi=no
--with-quip=no --with-plumed=no --with-sirius=no
--with-gsl=no --with-libvdwxc=no --with-spglib=no
--with-hdf5=no --with-spfft=no --with-spla=install
--with-cosma=install --with-libvori=no --with-openblas=no -j4</div><div><br></div><div>2. Manualy build and copy Libint, for the fortran makefile bug:</div><div><a href="https://github.com/cp2k/libint-cp2k/releases" rel="nofollow" target="_blank" data-saferedirecturl="https://www.google.com/url?hl=en&q=https://github.com/cp2k/libint-cp2k/releases&source=gmail&ust=1667316288466000&usg=AOvVaw1ygkzXfYXxzh1kr3X1deSF">https://github.com/cp2k/libint-cp2k/releases</a><br>cmake
-DCMAKE_C_COMPILER=icc -DCMAKE_C_FLAGS="-qopenmp -O2"
-DCMAKE_CXX_COMPILER=icpc -DCMAKE_CXX_FLAGS="-qopenmp -O2"
-DCMAKE_Fortran_COMPILER=ifort -DCMAKE_Fortran_FLAGS="-qopenmp -O2"
-DREQUIRE_CXX_API=OFF -DENABLE_FORTRAN=ON ..</div><div><br></div><div>3. Apply required CUDA patch to CP2K and compile:</div><div><a href="https://github.com/cp2k/cp2k/commit/ee6c3aa.patch" rel="nofollow" target="_blank" data-saferedirecturl="https://www.google.com/url?hl=en&q=https://github.com/cp2k/cp2k/commit/ee6c3aa.patch&source=gmail&ust=1667316288467000&usg=AOvVaw3R0oTCzKiwJDtx0XrVd5sF">https://github.com/cp2k/cp2k/commit/ee6c3aa.patch</a></div><div> make -j 4 ARCH=local_cuda VERSION="ssmp sdbg psmp pdbg"</div><div><br></div><div>PS:
The job-node is different from the compile node, could that might be an
issue? Also while the compilation went fine, there was an error in
symlinking the binaries.<br></div></div>
</blockquote></div></blockquote></div></blockquote></div>
<p></p>
-- <br />
You received this message because you are subscribed to the Google Groups "cp2k" group.<br />
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="mailto:cp2k+unsubscribe@googlegroups.com">cp2k+unsubscribe@googlegroups.com</a>.<br />
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/cp2k/658a9011-fda3-4b70-a30d-9d3dd4ef2307n%40googlegroups.com?utm_medium=email&utm_source=footer">https://groups.google.com/d/msgid/cp2k/658a9011-fda3-4b70-a30d-9d3dd4ef2307n%40googlegroups.com</a>.<br />