[CP2K-user] [CP2K:18594] Re: Install issues with IBM Power9 processors with Nvidia V100 GPU

[Alfio Lazzaro]

There is still something wrong in your local_cuda.psmp file.
In your output above I cannot find the flag `-D__parallel` . Isee only the 
followings:

-D__OFFLOAD_CUDA -D__DBCSR_ACC   -D__FFTW3  -D__LIBINT -D__LIBXC 
-D__SCALAPACK -D__COSMA -D__ELPA -D__ELPA_NVIDIA_GPU -D__GSL -D__HDF5 
-D__LIBVDWXC -D__SPGLIB -D__LIBVORI -D__SPFFT  -D__OFFLOAD_GEMM  -D__SPLA 
-D__SIRIUS    -D__CUDA

So my guess is that the toolchain was not able to recognize MPI (no idea 
why). Could you add -D__parallel on top of those flags?

Il giorno venerdì 31 marzo 2023 alle 00:08:29 UTC+2 Nathan Keilbart ha 
scritto:

> Thank Alfio. I wasn't sure what file was controlling that. I updated the 
> file to have those compilers and then did a make realclean. Afterwards, I 
> am now getting this error:
>
> /usr/gapps/qsg/codes/cp2k/lassen/v2023.1/src/fm/cp_blacs_env.F:192:19:
>
>              gcd_max = -1
>                    1
> Error: Symbol 'gcd_max' at (1) has no IMPLICIT type
> /usr/gapps/qsg/codes/cp2k/lassen/v2023.1/src/fm/cp_blacs_env.F:193:18:
>
>              DO ipe = 1, CEILING(SQRT(REAL(npe, dp)))
>                   1
> Error: Symbol 'ipe' at (1) has no IMPLICIT type
> /usr/gapps/qsg/codes/cp2k/lassen/v2023.1/src/fm/cp_blacs_env.F:194:18:
>
>                 jpe = npe/ipe
>                   1
> Error: Symbol 'jpe' at (1) has no IMPLICIT type
> /usr/gapps/qsg/codes/cp2k/lassen/v2023.1/src/fm/cp_blacs_env.F:185:29:
>
>           my_blacs_grid_layout = BLACS_GRID_SQUARE
>                              1
> Error: Symbol 'my_blacs_grid_layout' at (1) has no IMPLICIT type; did you 
> mean 'blacs_grid_layout'?
> /usr/gapps/qsg/codes/cp2k/lassen/v2023.1/src/fm/cp_blacs_env.F:221:25:
>
>        my_blacs_repeatable = .FALSE.
>                          1
> Error: Symbol 'my_blacs_repeatable' at (1) has no IMPLICIT type; did you 
> mean 'blacs_repeatable'?
> /usr/gapps/qsg/codes/cp2k/lassen/v2023.1/src/fm/cp_blacs_env.F:213:18:
>
>        my_row_major = .TRUE.
>                   1
> Error: Symbol 'my_row_major' at (1) has no IMPLICIT type; did you mean 
> 'row_major'?
> /usr/gapps/qsg/codes/cp2k/lassen/v2023.1/src/fm/cp_blacs_env.F:174:11:
>
>        npcol = 1
>            1
> Error: Symbol 'npcol' at (1) has no IMPLICIT type; did you mean 'ipcol'?
> /usr/gapps/qsg/codes/cp2k/lassen/v2023.1/src/fm/cp_blacs_env.F:175:9:
>
>        npe = blacs_env%n_pid
>          1
> Error: Symbol 'npe' at (1) has no IMPLICIT type
> /usr/gapps/qsg/codes/cp2k/lassen/v2023.1/src/fm/cp_blacs_env.F:173:11:
>
>        nprow = 1
>            1
> Error: Symbol 'nprow' at (1) has no IMPLICIT type; did you mean 'iprow'?
> /usr/gapps/qsg/codes/cp2k/lassen/v2023.1/src/fm/cp_blacs_env.F:188:22:
>
>           SELECT CASE (my_blacs_grid_layout)
>                       1
> Error: Argument of SELECT statement at (1) cannot be UNKNOWN
> make[3]: *** [/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/Makefile:519: 
> cp_blacs_env.o] Error 1
> make[2]: *** [/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/Makefile:146: all] 
> Error 2
>
> make[1]: *** [/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/Makefile:128: psmp] 
> Error 2
> make: *** [Makefile:123: all] Error 2
>
> On Thursday, March 30, 2023 at 12:22:43 AM UTC-7 Alfio Lazzaro wrote:
>
>> There is no relation with the DBCSR compilation itself, you see a problem 
>> in DBCSR simply because it is the first to compile in CP2K.
>> The error message is:
>>
>> /bin/sh: c: command not found
>>
>> and indeed you are using the command
>>
>> c -fno-omit-frame-pointer -fopenmp -g -mtune=native  -O3 -funroll-loops  
>>   ...
>>
>> for compiling, therefore there is something wrong in the compiler call.
>> I think the problem is that the local_cuda.psmp file has something wrong 
>> in the definition of the compilers, namely the lines
>>
>> CC             := mpicc
>> FC             := mpif90
>> LD             := mpif90
>> AR             := ar -r
>>
>> could you check if they are linking to the rights commands?
>>
>>
>>
>>
>> Il giorno giovedì 30 marzo 2023 alle 03:12:26 UTC+2 Nathan Keilbart ha 
>> scritto:
>>
>>> Hello everyone,
>>>
>>> I've been working on installing CP2K on a system with IBM Power9 
>>> processors and Nvidia V100 GPUs. I'm using the toolchain with these options:
>>>
>>> ./install_cp2k_toolchain.sh -j --with-cmake=system --mpi-mode=openmpi 
>>> --enable-cuda --gpu-ver=V100
>>>
>>> It installs all the dependencies without any errors so that I copy over 
>>> the files to the arch folder and then source the setup file followed by
>>>
>>> make -j ARCH=local_cuda VERSION=psmp
>>>
>>> The following is some of the last lines of output
>>>
>>> /usr/bin/env python3 
>>> /usr/gapps/qsg/codes/cp2k/lassen/v2023.1/exts/dbcsr/tools/build_utils/fypp/bin/fypp 
>>> -n --line-marker-format=gfortran5 
>>> /usr/gapps/qsg/codes/cp2k/lassen/v2023.1/exts/dbcsr/src/tensors/dbcsr_tensor_test.F 
>>> dbcsr_tensor_test.F90
>>> c -fno-omit-frame-pointer -fopenmp -g -mtune=native  -O3 -funroll-loops 
>>>   
>>>  -I'/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/tools/toolchain/install/openblas-0.3.21/include' 
>>> -I'/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/tools/toolchain/install/fftw-3.3.10/include' 
>>> -I'/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/tools/toolchain/install/libint-v2.6.0-cp2k-lmax-5/include' 
>>> -I'/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/tools/toolchain/install/libxc-6.0.0/include' 
>>> -I'/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/tools/toolchain/install/COSMA-2.6.2/include' 
>>> -I'/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/tools/toolchain/install/elpa-2022.11.001/nvidia/include/elpa_openmp-2022.11.001/modules' 
>>> -I'/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/tools/toolchain/install/elpa-2022.11.001/nvidia/include/elpa_openmp-2022.11.001/elpa' 
>>> -I'/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/tools/toolchain/install/gsl-2.7/include' 
>>> -I/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/tools/toolchain/install/hdf5-1.12.0/include 
>>> -I/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/tools/toolchain/install/libvdwxc-0.4.0/include 
>>> -I/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/tools/toolchain/install/spglib-1.16.2/include 
>>> -I'/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/tools/toolchain/install/SpFFT-1.0.6/include' 
>>> -I'/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/tools/toolchain/install/SpLA-1.5.4/include/spla' 
>>> -I/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/tools/toolchain/install/sirius-7.3.2/include/cuda 
>>> -fbacktrace -ffree-form -fimplicit-none -std=f2008  -Werror=aliasing 
>>> -Werror=ampersand -Werror=c-binding-type -Werror=intrinsic-shadow 
>>> -Werror=intrinsics-std -Werror=line-truncation -Werror=tabs 
>>> -Werror=target-lifetime -Werror=underflow -Werror=unused-but-set-variable 
>>> -Werror=unused-variable -Werror=unused-dummy-argument -Werror=conversion 
>>> -Werror=zerotrip -Wno-maybe-uninitialized -Wuninitialized 
>>> -Wuse-without-only  -D__OFFLOAD_CUDA -D__DBCSR_ACC   -D__FFTW3  -D__LIBINT 
>>> -D__LIBXC -D__SCALAPACK -D__COSMA -D__ELPA -D__ELPA_NVIDIA_GPU -D__GSL 
>>> -D__HDF5 -D__LIBVDWXC -D__SPGLIB -D__LIBVORI -D__SPFFT  -D__OFFLOAD_GEMM 
>>>  -D__SPLA -D__SIRIUS    -D__CUDA -D__SHORT_FILE__="\"dbcsr_tensor_test.F\"" 
>>> -I'/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/exts/dbcsr/src/tensors/' 
>>> -I'/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/exts/dbcsr/src' 
>>> dbcsr_tensor_test.F90 
>>> /bin/sh: c: command not found
>>> make[4]: 
>>> [/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/exts/build_dbcsr//Makefile:258: 
>>> dbcsr_tensor_test.o] Error 127 (ignored)
>>> /usr/bin/env python3 
>>> /usr/gapps/qsg/codes/cp2k/lassen/v2023.1/exts/dbcsr/tools/build_utils/fypp/bin/fypp 
>>> -n --line-marker-format=gfortran5 
>>> /usr/gapps/qsg/codes/cp2k/lassen/v2023.1/exts/dbcsr/src/tensors/dbcsr_tensor_api.F 
>>> dbcsr_tensor_api.F90
>>> c -fno-omit-frame-pointer -fopenmp -g -mtune=native  -O3 -funroll-loops 
>>>   
>>>  -I'/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/tools/toolchain/install/openblas-0.3.21/include' 
>>> -I'/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/tools/toolchain/install/fftw-3.3.10/include' 
>>> -I'/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/tools/toolchain/install/libint-v2.6.0-cp2k-lmax-5/include' 
>>> -I'/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/tools/toolchain/install/libxc-6.0.0/include' 
>>> -I'/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/tools/toolchain/install/COSMA-2.6.2/include' 
>>> -I'/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/tools/toolchain/install/elpa-2022.11.001/nvidia/include/elpa_openmp-2022.11.001/modules' 
>>> -I'/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/tools/toolchain/install/elpa-2022.11.001/nvidia/include/elpa_openmp-2022.11.001/elpa' 
>>> -I'/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/tools/toolchain/install/gsl-2.7/include' 
>>> -I/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/tools/toolchain/install/hdf5-1.12.0/include 
>>> -I/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/tools/toolchain/install/libvdwxc-0.4.0/include 
>>> -I/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/tools/toolchain/install/spglib-1.16.2/include 
>>> -I'/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/tools/toolchain/install/SpFFT-1.0.6/include' 
>>> -I'/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/tools/toolchain/install/SpLA-1.5.4/include/spla' 
>>> -I/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/tools/toolchain/install/sirius-7.3.2/include/cuda 
>>> -fbacktrace -ffree-form -fimplicit-none -std=f2008  -Werror=aliasing 
>>> -Werror=ampersand -Werror=c-binding-type -Werror=intrinsic-shadow 
>>> -Werror=intrinsics-std -Werror=line-truncation -Werror=tabs 
>>> -Werror=target-lifetime -Werror=underflow -Werror=unused-but-set-variable 
>>> -Werror=unused-variable -Werror=unused-dummy-argument -Werror=conversion 
>>> -Werror=zerotrip -Wno-maybe-uninitialized -Wuninitialized 
>>> -Wuse-without-only  -D__OFFLOAD_CUDA -D__DBCSR_ACC   -D__FFTW3  -D__LIBINT 
>>> -D__LIBXC -D__SCALAPACK -D__COSMA -D__ELPA -D__ELPA_NVIDIA_GPU -D__GSL 
>>> -D__HDF5 -D__LIBVDWXC -D__SPGLIB -D__LIBVORI -D__SPFFT  -D__OFFLOAD_GEMM 
>>>  -D__SPLA -D__SIRIUS    -D__CUDA -D__SHORT_FILE__="\"dbcsr_tensor_api.F\"" 
>>> -I'/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/exts/dbcsr/src/tensors/' 
>>> -I'/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/exts/dbcsr/src' 
>>> dbcsr_tensor_api.F90 
>>> /bin/sh: c: command not found
>>> make[4]: 
>>> [/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/exts/build_dbcsr//Makefile:258: 
>>> dbcsr_tensor_api.o] Error 127 (ignored)
>>> Updating archive 
>>> /usr/gapps/qsg/codes/cp2k/lassen/v2023.1/lib/local_cuda/psmp/exts/dbcsr/libdbcsr.a
>>> ar: creating 
>>> /usr/gapps/qsg/codes/cp2k/lassen/v2023.1/lib/local_cuda/psmp/exts/dbcsr/libdbcsr.a
>>> ar: dbcsr_cuda_profiling.o: No such file or directory
>>> make[4]: *** 
>>> [/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/exts/build_dbcsr//Makefile:330: 
>>> /usr/gapps/qsg/codes/cp2k/lassen/v2023.1/lib/local_cuda/psmp/exts/dbcsr/libdbcsr.a] 
>>> Error 1
>>> make[3]: *** 
>>> [/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/exts/build_dbcsr/Makefile:179: 
>>> libdbcsr] Error 2
>>> make[2]: *** 
>>> [/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/exts/Makefile.inc:38: dbcsr] 
>>> Error 2
>>> make[1]: *** [/usr/gapps/qsg/codes/cp2k/lassen/v2023.1/Makefile:128: 
>>> psmp] Error 2
>>> make: *** [Makefile:123: all] Error 2
>>>
>>> It seems that it is having issues with the DBCSR module. I initially had 
>>> an issue with this because I seemed to have left off the --recursive option 
>>> and after making sure my git clone had that it at least let me build most 
>>> of the serial version. It at least gave me the cp2k.sopt binary and it 
>>> seems to at least take inputs. I didn't have a chance to test it too much 
>>> yet. When I got this binary I had done 
>>>
>>> make -j ARCH=local_cuda VERSION="ssmp sdbg psmp pdbg"
>>>
>>> as suggested.
>>>
>>> Also, I've attempted to install with spack by using
>>>
>>> spack install 
>>> cp2k at 2023.1+cosma+cuda+elpa+libint+libxc+mpi+openmp+pexsi+plumed+sirius+spglib 
>>> smm=blas cuda_arch=70
>>>
>>> These are some of the last lines of output
>>>
>>>  >> 4028    collect2: error: ld returned 1 exit status
>>>   >> 4029    collect2: error: ld returned 1 exit status
>>>   >> 4030    make[3]: *** 
>>> [/tmp/keilbart/spack-stage/spack-stage-cp2k-2023.1-24dhoyt24tbnn4d423glgoeqqquibmb6/spack-src/obj/linux-rhel7-power9le-gcc/psmp/
>>>              all.dep:178: 
>>> /tmp/keilbart/spack-stage/spack-stage-cp2k-2023.1-24dhoyt24tbnn4d423glgoeqqquibmb6/spack-src/exe/linux-rhel7-power9le-gcc/cp2k.p
>>>              smp] Error 1
>>>      4031    make[3]: *** Waiting for unfinished jobs....
>>>   >> 4032    make[3]: *** 
>>> [/tmp/keilbart/spack-stage/spack-stage-cp2k-2023.1-24dhoyt24tbnn4d423glgoeqqquibmb6/spack-src/obj/linux-rhel7-power9le-gcc/psmp/
>>>              all.dep:194: 
>>> /tmp/keilbart/spack-stage/spack-stage-cp2k-2023.1-24dhoyt24tbnn4d423glgoeqqquibmb6/spack-src/exe/linux-rhel7-power9le-gcc/libcp2
>>>              k_unittest.psmp] Error 1
>>>   >> 4033    make[2]: *** 
>>> [/tmp/keilbart/spack-stage/spack-stage-cp2k-2023.1-24dhoyt24tbnn4d423glgoeqqquibmb6/spack-src/Makefile:146: 
>>> all] Error 2
>>>   >> 4034    make[1]: *** 
>>> [/tmp/keilbart/spack-stage/spack-stage-cp2k-2023.1-24dhoyt24tbnn4d423glgoeqqquibmb6/spack-src/Makefile:128: 
>>> psmp] Error 2
>>>   >> 4035    make: *** [Makefile:123: all] Error 2
>>>
>>> Finally, I also have some intel machines that I'm attempting to build on 
>>> and having issues as well but we can start with the IBM machine as we're 
>>> hoping to accelerate the simulations with the GPU.
>>>
>>> Please let me know what other information I can provide. Thank you.
>>>
>>> Nathan
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups "cp2k" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cp2k+unsubscribe at googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cp2k/fdc0d095-3de8-4fa5-a6c1-0d4d48af1a7cn%40googlegroups.com.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.cp2k.org/archives/cp2k-user/attachments/20230330/1bb8a890/attachment-0001.htm>