[CP2K-user] [CP2K:16922] Re: CP2K scaling with Intel ONEAPI MPI + ethernet

[hf.p...@gmail.com]

Like abin suggested, try using an HPC fabric such as InfiniBand or 
Omnipath. The main point here is about latency is too high over Ethernet 
for the communication patterns common in MD applications. Note, it is 
mostly about latency and not the transfer bandwidth (though bw is most 
advertised like "100G" or 200/400G). I conducted similar experiments in the 
past with CP2K and QE, and strong-scaling performance diminished quickly 
(often two nodes already needed a longer time to solution when compared to 
a single node/system). Of course, this depends on the workload but for 
instance reductions and all2all comms are the bottleneck in general.

Tat schrieb am Montag, 31. Januar 2022 um 17:37:03 UTC+1:

> Dear all,
> we are trying to improve the suboptimal scaling of CP2K we're experiencing 
> on a linux cluster with several physical nodes: the execution on 2 or more 
> nodes appears to be significantly slower than on a single one. 
> The system has nodes with 32-core Xeon Silver processors with 
> hyperthreading, Gigabit ethernet and the execution is done according to the 
> parameters provided by the plan.sh script, i.e.
>
> for 1 node:
> *mpirun -np 16 -genv I_MPI_PIN_DOMAIN=auto -genv I_MPI_PIN_ORDER=bunch 
> -genv OMP_PLACES=threads -genv OMP_PROC_BIND=SPREAD -genv OMP_NUM_THREADS=4 
> ~/cp2k-8.2/exe/Linux-x86-64-intelx/cp2k.psmp job.inp*
>
> for 2 nodes:
>
> *mpirun -r ssh -perhost 16 -host linux1,linux2 -genv I_MPI_PIN_DOMAIN=auto 
> -genv I_MPI_PIN_ORDER=bunch -genv OMP_PLACES=threads -genv 
> OMP_PROC_BIND=SPREAD -genv OMP_NUM_THREADS=4 
> ~/cp2k-8.2/exe/Linux-x86-64-intelx/cp2k.psmp job.inp*
>
> CP2K PSMP was compiled using Intel ONEAPI mpiifort 2021.3.0.
>
> What could be done to improve the performance? Can network communication 
> or SSH cause the bottleneck? 
> Any suggestions or references would be much appreciated.
> Thanks &regards,
>
> Attila
>

-- 
You received this message because you are subscribed to the Google Groups "cp2k" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cp2k+unsubscribe at googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cp2k/87c0b0de-885c-4695-add8-99188b807a60n%40googlegroups.com.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.cp2k.org/archives/cp2k-user/attachments/20220507/dc50cbdb/attachment-0001.htm>