[CP2K-user] [CP2K:18651] Multinode jobs have poor scaling

[Eric Patterson]

Hello Nathan,

What are your settings for MPI processes and OMP threads? On the machine I’m using (an older Intel machine with 48 physical cores per node and Omni-Path interconnect), I found good multi-node performance with 12 MPI processes per node and 4 OMP threads per process. Assigning all cores to MPI was nearly 3x slower and often resulted in memory issues. I did quite a bit of testing to come up with this configuration.

I imagine this could depend heavily on the type of job (mine are periodic cell optimizations and vibrational analysis, nothing fancy), so I definitely recommend doing some testing to see what works for you.

Cheers,
Eric


> On Apr 11, 2023, at 2:19 PM, Nathan Keilbart <nathankeilbart at gmail.com> wrote:
> 
> Hello,
> 
> I recently finished compiling on an intel based hpc machine and appear to have a working binary. I initially tested a case with a single node job and got what appears to be relatively good times. Upon increasing the number of nodes to two, I actually saw in increase in time per step instead of less. The nodes are connected with infiniband I believe so there shouldn't be an issue with node to node communication. I'm wondering if I set some flag wrong when compiling and what I should look into to find out what's going on here. Let me know what kind of information I can provide. Thanks
> 
> Nathan 
> 
> -- 
> You received this message because you are subscribed to the Google Groups "cp2k" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to cp2k+unsubscribe at googlegroups.com <mailto:cp2k+unsubscribe at googlegroups.com>.
> To view this discussion on the web visit https://groups.google.com/d/msgid/cp2k/d5c11154-82e8-4c09-9713-9d43f9eb4b7bn%40googlegroups.com <https://groups.google.com/d/msgid/cp2k/d5c11154-82e8-4c09-9713-9d43f9eb4b7bn%40googlegroups.com?utm_medium=email&utm_source=footer>.

-- 
You received this message because you are subscribed to the Google Groups "cp2k" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cp2k+unsubscribe at googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cp2k/5034EA71-3AB2-4498-861A-A9FB19B36BA8%40gmail.com.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.cp2k.org/archives/cp2k-user/attachments/20230411/6cc2dd0d/attachment.htm>