<div>Hi Eric,</div><div><br /></div><div>Thanks for the response. I went and tested this out as you suggested by your example. I have nodes with 56 processors each and I tested using two nodes comparing with time steps I'm getting for one node. I tested the following settings:</div><div><br /></div><div>1 MPI 112 Threads<br /></div><div>2 MPI 56 Threads</div><div>4 MPI 28 Threads</div><div>8 MPI 14 Threads</div><div>16 MPI 7 Threads</div><div><br /></div><div>I end up getting the best performance at and above 4 MPI processes and higher but this is still slower than simply using one node with 56 MPI processes. Thanks for the suggestion though. <br /></div><div><br /></div><div>I'm not an experienced user with CP2K but just using an input file my colleague gave me to test out the installation that I'm helping out with. I would expect that a system of 100 water molecules and a single Au atom would still see some acceleration by using two nodes but I might be wrong. Any other thoughts?<br /></div><br /><div class="gmail_quote"><div dir="auto" class="gmail_attr">On Tuesday, April 11, 2023 at 11:31:14 AM UTC-7 Eric Patterson wrote:<br/></div><blockquote class="gmail_quote" style="margin: 0 0 0 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;"><div style="line-break:after-white-space">Hello Nathan,<div><br></div><div>What are your settings for MPI processes and OMP threads? On the machine I’m using (an older Intel machine with 48 physical cores per node and Omni-Path interconnect), I found good multi-node performance with 12 MPI processes per node and 4 OMP threads per process. Assigning all cores to MPI was nearly 3x slower and often resulted in memory issues. I did quite a bit of testing to come up with this configuration.</div><div><br></div><div>I imagine this could depend heavily on the type of job (mine are periodic cell optimizations and vibrational analysis, nothing fancy), so I definitely recommend doing some testing to see what works for you.</div><div><br></div><div>Cheers,</div><div>Eric</div><div><br><div><br><blockquote type="cite"></blockquote></div></div></div><div style="line-break:after-white-space"><div><div><blockquote type="cite"><div>On Apr 11, 2023, at 2:19 PM, Nathan Keilbart <<a href data-email-masked rel="nofollow">nathank...@gmail.com</a>> wrote:</div><br></blockquote></div></div></div><div style="line-break:after-white-space"><div><div><blockquote type="cite"><div><div>Hello,</div><div><br></div><div>I recently finished compiling on an intel based hpc machine and appear to have a working binary. I initially tested a case with a single node job and got what appears to be relatively good times. Upon increasing the number of nodes to two, I actually saw in increase in time per step instead of less. The nodes are connected with infiniband I believe so there shouldn't be an issue with node to node communication. I'm wondering if I set some flag wrong when compiling and what I should look into to find out what's going on here. Let me know what kind of information I can provide. Thanks</div><div><br>Nathan <br></div><div><br></div></div></blockquote></div></div></div><div style="line-break:after-white-space"><div><div><blockquote type="cite"><div>
-- <br>
You received this message because you are subscribed to the Google Groups "cp2k" group.<br>
To unsubscribe from this group and stop receiving emails from it, send an email to <a href data-email-masked rel="nofollow">cp2k+uns...@googlegroups.com</a>.<br>
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/cp2k/d5c11154-82e8-4c09-9713-9d43f9eb4b7bn%40googlegroups.com?utm_medium=email&utm_source=footer" target="_blank" rel="nofollow" data-saferedirecturl="https://www.google.com/url?hl=en&q=https://groups.google.com/d/msgid/cp2k/d5c11154-82e8-4c09-9713-9d43f9eb4b7bn%2540googlegroups.com?utm_medium%3Demail%26utm_source%3Dfooter&source=gmail&ust=1681423263474000&usg=AOvVaw0Efl9kfYPZend-wJK_qJls">https://groups.google.com/d/msgid/cp2k/d5c11154-82e8-4c09-9713-9d43f9eb4b7bn%40googlegroups.com</a>.<br>
</div></blockquote></div><br></div></div></blockquote></div>
<p></p>
-- <br />
You received this message because you are subscribed to the Google Groups "cp2k" group.<br />
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="mailto:cp2k+unsubscribe@googlegroups.com">cp2k+unsubscribe@googlegroups.com</a>.<br />
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/cp2k/b9dde686-4391-46fe-9f46-4c0f55c30ab1n%40googlegroups.com?utm_medium=email&utm_source=footer">https://groups.google.com/d/msgid/cp2k/b9dde686-4391-46fe-9f46-4c0f55c30ab1n%40googlegroups.com</a>.<br />