<div dir="ltr"><br>Hi,<br><br><blockquote class="gmail_quote" style="margin: 0;margin-left: 0.8ex;border-left: 1px #ccc solid;padding-left: 1ex;"><div lang="DE" link="blue" vlink="purple"><div>
<p class="MsoNormal"><span> </span></p>
<p class="MsoNormal"><span lang="EN-GB">You can try to launch the jobs with I_MPI_DEBUG=5</span></p></div></div></blockquote><div>I 've tried that and you can find the debug output attached. In the MPI_startup() section it seems that each process is placed correctly on each core and no oversubscription or anything like that is observed. However I'm not sure what the last line means (I_MPI_PIN_MAPPING). Could you please guide me through how to make CP2K with IntelMPI scale across multiple nodes? Or is the performance as expected? I launch the executable by:</div><div><p style="font-stretch: normal; font-size: 10px; line-height: normal; font-family: Monaco; color: rgb(46, 174, 187); background-color: rgba(0, 0, 0, 0.85098);"><span style="font-variant-ligatures: no-common-ligatures; color: #c1651c">export</span><span style="font-variant-ligatures: no-common-ligatures"> OMP_NUM_THREADS=</span><span style="font-variant-ligatures: no-common-ligatures; color: #b42419">1</span></p>
<p style="font-stretch: normal; font-size: 10px; line-height: normal; font-family: Monaco; color: rgb(46, 174, 187); background-color: rgba(0, 0, 0, 0.85098);"><span style="font-variant-ligatures: no-common-ligatures; color: #c1651c">export</span><span style="font-variant-ligatures: no-common-ligatures"> I_MPI_PIN_DOMAIN=</span><span style="font-variant-ligatures: no-common-ligatures; color: #f2f2f2">core</span></p>
<p style="font-stretch: normal; font-size: 10px; line-height: normal; font-family: Monaco; color: rgb(46, 174, 187); background-color: rgba(0, 0, 0, 0.85098);"><span style="font-variant-ligatures: no-common-ligatures; color: #c1651c">export</span><span style="font-variant-ligatures: no-common-ligatures"> I_MPI_DEBUG=</span><span style="font-variant-ligatures: no-common-ligatures; color: #b42419">5</span></p>
<p style="font-stretch: normal; font-size: 10px; line-height: normal; font-family: Monaco; color: rgb(242, 242, 242); background-color: rgba(0, 0, 0, 0.85098);"><span style="font-variant-ligatures: no-common-ligatures">mpirun </span><span style="font-variant-ligatures: no-common-ligatures; color: #c814c9">-n</span><span style="font-variant-ligatures: no-common-ligatures"> </span><span style="font-variant-ligatures: no-common-ligatures; color: #b42419">144</span><span style="font-variant-ligatures: no-common-ligatures"> </span><span style="font-variant-ligatures: no-common-ligatures; color: #c814c9">-ppn</span><span style="font-variant-ligatures: no-common-ligatures"> </span><span style="font-variant-ligatures: no-common-ligatures; color: #b42419">36</span><span style="font-variant-ligatures: no-common-ligatures"> </span><span style="font-variant-ligatures: no-common-ligatures; color: #c814c9">-genv</span><span style="font-variant-ligatures: no-common-ligatures"> </span><span style="font-variant-ligatures: no-common-ligatures; color: #2eaebb">I_MPI_DEBUG</span><span style="font-variant-ligatures: no-common-ligatures">=</span><span style="font-variant-ligatures: no-common-ligatures; color: #b42419">5</span><span style="font-variant-ligatures: no-common-ligatures"> </span><span style="font-variant-ligatures: no-common-ligatures; color: #c814c9">-genv</span><span style="font-variant-ligatures: no-common-ligatures"> </span><span style="font-variant-ligatures: no-common-ligatures; color: #2eaebb">I_MPI_PIN_DOMAIN</span><span style="font-variant-ligatures: no-common-ligatures">=core </span><span style="font-variant-ligatures: no-common-ligatures; color: #c814c9">-genv</span><span style="font-variant-ligatures: no-common-ligatures"> </span><span style="font-variant-ligatures: no-common-ligatures; color: #2eaebb">OMP_NUM_THREADS</span><span style="font-variant-ligatures: no-common-ligatures">=</span><span style="font-variant-ligatures: no-common-ligatures; color: #b42419">1</span><span style="font-variant-ligatures: no-common-ligatures"> </span><span style="font-variant-ligatures: no-common-ligatures; color: #c814c9">-print-rank-map</span><span style="font-variant-ligatures: no-common-ligatures"> /lustre/home/d167/s1887443/scc/cp2k/exe/broadwell-o2-libs/cp2k.psmp </span><span style="font-variant-ligatures: no-common-ligatures; color: #c814c9">-i</span><span style="font-variant-ligatures: no-common-ligatures"> H2O-64.inp </span><span style="font-variant-ligatures: no-common-ligatures; color: #c814c9">-o</span><span style="font-variant-ligatures: no-common-ligatures"> out.txt</span></p>
<p style="font-stretch: normal; font-size: 10px; line-height: normal; font-family: Monaco; color: rgb(74, 0, 255); background-color: rgba(0, 0, 0, 0.85098);"><span style="font-variant-ligatures: no-common-ligatures">~ </span></p></div><div><br></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin: 0;margin-left: 0.8ex;border-left: 1px #ccc solid;padding-left: 1ex;"><div lang="DE" link="blue" vlink="purple"><div><p class="MsoNormal"><span lang="EN-GB">MKL should include FFTW3 and thus there
is no need for a separate installation of that lib.</span></p>
<p class="MsoNormal"><span lang="EN-GB"> </span></p></div></div></blockquote><div>Will the choice of FFTW3 affect the scaling performance though?</div><div><br></div><div>Best, </div><div>Chris</div></div>