<div dir="ltr">Hi,<div><br></div><div>I find it strange the error you get. CP2K OMP+MPI works fine for me. Just for the sake of validation, could you try :</div><div><br></div><div>export OMP_NUM_THREADS=X</div><div><br></div><div>mpirun -np Y cp2k.psmp -i input > output</div><div><br></div><div>rather than</div><div><br></div><div> mpirun -np Y cp2k.psmp -i input -o output</div><div><br></div><div>Just to check.</div><div><br></div><div>Regards,</div><div>Pierre<br><br>On Monday, September 30, 2019 at 4:38:32 AM UTC+1, Chn wrote:<blockquote class="gmail_quote" style="margin: 0;margin-left: 0.8ex;border-left: 1px #ccc solid;padding-left: 1ex;"><div dir="ltr">Hi,<div>The keyword NPROC_REP has a default value of {1}, it should be only one output *-r.out file in a job. So can I explain that I got six files just because I used six threads..? Is it normal in a parallel job..?</div><div>regards,</div><div>chn<br><div><br>在 2019年9月29日星期日 UTC+8上午3:08:34，Ari Paavo Seitsonen写道：<blockquote class="gmail_quote" style="margin:0;margin-left:0.8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Hello,</div><div><br></div><div> Maybe you received the six files because of this:<br><br><<<br><table summary="keyword_description"><tbody><tr><td><br></td>
<td><code>NPROC_REP</code><code> {Integer}</code></td>
</tr>
<tr>
<td><br></td>
<td><em>Specify the number of processors to
be used per replica environment (for parallel runs). In case of mode
selective calculations more than one replica will start a block Davidson
algorithm to track more than only one frequency</em><span style="font-size:small">
[<a title="See corresponding source code location." href="https://github.com/cp2k/cp2k/blob/master/src/motion/input_cp2k_vib.F#L83" rel="nofollow" target="_blank" onmousedown="this.href='https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fcp2k%2Fcp2k%2Fblob%2Fmaster%2Fsrc%2Fmotion%2Finput_cp2k_vib.F%23L83\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHpg8oNpla4SGVAUko_H_Z-BTRCTw';return true;" onclick="this.href='https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fcp2k%2Fcp2k%2Fblob%2Fmaster%2Fsrc%2Fmotion%2Finput_cp2k_vib.F%23L83\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHpg8oNpla4SGVAUko_H_Z-BTRCTw';return true;">Edit on GitHub</a>]
</span></td>
</tr>
<tr>
<td><br></td>
<td>
This keyword cannot be repeated
and it expects
precisely one integer.
</td>
</tr>
<tr>
<td><br></td>
<td>
Default value:
<code>1</code></td></tr></tbody></table>>><br><br><a href="https://manual.cp2k.org/trunk/CP2K_INPUT/VIBRATIONAL_ANALYSIS.html#list_NPROC_REP" rel="nofollow" target="_blank" onmousedown="this.href='https://www.google.com/url?q\x3dhttps%3A%2F%2Fmanual.cp2k.org%2Ftrunk%2FCP2K_INPUT%2FVIBRATIONAL_ANALYSIS.html%23list_NPROC_REP\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFAN4vKZOeApdljqRjWwLKoeD8UZg';return true;" onclick="this.href='https://www.google.com/url?q\x3dhttps%3A%2F%2Fmanual.cp2k.org%2Ftrunk%2FCP2K_INPUT%2FVIBRATIONAL_ANALYSIS.html%23list_NPROC_REP\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFAN4vKZOeApdljqRjWwLKoeD8UZg';return true;">https://manual.cp2k.org/trunk/<wbr>CP2K_INPUT/VIBRATIONAL_<wbr>ANALYSIS.html#list_NPROC_REP</a></div><div><br></div><div> Greetings from Paris,</div><div><br></div><div> apsi<br></div></div><br><div class="gmail_quote"><div dir="ltr">Le sam. 28 sept. 2019 à 14:14, Pierre-André Cazade <<a rel="nofollow">pie...@gmail.com</a>> a écrit :<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<div>
<div>
<div>
<div style="direction:ltr">Hi Nikhil,</div>
<div><br>
</div>
<div style="direction:ltr">As you are using a mix of MPI and OpenMP, you have to use the executable with the extension psmp.</div>
<div><br>
</div>
<div style="direction:ltr">You can find a table describing all the executables in the section 3 of the “how to compile” page:</div>
<div><br>
</div>
<div style="direction:ltr"><a href="https://www.cp2k.org/howto:compile" rel="nofollow" target="_blank" onmousedown="this.href='https://www.google.com/url?q\x3dhttps%3A%2F%2Fwww.cp2k.org%2Fhowto%3Acompile\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEvDG80Mk-DHgzS0cFPRBYmvY2k9g';return true;" onclick="this.href='https://www.google.com/url?q\x3dhttps%3A%2F%2Fwww.cp2k.org%2Fhowto%3Acompile\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEvDG80Mk-DHgzS0cFPRBYmvY2k9g';return true;">https://www.cp2k.org/howto:<wbr>compile</a></div>
<div><br>
</div>
<div style="direction:ltr">Yet, it does not explain why your calculation behaved as if there were 6 independent calculations.</div>
<div><br>
</div>
<div style="direction:ltr">Please try the same calculation with the psmp executable and let me know how it goes.</div>
<div><br>
</div>
<div style="direction:ltr">Regards,</div>
<div style="direction:ltr">Pierre </div>
</div>
<div><br>
</div>
<div>Get <a href="https://aka.ms/o0ukef" rel="nofollow" target="_blank" onmousedown="this.href='https://www.google.com/url?q\x3dhttps%3A%2F%2Faka.ms%2Fo0ukef\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNF-OL6Ba_F90zc8M39phIhsSQZpDg';return true;" onclick="this.href='https://www.google.com/url?q\x3dhttps%3A%2F%2Faka.ms%2Fo0ukef\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNF-OL6Ba_F90zc8M39phIhsSQZpDg';return true;">Outlook for iOS</a></div>
</div>
<div> </div>
<hr style="display:inline-block;width:98%">
<div dir="dir="ltr""><font style="font-size:11pt" face="Calibri, sans-serif"><b>From:</b> <a rel="nofollow">c...@googlegroups.com</a> on behalf of Chn <<a rel="nofollow">ch...@gmail.com</a>><br>
<b>Sent:</b> Saturday, September 28, 2019 3:03 a.m.<br>
<b>To:</b> cp2k<br>
<b>Subject:</b> Re: [CP2K:12283] Running Cp2k in parallel using thread in a PC
<div> </div>
</font></div>
<div dir="ltr">Hi Pierre,
<div>I tried to combine openMP with MPI as you mentioned above when I do vibrational analysis. I required 6 MPI threads and got 6 output files named as *-r-number.out, however in each file it printed that:</div>
<div>
<div style="background-color:rgb(250,250,250);border-color:rgb(187,187,187);border-style:solid;border-width:1px">
<code>
<div>
<div> GLOBAL| Total number of message passing processes 1</div>
<div> GLOBAL| Number of threads for this process 1</div>
<div> GLOBAL| This output is from process 0</div>
</div>
</code></div>
also I used 'top' command and found that only 6 cores were busy. </div>
<div>I use 2 x 24 core processor, and set:</div>
<div>
<div style="background-color:rgb(250,250,250);border-color:rgb(187,187,187);border-style:solid;border-width:1px">
<code>
<div>
<div>export OMP_NUM_THREADS=8</div>
<div>mpirun -n 6 /lib/CP2K/cp2k/exe/local/cp2k.<wbr>popt -i project.inp -o output.out</div>
</div>
</code></div>
Any suggestion will be greatly appreciated..<br>
<br>
<br>
<br>
在 2019年9月20日星期五 UTC+8下午10:45:55，Pierre Cazade写道：
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">Hello Nikhil,<br>
<br>
Withe command "mpirun -n 42 cp2k.pop -i inp.inp -o -out.out", you are requesting 42 MPI threads and not 42 OpenMP threads. MPI usually relies on replicated data which means that, for a poorly program software, it will request a total amount of memory which
the amount of memory required by a scalar execution times the number of threads. This can very quickly become problematic, in particular for QM calculations. OpenMP, however relies on shared memory, the data is normally not replicated but shared between threads
and therefore, in an ideal scenario, the amount of memory needed for 42 OpenMP threads is the same as a single one.<br>
<br>
This might explains why you calculation freezes. You are out of memory. On your workstation, you should only use the executable "cp2k.ssmp" which is the OpenMP version. Then you don't need the mpirun command:<br>
<br>
cp2k.ssmp -i inp.inp -o -out.out<br>
<br>
To control the number of OpenMP threads, set the env variable: OMP_NUM_THREADS, e.g. in bash, export OMP_NUM_THREADS=48<br>
<br>
Now, if you need to balance between MPI and OpenMP, you should use the executable named cp2k.psmp. Here is such an example:<br>
<br>
export OMP_NUM_THREADS=24<br>
mpirun -n 2 cp2k.psmp -i inp.inp -o -out.out<br>
<br>
In this example, I am requesting two MPI threads and each of them can use up to 24 OpenMP threads.<br>
<br>
Hope this clarifies things for you.<br>
<br>
Regards,<br>
Pierre<br>
<br>
<div>On 20/09/2019 14:09, Nikhil Maroli wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Dear all,
<div><br>
</div>
<div>I have installed all the versions of CP2K in my workstation with 2 x 12 core processor, total thread=48</div>
<div><br>
</div>
<div>I wanted to run cp2k in parallel using 42 threads, can anyone share the commands that i can use.</div>
<div><br>
</div>
<div>I have tried </div>
<div><br>
</div>
<div>mpirun -n 42 cp2k.pop -i inp.inp -o -out.out</div>
<div><br>
</div>
<div>After this command there is a rise in memory to 100 % and the whole system freezes. (i have 128GB ram).</div>
<div><br>
</div>
<div>Any suggestion will be greatly appreciated,</div>
</div>
-- <br>
You received this message because you are subscribed to the Google Groups "cp2k" group.<br>
To unsubscribe from this group and stop receiving emails from it, send an email to
<a rel="nofollow">c...@googlegroups.com</a>.<br>
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/cp2k/39284c57-f6eb-463e-81a6-3a123596a9f2%40googlegroups.com?utm_medium=email&utm_source=footer" rel="nofollow" target="_blank" onmousedown="this.href='https://groups.google.com/d/msgid/cp2k/39284c57-f6eb-463e-81a6-3a123596a9f2%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter';return true;" onclick="this.href='https://groups.google.com/d/msgid/cp2k/39284c57-f6eb-463e-81a6-3a123596a9f2%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter';return true;">
https://groups.google.com/d/<wbr>msgid/cp2k/39284c57-f6eb-463e-<wbr>81a6-3a123596a9f2%<wbr>40googlegroups.com</a>.<br>
</blockquote>
<br>
<pre cols="72">--
Dr Pierre Cazade, PhD
AD3-023, Bernal Institute,
University of Limerick,
Plassey Park Road,
Castletroy, co. Limerick,
Ireland</pre>
</div>
</blockquote>
</div>
</div>
<p></p>
-- <br>
You received this message because you are subscribed to the Google Groups "cp2k" group.<br>
To unsubscribe from this group and stop receiving emails from it, send an email to
<a rel="nofollow">c...@googlegroups.com</a>.<br>
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/cp2k/2d26c264-d73a-44de-af89-13ff0ac86f69%40googlegroups.com?utm_medium=email&utm_source=footer" rel="nofollow" target="_blank" onmousedown="this.href='https://groups.google.com/d/msgid/cp2k/2d26c264-d73a-44de-af89-13ff0ac86f69%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter';return true;" onclick="this.href='https://groups.google.com/d/msgid/cp2k/2d26c264-d73a-44de-af89-13ff0ac86f69%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter';return true;">
https://groups.google.com/d/<wbr>msgid/cp2k/2d26c264-d73a-44de-<wbr>af89-13ff0ac86f69%<wbr>40googlegroups.com</a>.<br>
</div>
</div>
<p></p>
-- <br>
You received this message because you are subscribed to the Google Groups "cp2k" group.<br>
To unsubscribe from this group and stop receiving emails from it, send an email to <a rel="nofollow">c...@googlegroups.com</a>.<br>
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/cp2k/AM6PR10MB26968C27CCEC14903050632EAE800%40AM6PR10MB2696.EURPRD10.PROD.OUTLOOK.COM?utm_medium=email&utm_source=footer" rel="nofollow" target="_blank" onmousedown="this.href='https://groups.google.com/d/msgid/cp2k/AM6PR10MB26968C27CCEC14903050632EAE800%40AM6PR10MB2696.EURPRD10.PROD.OUTLOOK.COM?utm_medium\x3demail\x26utm_source\x3dfooter';return true;" onclick="this.href='https://groups.google.com/d/msgid/cp2k/AM6PR10MB26968C27CCEC14903050632EAE800%40AM6PR10MB2696.EURPRD10.PROD.OUTLOOK.COM?utm_medium\x3demail\x26utm_source\x3dfooter';return true;">https://groups.google.com/d/<wbr>msgid/cp2k/<wbr>AM6PR10MB26968C27CCEC149030506<wbr>32EAE800%40AM6PR10MB2696.<wbr>EURPRD10.PROD.OUTLOOK.COM</a>.<br>
</blockquote></div><br clear="all"><br>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr">-=*=-=*=-=*=-=*=-=*=-=*=-=*=-=<wbr>*=-=*=-=*=-=*=-=*=-=*=-=*=-=*=<wbr>-=*=-=*=-=*=-=*=-<br> Ari Paavo Seitsonen / <a rel="nofollow">Ar...@iki.fi</a> / <a href="http://www.iki.fi/~apsi/" rel="nofollow" target="_blank" onmousedown="this.href='http://www.google.com/url?q\x3dhttp%3A%2F%2Fwww.iki.fi%2F~apsi%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNG0aHwDkzSRT8XFNp-VAYw9AsYKJg';return true;" onclick="this.href='http://www.google.com/url?q\x3dhttp%3A%2F%2Fwww.iki.fi%2F~apsi%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNG0aHwDkzSRT8XFNp-VAYw9AsYKJg';return true;">http://www.iki.fi/~apsi/</a><br> Ecole Normale Supérieure (ENS), Département de Chimie, Paris<br> Mobile (F) : +33 789 37 24 25 (CH) : +41 79 71 90 935<br></div></div></div></div></div></div>
</blockquote></div></div></div></blockquote></div></div>