<span><span><span>Hello, I am also participating in HPC. I would like to ask for some questions about running test files.</span></span></span> <br><br><div class="gmail_quote"><div dir="auto" class="gmail_attr">在2021年4月22日星期四 UTC+8 下午7:42:16<ASSIDUO Network> 写道：<br/></div><blockquote class="gmail_quote" style="margin: 0 0 0 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;"><div dir="ltr"><div dir="ltr"><div dir="ltr">I am using 30 threads now over 3 GPUs, so I used:</div><div dir="ltr"><br><div>export OMP_NUM_THREADS=10<br></div><div>mpiexec -n 3 cp2k.psmp -i gold50.inp -o gold50.out<br></div><div><br></div></div></div></div><br><div class="gmail_quote"></div><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Apr 22, 2021 at 1:34 PM Alfio Lazzaro <<a href data-email-masked rel="nofollow">al...@gmail.com</a>> wrote:<br></div></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Wait, I see you have 32 threads in total, so need to have 32/4 = 8 threads.<div>Please change</div><div><br></div><div>export OMP_NUM_THREADS=8<br><br></div><div class="gmail_quote"><div dir="auto" class="gmail_attr">Il giorno giovedì 22 aprile 2021 alle 13:27:59 UTC+2 ASSIDUO Network ha scritto:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Shall do. I already set it up, but it's in a long queue.</div><br><div class="gmail_quote"></div><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Apr 22, 2021 at 1:22 PM Alfio Lazzaro <<a rel="nofollow">al...@gmail.com</a>> wrote:<br></div></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Could you try what I suggested:<div><br></div><div><div>export OMP_NUM_THREADS=10</div><div>mpirun -np 4 ./cp2k.psmp -i gold.inp -o gold_pbc.out</div><br></div><div>Please check the corresponding log.</div><div><br></div><div>As I said above, you need an MPI rank per GPU and you told us that you have 4 GPUs, so you need 4 ranks (or multiple). With 10 you get unbalance.</div><div><br></div><div><br></div><div class="gmail_quote"><div dir="auto" class="gmail_attr">Il giorno giovedì 22 aprile 2021 alle 10:17:27 UTC+2 ASSIDUO Network ha scritto:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">Correction, he told me to use:<div><br></div><div><div>mpirun -np 10 cp2k.psmp -i gold.inp -o gold_pbc.out</div><div><br></div></div><div>but it didn't run correctly.</div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Apr 22, 2021 at 9:51 AM Lenard Carroll <<a rel="nofollow">len...@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">He suggested I try out:<div><div>mpirun -n 10 cp2k.psmp -i gold.inp -o gold_pbc.out</div><div><br></div><div>as he is hoping that will cause the 1 GPU to use 10 CPUs over the selected 4 GPUs.</div><div><br></div></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Apr 22, 2021 at 9:48 AM Alfio Lazzaro <<a rel="nofollow">al...@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi,<div>Your command to run CP2K doesn't mention MPI (mpirun, mpiexc, ...). Are you running with multiple ranks?</div><div><br></div><div>You can check those lines in the output:</div><div><br></div><div><div> GLOBAL| Total number of message passing processes 32</div><div> GLOBAL| Number of threads for this process 4</div></div><div><br></div><div>And check your numbers.</div><div>I can guess you have 1 rank and 40 threads.</div><div>To use 4 GPUs you need 4 ranks (and less threads per rank), i.e. something like</div><div><br></div><div>export OMP_NUM_THREADS=10</div><div>mpiexec -n 4 ./cp2k.psmp -i gold.inp -o gold_pbc.out</div><div><br></div><div>Please check with your sysadmin on how to run with multiple MPI ranks.</div><div><br></div><div>Hope it helps.</div><div><br></div><div>Alfio</div><div><br></div><div><br><br></div><div class="gmail_quote"><div dir="auto" class="gmail_attr">Il giorno mercoledì 21 aprile 2021 alle 09:26:53 UTC+2 ASSIDUO Network ha scritto:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">This is what my PBS file looks like:<div><br></div><div><div>#!/bin/bash</div><div>#PBS -P <PROJECT></div><div>#PBS -N <JOBNAME></div><div>#PBS -l select=1:ncpus=40:ngpus=4</div><div>#PBS -l walltime=08:00:00</div><div>#PBS -q gpu_4</div><div>#PBS -m be</div><div>#PBS -M none</div><div><br></div><div>module purge</div><div><span style="background-color:transparent">module load chpc/cp2k/8.1.0/cuda10.1/openmpi-4.0.0/gcc-7.3.0</span><br></div><div><span style="background-color:transparent">source $SETUP</span><br></div><div><span style="background-color:transparent">cd $PBS_O_WORKDIR</span><br></div><div><br></div><div>cp2k.psmp -i gold.inp -o gold_pbc.out</div><div>~ ~ </div></div></div></div><br><div class="gmail_quote"></div><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Apr 21, 2021 at 9:22 AM Alfio Lazzaro <<a rel="nofollow">al...@gmail.com</a>> wrote:<br></div></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">The way to use 4 GPUs per node is to use 4 MPI ranks. How many ranks are you using?<br><br><div class="gmail_quote"><div dir="auto" class="gmail_attr">Il giorno martedì 20 aprile 2021 alle 19:44:15 UTC+2 ASSIDUO Network ha scritto:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">I'm asking, since the administrator running my country's HPC is saying that although I'm requesting access to 4 GPUs, CP2K is only using 1. I checked the following output:<div><div> DBCSR| ACC: Number of devices/node 4<br></div><div><br></div><div>And it shows that CP2K is picking up 4 GPUs.</div><br></div><div class="gmail_quote"><div dir="auto" class="gmail_attr">On Tuesday, April 20, 2021 at 3:00:17 PM UTC+2 ASSIDUO Network wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">I currently have access to 4 GPUs to run an AIMD simulation, but only one of the GPUs are being used. Is there a way to use the other 3, and if so, can you tell me how to set it up with a PBS job?</blockquote></div></blockquote></div>
<p></p></blockquote></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
-- <br>
You received this message because you are subscribed to the Google Groups "cp2k" group.<br>
To unsubscribe from this group and stop receiving emails from it, send an email to <a rel="nofollow">cp...@googlegroups.com</a>.<br>
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/cp2k/70ba0fce-8636-4b75-940d-133ce4dbf0can%40googlegroups.com?utm_medium=email&utm_source=footer" rel="nofollow" target="_blank" data-saferedirecturl="https://www.google.com/url?hl=zh-CN&q=https://groups.google.com/d/msgid/cp2k/70ba0fce-8636-4b75-940d-133ce4dbf0can%2540googlegroups.com?utm_medium%3Demail%26utm_source%3Dfooter&source=gmail&ust=1619240859868000&usg=AFQjCNFC0pyEUJVk0pVmrz91bADIhjb7VQ">https://groups.google.com/d/msgid/cp2k/70ba0fce-8636-4b75-940d-133ce4dbf0can%40googlegroups.com</a>.<br>
</blockquote></div>
</blockquote></div>
<p></p>
-- <br>
You received this message because you are subscribed to the Google Groups "cp2k" group.<br>
To unsubscribe from this group and stop receiving emails from it, send an email to <a rel="nofollow">cp...@googlegroups.com</a>.<br>
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/cp2k/92e4f88d-fde8-4127-ab5f-0b98bbbba8ebn%40googlegroups.com?utm_medium=email&utm_source=footer" rel="nofollow" target="_blank" data-saferedirecturl="https://www.google.com/url?hl=zh-CN&q=https://groups.google.com/d/msgid/cp2k/92e4f88d-fde8-4127-ab5f-0b98bbbba8ebn%2540googlegroups.com?utm_medium%3Demail%26utm_source%3Dfooter&source=gmail&ust=1619240859868000&usg=AFQjCNEYPGMYUynxFV2riSfnhh1A3LO7AQ">https://groups.google.com/d/msgid/cp2k/92e4f88d-fde8-4127-ab5f-0b98bbbba8ebn%40googlegroups.com</a>.<br>
</blockquote></div>
</blockquote></div>
</blockquote></div>
<p></p>
-- <br>
You received this message because you are subscribed to the Google Groups "cp2k" group.<br>
To unsubscribe from this group and stop receiving emails from it, send an email to <a rel="nofollow">cp...@googlegroups.com</a>.<br></blockquote></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/cp2k/59a635d8-0f0c-4dc5-abaf-b8bbe3c18da5n%40googlegroups.com?utm_medium=email&utm_source=footer" rel="nofollow" target="_blank" data-saferedirecturl="https://www.google.com/url?hl=zh-CN&q=https://groups.google.com/d/msgid/cp2k/59a635d8-0f0c-4dc5-abaf-b8bbe3c18da5n%2540googlegroups.com?utm_medium%3Demail%26utm_source%3Dfooter&source=gmail&ust=1619240859868000&usg=AFQjCNG9ga-4kTsvsoTHVdNwIf_NEGmOrA">https://groups.google.com/d/msgid/cp2k/59a635d8-0f0c-4dc5-abaf-b8bbe3c18da5n%40googlegroups.com</a>.<br>
</blockquote></div>
</blockquote></div>
<p></p>
-- <br>
You received this message because you are subscribed to the Google Groups "cp2k" group.<br>
To unsubscribe from this group and stop receiving emails from it, send an email to <a href data-email-masked rel="nofollow">cp...@googlegroups.com</a>.<br></blockquote></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/cp2k/ec4efd81-6314-4ce7-b22c-148b362d2ba6n%40googlegroups.com?utm_medium=email&utm_source=footer" target="_blank" rel="nofollow" data-saferedirecturl="https://www.google.com/url?hl=zh-CN&q=https://groups.google.com/d/msgid/cp2k/ec4efd81-6314-4ce7-b22c-148b362d2ba6n%2540googlegroups.com?utm_medium%3Demail%26utm_source%3Dfooter&source=gmail&ust=1619240859868000&usg=AFQjCNEyFSRfxnFQENl-3HBHNoMGCKkPsQ">https://groups.google.com/d/msgid/cp2k/ec4efd81-6314-4ce7-b22c-148b362d2ba6n%40googlegroups.com</a>.<br>
</blockquote></div>
</blockquote></div>