<div dir="ltr"><div>Hi, <br></div><div><br></div><div>I tested the provided docker. When running in the container it again works great, outside the container starting the job with mpirun -n works and sets the number of processes, but starting with srun (either through bash file or inline) produces the following error: <br></div><div><br></div><div>--------------------------------------------------------------------------<br>The application appears to have been direct launched using "srun",<br>but OMPI was not built with SLURM's PMI support and therefore cannot<br>execute. There are several options for building PMI support under<br>SLURM, depending upon the SLURM version you are using:<br><br> version 16.05 or later: you can use SLURM's PMIx support. This<br> requires that you configure and build SLURM --with-pmix.<br><br> Versions earlier than 16.05: you must use either SLURM's PMI-1 or<br> PMI-2 support. SLURM builds PMI-1 by default, or you can manually<br> install PMI-2. You must then build Open MPI using --with-pmi pointing<br> to the SLURM PMI library location.<br><br>Please configure as appropriate and try again.<br>--------------------------------------------------------------------------</div><div><br></div><div>So my guess would be that I should configure my version of OpenMPI to work with slurm? <br></div><div><br></div><div>thanks for all the help, Ava<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">V V čet., 19. okt. 2023 ob 12:46 je oseba Krack Matthias <<a href="mailto:matthias.krack@psi.ch">matthias.krack@psi.ch</a>> napisala:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="msg-3164332292538549827"> <div style="overflow-wrap: break-word;" lang="en-CH"> <div class="m_-3164332292538549827WordSection1"> <p class="MsoNormal"><span style="font-size:11pt" lang="DE-CH">Hi Ava<u></u><u></u></span></p> <p class="MsoNormal"><span style="font-size:11pt" lang="DE-CH"><u></u> <u></u></span></p> <p class="MsoNormal"><span style="font-size:11pt" lang="EN-US">Thanks for testing. Indeed, the .sif file does not seem to work with the host MPI properly. These .sif files were directly built with apptainer. This was a test and we will not provide such containers in the future any longer. Instead, I suggest the download of a cp2k production docker container. Could you test such a docker container with apptainer? You can download it with the command <u></u><u></u></span></p> <p class="MsoNormal" style="text-indent:36pt"><span style="font-size:11pt" lang="EN-US">apptainer pull docker://mkrack/cp2k:2023.2_mpich_generic_psmp<u></u><u></u></span></p> <p class="MsoNormal"><span style="font-size:11pt" lang="EN-US">or<u></u><u></u></span></p> <p class="MsoNormal" style="text-indent:36pt"><span style="font-size:11pt" lang="EN-US">apptainer pull docker://mkrack/cp2k:2023.2_openmpi_generic_psmp<u></u><u></u></span></p> <p class="MsoNormal"><span style="font-size:11pt" lang="EN-US">depending on your host MPI version.<u></u><u></u></span></p> <p class="MsoNormal"><span style="font-size:11pt" lang="EN-US">A list of the available docker containers can be found <a href="https://hub.docker.com/repository/docker/mkrack/cp2k/tags?page=1&ordering=last_updated" target="_blank"> here</a>.<u></u><u></u></span></p> <p class="MsoNormal"><span style="font-size:11pt" lang="EN-US"><u></u> <u></u></span></p> <p class="MsoNormal"><span style="font-size:11pt" lang="EN-US">Best<u></u><u></u></span></p> <p class="MsoNormal"><span style="font-size:11pt" lang="EN-US"><u></u> <u></u></span></p> <p class="MsoNormal"><span style="font-size:11pt" lang="EN-US">Matthias<u></u><u></u></span></p> <p class="MsoNormal"><span style="font-size:11pt" lang="EN-US"><u></u> <u></u></span></p> <div id="m_-3164332292538549827mail-editor-reference-message-container"> <div> <div style="border-color:rgb(181,196,223) currentcolor currentcolor;border-style:solid none none;border-width:1pt medium medium;padding:3pt 0cm 0cm"> <p class="MsoNormal" style="margin-right:0cm;margin-bottom:12pt;margin-left:36pt"> <b><span style="font-size:12pt;color:black">From: </span></b><span style="font-size:12pt;color:black"><a href="mailto:cp2k@googlegroups.com" target="_blank">cp2k@googlegroups.com</a> <<a href="mailto:cp2k@googlegroups.com" target="_blank">cp2k@googlegroups.com</a>> on behalf of Ava Rajh <<a href="mailto:ava.rajh@gmail.com" target="_blank">ava.rajh@gmail.com</a>><br> <b>Date: </b>Thursday, 19 October 2023 at 12:17<br> <b>To: </b>cp2k <<a href="mailto:cp2k@googlegroups.com" target="_blank">cp2k@googlegroups.com</a>><br> <b>Subject: </b>Re: [CP2K:19391] Help with running cp2k in parallel with Slurm<u></u><u></u></span></p> </div> <p class="MsoNormal" style="margin-left:36pt"><span style="font-size:11pt"><u></u> <u></u></span></p> <div> <p class="MsoNormal" style="margin-left:36pt"><span style="font-size:11pt">Sorry for the confusion, I read the output wrong, <u></u><u></u></span></p> </div> <div> <p class="MsoNormal" style="margin-left:36pt"><span style="font-size:11pt">and if I put the ntasks in the header of the batch file I get the same result as running just with srun, so the number of tasks determines the number of programs running with one MPI rank each. <u></u><u></u></span></p> </div> <div> <p class="MsoNormal" style="margin-left:36pt"><span style="font-size:11pt"><u></u> <u></u></span></p> </div> <div> <div> <p class="MsoNormal" style="margin-left:36pt"><span style="font-size:11pt">On Thursday, October 19, 2023 at 10:20:39 AM UTC+2 Ava Rajh wrote:<u></u><u></u></span></p> </div> <blockquote style="border-color:currentcolor currentcolor currentcolor rgb(204,204,204);border-style:none none none solid;border-width:medium medium medium 1pt;padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm"> <p class="MsoNormal" style="margin-left:36pt"><span style="font-size:11pt"><u></u> <u></u></span></p> <div> <p class="MsoNormal" style="margin-left:36pt"><span style="font-size:11pt">Hi Matthias, <u></u><u></u></span></p> </div> <div> <p class="MsoNormal" style="margin-left:36pt"><span style="font-size:11pt"><u></u> <u></u></span></p> </div> <div> <p class="MsoNormal" style="margin-left:36pt"><span style="font-size:11pt">Thank you for the reply<u></u><u></u></span></p> </div> <div> <p class="MsoNormal" style="margin-left:36pt"><span style="font-size:11pt"><u></u> <u></u></span></p> </div> <div> <p class="MsoNormal" style="margin-left:36pt"><span style="font-size:11pt">I tried both, first just with the srun command and as part of the #SBATCH file. <u></u><u></u></span></p> </div> <div> <p class="MsoNormal" style="margin-left:36pt"><span style="font-size:11pt">If I run the "srun" command and specify the number of tasks I get the before mentioned multiple copies of the same program with 1 MPI rank each. The same happens if I run the command outside the container without the srun and specify the number of tasks with just: mpiexec -n 4 <u></u><u></u></span></p> </div> <div> <p class="MsoNormal" style="margin-left:36pt"><span style="font-size:11pt"><u></u> <u></u></span></p> </div> <div> <p class="MsoNormal" style="margin-left:36pt"><span style="font-size:11pt">and if I put the ntasks in the header of the batch file, then I just get one copy of the program with 1 MPI rank, no matter how many -ntasks are defined. <u></u><u></u></span></p> </div> <div> <p class="MsoNormal" style="margin-left:36pt"><span style="font-size:11pt"><u></u> <u></u></span></p> </div> <div> <p class="MsoNormal" style="margin-left:36pt"><span style="font-size:11pt">Any idea about how to start my calculation would be very helpful, thank you. <u></u><u></u></span></p> </div> <div> <p class="MsoNormal" style="margin-left:36pt"><span style="font-size:11pt">kind regards, Ava<u></u><u></u></span></p> </div> <div> <p class="MsoNormal" style="margin-left:36pt"><span style="font-size:11pt"><u></u> <u></u></span></p> </div> <div> <p class="MsoNormal" style="margin-left:36pt"><span style="font-size:11pt"><u></u> <u></u></span></p> </div> <div> <div> <p class="MsoNormal" style="margin-left:36pt"><span style="font-size:11pt">On Wednesday, October 18, 2023 at 7:21:22 PM UTC+2 Krack Matthias wrote:<u></u><u></u></span></p> </div> <blockquote style="border-color:currentcolor currentcolor currentcolor rgb(204,204,204);border-style:none none none solid;border-width:medium medium medium 1pt;padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm"> <div> <div> <p class="MsoNormal" style="margin-left:36pt"> <span style="font-size:11pt" lang="DE-CH">Hi Ava</span><span style="font-size:11pt"><u></u><u></u></span></p> <p class="MsoNormal" style="margin-left:36pt"> <span style="font-size:11pt" lang="DE-CH"> </span><span style="font-size:11pt"><u></u><u></u></span></p> <p class="MsoNormal" style="margin-left:36pt"> <span style="font-size:11pt" lang="EN-US">Do you run the “srun” command as part of a SLURM batch job file with a #SBATCH header section or interactively?</span><span style="font-size:11pt"><u></u><u></u></span></p> <p class="MsoNormal" style="margin-left:36pt"> <span style="font-size:11pt" lang="EN-US">Your guess is right, the –ntasks flag defines the number of MPI ranks.</span><span style="font-size:11pt"><u></u><u></u></span></p> <p class="MsoNormal" style="margin-left:36pt"> <span style="font-size:11pt" lang="EN-US"> </span><span style="font-size:11pt"><u></u><u></u></span></p> <p class="MsoNormal" style="margin-left:36pt"> <span style="font-size:11pt" lang="EN-US">Best</span><span style="font-size:11pt"><u></u><u></u></span></p> <p class="MsoNormal" style="margin-left:36pt"> <span style="font-size:11pt" lang="EN-US"> </span><span style="font-size:11pt"><u></u><u></u></span></p> <p class="MsoNormal" style="margin-left:36pt"> <span style="font-size:11pt" lang="EN-US">Matthias</span><span style="font-size:11pt"><u></u><u></u></span></p> <p class="MsoNormal" style="margin-left:36pt"> <span style="font-size:11pt" lang="EN-US"> </span><span style="font-size:11pt"><u></u><u></u></span></p> </div> </div> <div> <div> <div> <div> <div style="border-color:rgb(181,196,223) currentcolor currentcolor;border-style:solid none none;border-width:1pt medium medium;padding:3pt 0cm 0cm"> <p class="MsoNormal" style="margin-bottom:12pt;margin-left:72pt"> <b><span style="font-size:12pt;color:black">From: </span></b><span style="font-size:12pt;color:black"><a href="mailto:cp...@googlegroups.com" target="_blank">cp...@googlegroups.com</a> <<a href="mailto:cp...@googlegroups.com" target="_blank">cp...@googlegroups.com</a>> on behalf of Ava Rajh <<a href="mailto:ava....@gmail.com" target="_blank">ava....@gmail.com</a>><br> <b>Date: </b>Wednesday, 18 October 2023 at 14:17<br> <b>To: </b>cp2k <<a href="mailto:cp...@googlegroups.com" target="_blank">cp...@googlegroups.com</a>><br> <b>Subject: </b>[CP2K:19379] Help with running cp2k in parallel with Slurm</span><span style="font-size:11pt"><u></u><u></u></span></p> </div> <p class="MsoNormal" style="margin-left:72pt"> <span style="font-size:11pt">Dear all, <br> <br> I am trying to run cp2k on our HPC cluster and I am new in doing any kind of parallel computing and work on a cluster, so I would appreciate if some help and I apologize if I am missing something obvious. <u></u><u></u></span></p> <div> <p class="MsoNormal" style="margin-left:72pt"> <span style="font-size:11pt">I am trying to use Cp2k in combination with apptainer and I followed instructions at <a href="https://github.com/cp2k/cp2k/tree/master/tools/apptainer" target="_blank"> https://github.com/cp2k/cp2k/tree/master/tools/apptainer</a> <u></u><u></u></span></p> </div> <div> <p class="MsoNormal" style="margin-left:72pt"> <span style="font-size:11pt"> <u></u><u></u></span></p> </div> <div> <p class="MsoNormal" style="margin-left:72pt"> <span style="font-size:11pt">I have a .sif file in my work directory and if I work within it (Running MPI within the container), everything works perfectly and I can set the number of MPI threads. <u></u><u></u></span></p> </div> <p class="MsoNormal" style="margin-left:72pt"> <span style="font-size:11pt"> <u></u><u></u></span></p> <div> <p class="MsoNormal" style="margin-left:72pt"> <span style="font-size:11pt">But when trying to run it through slurm, I can't seem to be able to set the number of MPI processes per node. If i start a command like: <u></u><u></u></span></p> </div> <div> <p class="MsoNormal" style="margin-left:72pt"> <span style="font-size:11pt">srun --ntasks=2 apptainer run -B $PWD cp2k-2023.2_mpich_generic_psmp.sif cp2k -i H2O-32.in<u></u><u></u></span></p> </div> <div> <p class="MsoNormal" style="margin-left:72pt"> <span style="font-size:11pt">it just starts 2 instances of the program that run at the same time, and for each, the total number of message passing processes is 1. I am able to set and change the number of OpenMP threads though. <u></u><u></u></span></p> </div> <div> <p class="MsoNormal" style="margin-left:72pt"> <span style="font-size:11pt">So my question would be first, am I wrongly assuming that --ntasks X should correspond to the number of MPI threads? And if I am, how would I set it. <u></u><u></u></span></p> </div> <div> <p class="MsoNormal" style="margin-left:72pt"> <span style="font-size:11pt"> <u></u><u></u></span></p> </div> <div> <p class="MsoNormal" style="margin-left:72pt"> <span style="font-size:11pt">Please let me know if I need to provide any more information to diagnose the issue. <u></u><u></u></span></p> </div> <div> <p class="MsoNormal" style="margin-left:72pt"> <span style="font-size:11pt">Thank you very much for the help and kind regards, <u></u><u></u></span></p> </div> <div> <p class="MsoNormal" style="margin-left:72pt"> <span style="font-size:11pt">Ava Rajh<u></u><u></u></span></p> </div> </div> </div> </div> </div> <div> <div> <div> <div> <p class="MsoNormal" style="margin-left:72pt"> <span style="font-size:11pt">-- <br> You received this message because you are subscribed to the Google Groups "cp2k" group.<br> To unsubscribe from this group and stop receiving emails from it, send an email to <a href="mailto:cp2k%2Buns...@googlegroups.com" target="_blank">cp2k+uns...@googlegroups.com</a>.<br> To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/cp2k/2393b350-7de0-4599-97e0-4f52ffec0c5en%40googlegroups.com?utm_medium=email&utm_source=footer" target="_blank"> https://groups.google.com/d/msgid/cp2k/2393b350-7de0-4599-97e0-4f52ffec0c5en%40googlegroups.com</a>.<u></u><u></u></span></p> </div> </div> </div> </div> </blockquote> </div> </blockquote> </div> <p class="MsoNormal" style="margin-left:36pt"><span style="font-size:11pt">-- <br> You received this message because you are subscribed to the Google Groups "cp2k" group.<br> To unsubscribe from this group and stop receiving emails from it, send an email to <a href="mailto:cp2k+unsubscribe@googlegroups.com" target="_blank">cp2k+unsubscribe@googlegroups.com</a>.<br> To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/cp2k/1a89bd22-860c-4922-a2b6-bd20232ca770n%40googlegroups.com?utm_medium=email&utm_source=footer" target="_blank"> https://groups.google.com/d/msgid/cp2k/1a89bd22-860c-4922-a2b6-bd20232ca770n%40googlegroups.com</a>.<u></u><u></u></span></p> </div> </div> </div> </div> <p></p> -- <br> You received this message because you are subscribed to a topic in the Google Groups "cp2k" group.<br> To unsubscribe from this topic, visit <a href="https://groups.google.com/d/topic/cp2k/or6vb6En6aU/unsubscribe" target="_blank">https://groups.google.com/d/topic/cp2k/or6vb6En6aU/unsubscribe</a>.<br> To unsubscribe from this group and all its topics, send an email to <a href="mailto:cp2k+unsubscribe@googlegroups.com" target="_blank">cp2k+unsubscribe@googlegroups.com</a>.<br> To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/cp2k/ZRAP278MB0827D6DA09823659200590A5F4D4A%40ZRAP278MB0827.CHEP278.PROD.OUTLOOK.COM?utm_medium=email&utm_source=footer" target="_blank">https://groups.google.com/d/msgid/cp2k/ZRAP278MB0827D6DA09823659200590A5F4D4A%40ZRAP278MB0827.CHEP278.PROD.OUTLOOK.COM</a>.<br> </div></blockquote></div> <p></p> -- <br /> You received this message because you are subscribed to the Google Groups "cp2k" group.<br /> To unsubscribe from this group and stop receiving emails from it, send an email to <a href="mailto:cp2k+unsubscribe@googlegroups.com">cp2k+unsubscribe@googlegroups.com</a>.<br /> To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/cp2k/CADBW8Us_6gVF8ABCEaUF%2B9VpK_84T8%3DNcU%2BLONq491L9DkwNVw%40mail.gmail.com?utm_medium=email&utm_source=footer">https://groups.google.com/d/msgid/cp2k/CADBW8Us_6gVF8ABCEaUF%2B9VpK_84T8%3DNcU%2BLONq491L9DkwNVw%40mail.gmail.com</a>.<br />