<div dir="ltr">Hi Frank,<div>well, you can set a given affinity mask (by using KMP_AFFINITY if you are using the intel compiler) for a single job, while it doesn't work across multiple jobs... </div><div>In any case, tiny phase is not so important to get the best result, but this is not the case for the small phase where you really want to use a single core per job.</div><div>Is there any specific reason why you cannot use parallel compilation on the cluster?</div><div><br></div><div>Alfio</div><div><br>Il giorno lunedì 15 giugno 2015 14:23:13 UTC+2, Frank Uhlig ha scritto:<blockquote class="gmail_quote" style="margin: 0;margin-left: 0.8ex;border-left: 1px #ccc solid;padding-left: 1ex;"><div dir="ltr"><div><div><div><div><div>Hi Alfio, <br><br></div>I agree that it is not the best choice. I am running it anyhow right now and will compare it to 'something more reasonable'.<br></div>Because what I don't see, and maybe I am not involved in it enough, how running it in the background on a sufficient number of processors will give you different results from the hypothetical <br><br></div>./generate -c ... -j 0 -t 40 tiny1<br><br></div><div>for let's say a local machine with 40 cores (and if it would work).<br><br></div>Best,<br><br></div>Frank<br></div><div><br><div class="gmail_quote">On Mon, Jun 15, 2015 at 2:06 PM, Alfio Lazzaro <span dir="ltr"><<a href="javascript:" target="_blank" gdf-obfuscated-mailto="_X_z4SYl-uMJ" rel="nofollow" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;">alfi...@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Frank,<div>running the jobs in background is not a good idea, indeed they will run on the same node and clearly you will have wrong performance results from tiny phase (multiple jobs on the same core).</div><div><br></div><div>I'm glad that my solution works (l can add it to the SVN repository).</div><div><br></div><div>Best regards,</div><div><br></div><div>Alfio</div><div><br></div><div><br></div><div><span><br><br>Il giorno lunedì 15 giugno 2015 10:19:47 UTC+2, Frank Uhlig ha scritto:</span><blockquote class="gmail_quote" style="margin:0;margin-left:0.8ex;border-left:1px #ccc solid;padding-left:1ex"><span><div dir="ltr"><div><div><div>Hi Rolf and Alfio,<br><br></div>pity that the ulimit version did not work. Then it might be some other limit, but I am not sure right now.<br><br>I like Alfio's idea of the local submission. It runs in sequential and I think the reason is the following. The for loop around line 127 in the generate.bash file will always wait until the execution of the following command in the run_make function (line 136)<br><br>${run_cmd} make -j ${ntasks} -f ../${make_file} ${target} SI=${element_start} EI=${element_end}<br><br></div>has finished and won't actually run in parallel. This is no problem with queuing systems, because the submit command will always exit, but the job will run (in the queuing system). This is not the case with the current local version.<br><br></div>So you could change the command to run in the background that seems to work for me right now. Like the following:<br><br><div>batch_cmd() {</div><div>$@ &<br></div><div>}<br><br></div><div>Best,<br><br></div><div>Frank<br></div><br><br><div><div><div><br></div></div></div></div><br></span><div class="gmail_quote"><div><div><div dir="ltr">On Sat, Jun 13, 2015 at 3:06 PM Alfio Lazzaro <<a rel="nofollow">alfi...@gmail.com</a>> wrote:<br></div></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div><div dir="ltr">Hi Rolf,<div>the problem is that the list of arguments is too long (there are 13824 entries, each one with at least 10 chars) <br><div>Could you run the test in parallel? I mean you can use "-j #". In this case you have also to specify the wlm by using "-w " flag (pbs/slurm or a new one for your system). Please see the README for the parallel execution steps.</div><div>If you don't want to run on the cluster, you can still use the parallel execution on the login node, but then you have to declare an "empty" wlm file under config directory:</div><div><br></div><div>> cat no.wlm</div><div>batch_cmd() {</div><div>$@</div><div>}</div><div> </div><div>Therefore you can use "-j 100 -w no".</div><div>Note that I have never tried such a case, so I'm not sure it will work out-of-the-box. Let me how it goes.</div><div><br></div><div>Cheers,</div><div><br></div><div>Alfio</div></div></div><div dir="ltr"><div><div> </div><div><br><br>Il giorno venerdì 12 giugno 2015 20:26:47 UTC+2, Rolf David ha scritto:<blockquote class="gmail_quote" style="margin:0;margin-left:0.8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi,<div><br></div><div>The advices worked well. I'm now error free.</div><div><br></div><div>I've moved on compiling libsmm and libgrid. <div><br></div><div>Now i've run in another problem, the generation of libsmm.</div><div><br></div><div>The tiny part : </div><div><br></div><div>
<p>./generate -c config/linux.intel -j 0 -t 16 tiny1</p><p><br></p></div><div style="border:1px solid rgb(187,187,187);word-wrap:break-word;background-color:rgb(250,250,250)"><code><div><span style="color:#606">Generate</span><span style="color:#000"> master file output_linux</span><span style="color:#660">.</span><span style="color:#000">intel</span><span style="color:#660">/</span><span style="color:#000">tiny_find_<wbr>1_1_1__24_24_24</span><span style="color:#660">.</span><span style="color:#000">f90 <br><br>make</span><span style="color:#660">:</span><span style="color:#000"> execvp</span><span style="color:#660">:</span><span style="color:#000"> </span><span style="color:#080">/bin/</span><span style="color:#000">sh</span><span style="color:#660">:</span><span style="color:#000"> </span><span style="color:#606">Argument</span><span style="color:#000"> list too </span><span style="color:#008">long</span><span style="color:#000"> <br><br>make</span><span style="color:#660">:</span><span style="color:#000"> </span><span style="color:#660">***</span><span style="color:#000"> </span><span style="color:#660">[</span><span style="color:#000">output_linux</span><span style="color:#660">.</span><span style="color:#000">intel</span><span style="color:#660">/</span><span style="color:#000">tiny_find_<wbr>1_1_1__24_24_24</span><span style="color:#660">.</span><span style="color:#000">f90</span><span style="color:#660">]</span><span style="color:#000"> </span><span style="color:#606">Error</span><span style="color:#000"> </span><span style="color:#066">127</span></div></code></div><p><br></p><p><br></p><p>The problem comes from the command by verbosing:</p><p><br></p><p>make -j 16 -f ../Makefile.tiny_dnn_linux.<wbr>intel all SI=1 EI=13824</p><p><br></p><p>and ifort died </p><p><br></p><p>if anobody has any idea of what I'm doing wrong, I'll be glad (and relieved !)</p><p><br></p><p>Kinds regards</p><p><br></p><p>Rolf</p>
</div></div></blockquote></div></div></div>
<p></p>
-- <br>
You received this message because you are subscribed to the Google Groups "cp2k" group.<br></div></div>
To unsubscribe from this group and stop receiving emails from it, send an email to <a rel="nofollow">cp2k+...@googlegroups.com</a>.<br>
To post to this group, send email to <a rel="nofollow">cp...@googlegroups.com</a>.<span><br>
Visit this group at <a href="http://groups.google.com/group/cp2k" rel="nofollow" target="_blank" onmousedown="this.href='http://groups.google.com/group/cp2k';return true;" onclick="this.href='http://groups.google.com/group/cp2k';return true;">http://groups.google.com/<wbr>group/cp2k</a>.<br>
For more options, visit <a href="https://groups.google.com/d/optout" rel="nofollow" target="_blank" onmousedown="this.href='https://groups.google.com/d/optout';return true;" onclick="this.href='https://groups.google.com/d/optout';return true;">https://groups.google.com/d/<wbr>optout</a>.<br>
</span></blockquote></div>
</blockquote></div></div><div><div>
<p></p>
-- <br>
You received this message because you are subscribed to the Google Groups "cp2k" group.<br>
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="_X_z4SYl-uMJ" rel="nofollow" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;">cp2k+uns...@googlegroups.<wbr>com</a>.<br>
To post to this group, send email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="_X_z4SYl-uMJ" rel="nofollow" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;">cp...@googlegroups.com</a>.<br>
Visit this group at <a href="http://groups.google.com/group/cp2k" target="_blank" rel="nofollow" onmousedown="this.href='http://groups.google.com/group/cp2k';return true;" onclick="this.href='http://groups.google.com/group/cp2k';return true;">http://groups.google.com/<wbr>group/cp2k</a>.<br>
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href='https://groups.google.com/d/optout';return true;" onclick="this.href='https://groups.google.com/d/optout';return true;">https://groups.google.com/d/<wbr>optout</a>.<br>
</div></div></blockquote></div><br></div>
</blockquote></div></div>