<div dir="ltr">Thank you Alfio for your suggestion, I will try it in the way you advised.<br><br>在 2017年2月25日星期六 UTC-6上午12:26:40，Alfio Lazzaro写道：<blockquote class="gmail_quote" style="margin: 0;margin-left: 0.8ex;border-left: 1px #ccc solid;padding-left: 1ex;"><div dir="ltr">The CP2K output is fine. <div>Now, if you rely on top command to see how many threads you are running, that is somehow underestimated (it will show you the average CPU utilization). You can use the options in the second answer of this post <a href="http://stackoverflow.com/questions/15933801/openmp-creates-many-threads-but-seems-to-use-only-one-core" target="_blank" rel="nofollow" onmousedown="this.href='http://www.google.com/url?q\x3dhttp%3A%2F%2Fstackoverflow.com%2Fquestions%2F15933801%2Fopenmp-creates-many-threads-but-seems-to-use-only-one-core\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHvbhqEWmJY_TSPNf2cN8-PL6UUZQ';return true;" onclick="this.href='http://www.google.com/url?q\x3dhttp%3A%2F%2Fstackoverflow.com%2Fquestions%2F15933801%2Fopenmp-creates-many-threads-but-seems-to-use-only-one-core\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHvbhqEWmJY_TSPNf2cN8-PL6UUZQ';return true;">http://stackoverflow.com/<wbr>questions/15933801/openmp-<wbr>creates-many-threads-but-<wbr>seems-to-use-only-one-core</a> to disaggregate all the running threads.</div><div>Another test is to set less number of threads (for example OMP_NUM_THREADS=4) and see how it goes.</div><div>In any case, as I mentioned in my previous email, for such large number of cores you should consider to run the PSMP version, i.e. MPI+OpenMP (your 40 cores are using different NUMA domains, so you should set at least one MPI rank per NUMA domain).</div><div><br></div><div>Alfio<br><br>Il giorno venerdì 24 febbraio 2017 17:06:49 UTC+1, William Tao ha scritto:<blockquote class="gmail_quote" style="margin:0;margin-left:0.8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Dear Alfio,<div><br></div><div>I did set up the OMP_NUM_THREADS as 40 for my calculation. And here is part of my output in the very beginning.</div><div><br></div><div><div> GLOBAL| Force Environment number 1</div><div> GLOBAL| Basis set file name HFX_BASIS</div><div> GLOBAL| Potential file name GTH_POTENTIALS</div><div> GLOBAL| MM Potential file name MM_POTENTIAL</div><div> GLOBAL| Coordinate file name <a href="http://atm309.xyz" rel="nofollow" target="_blank" onmousedown="this.href='http://www.google.com/url?q\x3dhttp%3A%2F%2Fatm309.xyz\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEaH5wHXbg0PbhQcg_iZK2fjXcNhQ';return true;" onclick="this.href='http://www.google.com/url?q\x3dhttp%3A%2F%2Fatm309.xyz\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEaH5wHXbg0PbhQcg_iZK2fjXcNhQ';return true;">atm309.xyz</a></div><div> GLOBAL| Method name CP2K</div><div> GLOBAL| Project name ATOM-309</div><div> GLOBAL| Preferred FFT library FFTW3</div><div> GLOBAL| Preferred diagonalization lib. SL</div><div> GLOBAL| Run type MD</div><div> GLOBAL| All-to-all communication in single precision F</div><div> GLOBAL| FFTs using library dependent lengths F</div><div> GLOBAL| Global print level LOW</div><div> GLOBAL| Total number of message passing processes 1</div><div><b> GLOBAL| Number of threads for this process 40</b></div><div> GLOBAL| This output is from process 0</div><div> GLOBAL| CPU model name : Intel(R) Xeon(R) CPU E7-8891 v3 @ 2.80GHz</div></div><div><br></div><div><br></div><div>However, when running, it could only use up to 8 threads.</div><div><br></div><div>Thank you.</div><div><br></div><div>William</div><div><br></div><div><br><br>在 2017年2月24日星期五 UTC-6上午1:47:37，Alfio Lazzaro写道：<blockquote class="gmail_quote" style="margin:0;margin-left:0.8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Dear William,<div>CP2K uses OpenMP for the threading. The number of threads is set by using the environment variable OMP_NUM_THREADS and it goes at runtime, therefore there is no relation with the compilation (you can specify the number of threads at runtime). By default, if you don't set the variable, OpenMP assumes the maximum number of available threads on the system (8 and 40 in your case). That's why you see 40 threads. However, you should consider that it is very hard to get good scalability for such large number of threads (actually it depends on your workload), that's why in "average" (what you see from top command) you are using 8 corresponding fully loaded threads.</div><div><br></div><div>I can suggest to experiment with setting a different number of threads. Just use:</div><div><br></div><div>export OMP_NUM_THREADS=<any number between 1 and 40></div><div><br></div><div>before running CP2K.</div><div><br></div><div>Another better solution would be to use MPI and OpenMP (psmp version), by using 4 MPI ranks and 10 threads (for example). Likely it would give you better performance...</div><div><br></div><div>Alfio</div><div><br></div><div><br><br>Il giorno giovedì 23 febbraio 2017 22:48:01 UTC+1, William Tao ha scritto:<blockquote class="gmail_quote" style="margin:0;margin-left:0.8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Dear friends,</div><div><br></div>I compiled the cp2k version 4.1 on a 8-core CPU machine.<div>When I run the binary cp2k.ssmp on another machine with 40-core CPU, the output prints that the number of threads is 40.</div><div>However, when I check with "top" command, the process could use up to 800% CPU.</div><div><br></div><div>Does anybody know what is going on?</div><div><br></div><div><br></div><div>William </div><div><br></div></div></blockquote></div></div></blockquote></div></div></blockquote></div></div></blockquote></div>