<div dir="ltr">Ron,<div><br></div><div>we have recently compared the H2O benchmark results for different platforms in terms of their peak performace (the paper is attached, see fig.1). The "last step timings" have been considered.</div><div><br></div><div>I have put your data on the plot (attached) for comparison with the 64 H2O datasets.</div><div><br></div><div>If I use 4 flops/cycle/core then your "last step" data follow pretty well the results for our old K-100 cluster based on Intel Xeon X5670 and the data for Cray XT3 and Cray XT5.</div><div><br></div><div>However as I understand you have E5-2665 Xeon CPUs with AVX and give 8 flops/cycle/core.</div><div>For the corresponding Rpeak values your "last step" times are too high.</div><div>Perhaps I am missing here some details of your OpenMP tuning.</div><div><br></div><div>Vladimir</div><div><br></div><div><br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Mar 24, 2016 at 7:50 PM, Cohen, Ronald <span dir="ltr"><<a href="mailto:rco...@carnegiescience.edu" target="_blank">rco...@carnegiescience.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Attached are my benchmark results. Do you think this could be further improved? The machine is <span style="color:rgb(0,0,0);font-family:Arial,Helvetica,Verdana,'Bitstream Vera Sans',sans-serif;font-size:12px;line-height:18px">40 nodes (n001-n040) with 16proc/ node = 640 Intel® Xeon® E5 Cloud Ready 2.4 GHz Compute Processor Cores based on E5-2665
544GB DDR3 1600Mhz ECC REG System Memory 1GB Memory Per Compute Processor Core 4x FDR infiniband Mellanox. Best performance is for </span>
<span style="font-family:'Helvetica Neue';font-size:14px">4 threads 16 mpi processes and 4 nodes, speedup of ~32 .</span><div><font color="#000000" face="Arial, Helvetica, Verdana, Bitstream Vera Sans, sans-serif"><span style="font-size:12px;line-height:18px"><br></span></font></div><div><font color="#000000" face="Arial, Helvetica, Verdana, Bitstream Vera Sans, sans-serif"><span style="font-size:12px;line-height:18px">My archfile is </span></font></div><div><font color="#000000" face="Arial, Helvetica, Verdana, Bitstream Vera Sans, sans-serif"><span style="font-size:12px;line-height:18px"> </span></font></div><div><font color="#000000" face="Arial, Helvetica, Verdana, Bitstream Vera Sans, sans-serif"><div><span style="font-size:12px;line-height:18px">CC = gcc</span></div><div><span style="font-size:12px;line-height:18px">CPP =</span></div><div><span style="font-size:12px;line-height:18px">FC = mpif90</span></div><div><span style="font-size:12px;line-height:18px">LD = mpif90</span></div><div><span style="font-size:12px;line-height:18px">AR = ar -r</span></div><div><span style="font-size:12px;line-height:18px">PREFIX = /home/rcohen</span></div><div><span style="font-size:12px;line-height:18px">FFTW_INC = $(PREFIX)/include</span></div><div><span style="font-size:12px;line-height:18px">FFTW_LIB = $(PREFIX)/lib</span></div><div><span style="font-size:12px;line-height:18px">LIBINT_INC = $(PREFIX)/include</span></div><div><span style="font-size:12px;line-height:18px">LIBINT_LIB = $(PREFIX)/lib</span></div><div><span style="font-size:12px;line-height:18px">LIBXC_INC = $(PREFIX)/include</span></div><div><span style="font-size:12px;line-height:18px">LIBXC_LIB = $(PREFIX)/lib</span></div><div><span style="font-size:12px;line-height:18px">GCC_LIB = $(PREFIX)/gcc-trunk/lib</span></div><div><span style="font-size:12px;line-height:18px">GCC_LIB64 = $(PREFIX)/gcc-trunk/lib64</span></div><div><span style="font-size:12px;line-height:18px">GCC_INC = $(PREFIX)/gcc-trunk/include</span></div><div><span style="font-size:12px;line-height:18px">DFLAGS = -D__FFTW3 -D__LIBINT -D__LIBXC2\</span></div><div><span style="font-size:12px;line-height:18px"> -D__LIBINT_MAX_AM=7 -D__LIBDERIV_MAX_AM1=6 -D__MAX_CONTR=4\</span></div><div><span style="font-size:12px;line-height:18px"> -D__parallel -D__SCALAPACK -D__HAS_smm_dnn -D__ELPA3 </span></div><div><span style="font-size:12px;line-height:18px">CPPFLAGS =</span></div><div><span style="font-size:12px;line-height:18px">FCFLAGS = $(DFLAGS) -O2 -ffast-math -ffree-form -ffree-line-length-none\</span></div><div><span style="font-size:12px;line-height:18px"> -fopenmp -ftree-vectorize -funroll-loops\</span></div><div><span style="font-size:12px;line-height:18px"> -mtune=native \</span></div><div><span style="font-size:12px;line-height:18px"> -I$(FFTW_INC) -I$(LIBINT_INC) -I$(LIBXC_INC) -I$(MKLROOT)/include \</span></div><div><span style="font-size:12px;line-height:18px"> -I$(GCC_INC) -I$(PREFIX)/include/elpa_openmp-2015.11.001/modules</span></div><div><span style="font-size:12px;line-height:18px">LIBS = \</span></div><div><span style="font-size:12px;line-height:18px"> $(PREFIX)/lib/libscalapack.a $(PREFIX)/lib/libsmm_dnn_sandybridge-2015-11-10.a \</span></div><div><span style="font-size:12px;line-height:18px"> $(FFTW_LIB)/libfftw3.a\</span></div><div><span style="font-size:12px;line-height:18px"> $(FFTW_LIB)/libfftw3_threads.a\</span></div><div><span style="font-size:12px;line-height:18px"> $(LIBXC_LIB)/libxcf90.a\</span></div><div><span style="font-size:12px;line-height:18px"> $(LIBXC_LIB)/libxc.a\</span></div><div><span style="font-size:12px;line-height:18px"> $(PREFIX)/lib/liblapack.a $(PREFIX)/lib/libtmglib.a $(PREFIX)/lib/libgomp.a \</span></div><div><span style="font-size:12px;line-height:18px"> $(PREFIX)/lib/libderiv.a $(PREFIX)/lib/libint.a -lelpa_openmp -lgomp -lopenblas</span></div><div><span style="font-size:12px;line-height:18px">LDFLAGS = $(FCFLAGS) -L$(GCC_LIB64) -L$(GCC_LIB) -static-libgfortran -L$(PREFIX)/lib </span></div><div style="font-size:12px;line-height:18px"><br></div><div style="font-size:12px;line-height:18px">gcc is 6.0.0</div><div style="font-size:12px;line-height:18px">gfortran</div><div style="font-size:12px;line-height:18px">openmpi 1.10.2</div><div style="font-size:12px;line-height:18px">scalapack 2.0.2</div><div><span style="font-size:12px;line-height:18px">elpa-2015.11.001</span><br></div><div><span style="font-size:12px;line-height:18px">libint 1.1.5 (I tried libint 2.0.3 but seems to be missing derivs)</span></div><div><span style="font-size:12px;line-height:18px">libxc 2.2.2</span></div><div><span style="font-size:12px;line-height:18px">openblas xianyi-OpenBLAS-c679dd1</span></div><div><span style="font-size:12px;line-height:18px">mm_dnn_sandybridge-2015-11-10</span><span style="font-size:12px;line-height:18px"><br></span></div><div><span style="font-size:12px;line-height:18px">fftw 3.3.4</span></div><div><span style="font-size:12px;line-height:18px"><br></span></div><div><span style="font-size:12px;line-height:18px"><img src="cid:ii_153a9858f38a8bb9" alt="Inline image 1" width="362" height="218"><br></span></div><div><span style="font-size:12px;line-height:18px">Speedup total is for the whole benchmark, setup plus 30 timesteps.</span></div><div><span style="font-size:12px;line-height:18px">Speedup step is for the time for the last time step.</span></div><div><span style="font-size:12px;line-height:18px">64 Molecules H2O FPMD NVT within LDA</span></div><div style="font-size:12px;line-height:18px"><br></div></font><div><div>---<br>Ronald Cohen<br>Geophysical Laboratory<br>Carnegie Institution<br>5251 Broad Branch Rd., N.W.<br>Washington, D.C. 20015<br><a href="mailto:rco...@carnegiescience.edu" target="_blank">rco...@carnegiescience.edu</a><br>office: 202-478-8937<br>skype: ronaldcohen<br><a href="https://twitter.com/recohen3" target="_blank">https://twitter.com/recohen3</a><br><a href="https://www.linkedin.com/profile/view?id=163327727" target="_blank">https://www.linkedin.com/profile/view?id=163327727</a><span class="HOEnZb"><font color="#888888"><br></font></span></div></div><span class="HOEnZb"><font color="#888888"><div><br></div>
</font></span></div></div><span class="HOEnZb"><font color="#888888">
<p></p>
-- <br>
You received this message because you are subscribed to the Google Groups "cp2k" group.<br>
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="mailto:cp2k+uns...@googlegroups.com" target="_blank">cp2k+uns...@googlegroups.com</a>.<br>
To post to this group, send email to <a href="mailto:cp...@googlegroups.com" target="_blank">cp...@googlegroups.com</a>.<br>
Visit this group at <a href="https://groups.google.com/group/cp2k" target="_blank">https://groups.google.com/group/cp2k</a>.<br>
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank">https://groups.google.com/d/optout</a>.<br>
</font></span></blockquote></div><br></div>