Dear all,<div> I am doing geometry optimization of a system containing 576 atoms. When i ran it a machine having intel <span style="font-family: arial, sans-serif; font-size: 12.499999046325684px; ">X5460 processor, i didn't see any oscillations in total energies. It converges within some steps. But, the same system shows oscillations on different machine (intel </span><font face="arial, sans-serif"><span style="font-size: 12.666666984558105px;">X5670 processor). </span></font></div><div><font face="arial, sans-serif"><span style="font-size: 12.666666984558105px;"><br></span></font></div><div><font face="arial, sans-serif"><span style="font-size: 12.666666984558105px;"><b>Total energy :<br></b></span></font><div><br></div><div><div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.857268930485588</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.857593977155830</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.856003472735210</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.857548468864024</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.854062773488295</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.857594769232037</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.854626921172439</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.857481194438606</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.852779045098487</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.857273394239655</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.851571809396773</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.856224749560624</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.846230635967004</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.841851463337662</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.847479397051302</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.831469604048834</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.842533189635105</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.826210614017327</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.837802791426839</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.809268982869980</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.832563770153229</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.794880845741773</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.819193957763218</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.745714916703946</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.820245638992219</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.648271216216926</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.811590467374117</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.573351584592274</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.807336936256434</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.548983575820785</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.804035882624248</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.454113226169284</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.798586604405500</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.406644262456666</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.795520566227879</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.400278104366862</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.804283443121221</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.585113812539021</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.813855948710625</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.532866042457499</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.817657287906968</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.571135385605430</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.819301508552599</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.565533391461031</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.817356171838128</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.594185149152509</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.816696580745429</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.537045230232707</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.813409116061393</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.549188327705451</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.806053104601233</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.659599977649577</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.810634872865649</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.540129332885954</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.808556044454690</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.522990522475538</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.821163895395784</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.541664381568353</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.811375715670692</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.514473791798991</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.801293038222866</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.521168700564431</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.806579234558285</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.456123853706231</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.811449793531210</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.576983119674424</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.816395686705164</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.550635824096389</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.815701719725894</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.489804006631857</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.819771494755969</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.566217729472100</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.802585890487080</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.508774924785939</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.800921016420943</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.544680224335480</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.822501357248711</div><div> ENERGY| Total FORCE_EVAL ( QS ) energy (a.u.): -3895.651280964678790</div></div></div></div><div><br></div><div><br></div><div><b>Makefile :</b></div><div><br></div><div><br></div><div><div>INTEL_INC=/opt/intel/mkl/10.2.0.013/include</div><div>INTEL_LIB=/opt/intel/mkl/10.2.0.013/lib/em64t/</div><div>FFT_INC=/opt/intel/mkl/10.2.0.013/include/fftw</div><div><br></div><div><br></div><div>CC = cc</div><div>CPP = </div><div>FC = mpiifort</div><div>LD = mpiifort</div><div>AR = ar -r</div><div>DFLAGS = -D__INTEL -D__FFTSG -D__parallel -D__BLACS -D__SCALAPACK -D__FFTW3</div><div>CPPFLAGS = </div><div>FCFLAGS = $(DFLAGS) -I$(INTEL_INC) -I$(FFT_INC) -I/opt/intel/impi/4.0.2/include64 -O3 -msse2 -heap-arrays 64 -funroll-loops -fpp -free</div><div>FCFLAGS2 = $(DFLAGS) -I$(INTEL_INC) -I$(FFT_INC) -O1 -msse2 -heap-arrays 64 -fpp -free</div><div>LDFLAGS = $(FCFLAGS) -I$(INTEL_INC)</div><div>LIBS = -L$(MKLROOT)/lib/intel64 $(MKLROOT)/lib/intel64/libmkl_blas95_lp64.a $(MKLROOT)/lib/intel64/libmkl_lapack95_lp64.a \</div><div><span class="Apple-tab-span" style="white-space:pre"> </span> -lmkl_scalapack_lp64 -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lmkl_blacs_intelmpi_lp64 -lpthread -lm $(INTEL_LIB)/libfftw3xf_intel.a </div><div><br></div><div></div><div><br></div><div>OBJECTS_ARCHITECTURE = machine_intel.o</div><div><br></div><div><br></div><div>graphcon.o: graphcon.F</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>$(FC) -c $(FCFLAGS2) $<</div></div><div><br></div><div>--------------------------------------------------------------------------</div><div><br></div><div>I used same Makefile on both architectures (Intel compilers 12.1). I tried with different optimization levels, but this too didn't help. And also, it's not scaling more than one node on this machine. (each node contains 12 processors).</div><div><br></div><div>Do anybody faced similar problem ? Any help in this regard is greatly appreciated.</div><div><br></div><div>Regards,<br>Sandeep</div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div>