<div dir="ltr"><div>Thank you for sharing the root cause! I had a similar issue recently with POPT (Thread number has changed), which was root-caused to an issue with GFortran (see https://github.com/cp2k/dbcsr/issues/217#issuecomment-519066844).<br></div><div><br></div><div> Hans<br></div><div><br></div><br>Am Freitag, 16. August 2019 14:16:33 UTC+2 schrieb Ronald Cohen:<blockquote class="gmail_quote" style="margin: 0;margin-left: 0.8ex;border-left: 1px #ccc solid;padding-left: 1ex;"><div style="word-wrap:break-word;line-break:after-white-space">I got it working —the problem was elpa after all. Doing enable-omp in the configure was not enough. I also needed to add -fopenmp to the FLAGS. Now psmp works<div>fine with threads.</div><div><br></div><div>Ron</div><div> <br><div> <div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word;line-break:after-white-space"><div style="color:rgb(0,0,0);font-family:Helvetica;font-style:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:-webkit-auto;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word;line-break:after-white-space"><div style="color:rgb(0,0,0);font-family:Helvetica;font-style:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:-webkit-auto;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word;line-break:after-white-space">---</div><div style="color:rgb(0,0,0);font-family:Helvetica;font-style:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:-webkit-auto;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word;line-break:after-white-space">Ronald Cohen</div><div style="color:rgb(0,0,0);font-family:Helvetica;font-style:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:-webkit-auto;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word;line-break:after-white-space">Extreme Materials Initiative<br>Geophysical Laboratory<br>Carnegie Institution<br>5251 Broad Branch Rd., N.W.<br>Washington, D.C. 20015<br><a href="javascript:" target="_blank" gdf-obfuscated-mailto="-c6JAqWQAQAJ" rel="nofollow" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;">r...@carnegiescience.edu</a><br>office: 202-478-8937<br>skype: ronaldcohen</div><div style="color:rgb(0,0,0);font-family:Helvetica;font-style:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:-webkit-auto;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word;line-break:after-white-space">twitter: <span style="text-align:-webkit-auto">@</span><span style="text-align:-webkit-auto">recohen3</span></div></div></div> </div> <div><br><blockquote type="cite"><div>On Aug 15, 2019, at 3:50 PM, Ronald Cohen <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="-c6JAqWQAQAJ" rel="nofollow" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;">re...@gmail.com</a>> wrote:</div><br><div><div dir="ltr"><div>This is for the release 6.1 tarball .The error occurs in dbcsr/dist/dbcsr_dist_method. I have rebuilt many times using different option but always get this</div><div>with the local.psmp arch. The code riuns fine with <br></div><div>export OMP_NUM_THREADS=1</div><div>but fails for >1 with:</div><div><br></div><div>******************************<wbr>******************************<wbr>*******************<br> * ___ <wbr> <wbr> *<br> * / \ <wbr> <wbr> *<br> * [ABORT] <wbr> <wbr> *<br> * \___/ Thread number has changed *<br> * | <wbr> <wbr> *<br> * O/| <wbr> <wbr> *<br> * /| | <wbr> <wbr> *<br> * / \ <wbr> dbcsr/dist/dbcsr_dist_methods.<wbr>F:515 *<br> *****************************<wbr>******************************<wbr>********************<br><br><br> ===== Routine Calling Stack ===== <br><br> 10 dbcsr_create_new<br> 9 build_overlap_matrix<br> 8 build_core_hamiltonian_matrix<br> 7 qs_energies_init_hamiltonians<br> 6 qs_energies<br> 5 qs_forces<br> 4 cp_eval_at<br> 3 geoopt_bfgs<br> 2 cp_geo_opt<br> 1 CP2K<br></div><div>It fails whether I use elpa or not.</div><div><br></div><div>My toolchain command is:</div><div><br></div><div> <div>./install_cp2k_toolchain.sh --with-cmake=install --with-libint=install --with-<wbr>libxc=install --with-elpa=/home/rcohen/gnu --enable-cuda=no --enable-omp=yes --with-gcc=install --with-binutils=install <br></div><div><br></div><div>and local.psmp is:</div><div><br></div><div>C = /home/rcohen/CP2K/cp2k-6.1.0/<wbr>tools/toolchain/install/gcc-7.<wbr>3.0/bin/gcc<br>CXX = /home/rcohen/CP2K/cp2k-6.1.0/<wbr>tools/toolchain/install/gcc-7.<wbr>3.0/bin/g++<br>CPP =<br>AR = ar -r<br>FC = mpif90<br>LD = mpif90<br>#<br>DFLAGS = -D__LIBXSMM -D__parallel -D__FFTW3 -D__LIBINT -D__LIBINT_MAX_AM=6 -D__LIBDERIV_MAX_AM1=5 -D__LIBXC -D__SCALAPACK -D__ELPA=201705 <br>#<br>WFLAGS = -Werror=aliasing -Werror=ampersand -Werror=c-binding-type -Werror=intrinsic-shadow -Werror=intrinsics-std -Werror=line-truncation -Werror=tabs -Werror=realloc-lhs-all -Werror=target-lifetime -Werror=underflow -Werror=unused-but-set-<wbr>variable -Werror=unused-variable -Werror=unused-dummy-argument -Werror=conversion -Werror=zerotrip -Werror=uninitialized -Wno-maybe-uninitialized -Wuse-without-only <br>#<br>FCDEBFLAGS = -ffree-form -std=f2003 -fimplicit-none <br>CFLAGS = -march=knl -fno-omit-frame-pointer -g -O3 -funroll-loops -ffast-math $(PROFOPT) -fopenmp -I'/home/rcohen/CP2K/cp2k-6.1.<wbr>0/tools/toolchain/install/<wbr>mpich-3.2.1/include' -m64 -I/mnt/beegfs/intel/compilers_<wbr>and_libraries_2019.4.243/<wbr>linux/mkl/include -I'/home/rcohen/CP2K/cp2k-6.1.<wbr>0/tools/toolchain/install/<wbr>fftw-3.3.7/include' -I'/home/rcohen/CP2K/cp2k-6.1.<wbr>0/tools/toolchain/install/<wbr>libint-1.1.6/include' -I'/home/rcohen/CP2K/cp2k-6.1.<wbr>0/tools/toolchain/install/<wbr>libxc-4.0.3/include' -I'/home/rcohen/CP2K/cp2k-6.1.<wbr>0/tools/toolchain/install/<wbr>libxsmm-1.9.0/include' -I'/home/rcohen/gnu/include/<wbr>elpa_openmp-2018.11.001/<wbr>modules' -I'/home/rcohen/gnu/include/<wbr>elpa_openmp-2018.11.001/elpa' $(DFLAGS)<br>FCFLAGS = -march=knl -fno-omit-frame-pointer -g -O3 -funroll-loops -ffast-math $(PROFOPT) -fopenmp -I${MKLROOT}/include/intel64/<wbr>lp64 -m64 -I${MKLROOT}/include -I'/home/rcohen/CP2K/cp2k-6.1.<wbr>0/tools/toolchain/install/<wbr>mpich-3.2.1/include' -m64 -I/mnt/beegfs/intel/compilers_<wbr>and_libraries_2019.4.243/<wbr>linux/mkl/include -I'/home/rcohen/CP2K/cp2k-6.1.<wbr>0/tools/toolchain/install/<wbr>fftw-3.3.7/include' -I'/home/rcohen/CP2K/cp2k-6.1.<wbr>0/tools/toolchain/install/<wbr>libint-1.1.6/include' -I'/home/rcohen/CP2K/cp2k-6.1.<wbr>0/tools/toolchain/install/<wbr>libxc-4.0.3/include' -I'/home/rcohen/CP2K/cp2k-6.1.<wbr>0/tools/toolchain/install/<wbr>libxsmm-1.9.0/include' -I'/home/rcohen/gnu/include/<wbr>elpa_openmp2018.11.001/elpa''</div><div>FCDEBFLAGS = -ffree-form -std=f2003 -fimplicit-none <br>CFLAGS = -march=knl -fno-omit-frame-pointer -g -O3 -funroll-loops -ffast-math $(PROFOPT) -fopenmp -I'/home/rcohen/CP2K/cp2k-6.1.<wbr>0/tools/toolchain/install/<wbr>mpich-3.2.1/include' -m64 -I/mnt/beegfs/intel/compilers_<wbr>and_libraries_2019.4.243/<wbr>linux/mkl/include -I'/home/rcohen/CP2K/cp2k-6.1.<wbr>0/tools/toolchain/install/<wbr>fftw-3.3.7/include' -I'/home/rcohen/CP2K/cp2k-6.1.<wbr>0/tools/toolchain/install/<wbr>libint-1.1.6/include' -I'/home/rcohen/CP2K/cp2k-6.1.<wbr>0/tools/toolchain/install/<wbr>libxc-4.0.3/include' -I'/home/rcohen/CP2K/cp2k-6.1.<wbr>0/tools/toolchain/install/<wbr>libxsmm-1.9.0/include' -I'/home/rcohen/gnu/include/<wbr>elpa_openmp-2018.11.001/<wbr>modules' -I'/home/rcohen/gnu/include/<wbr>elpa_openmp-2018.11.001/elpa' $(DFLAGS)<br>FCFLAGS = -march=knl -fno-omit-frame-pointer -g -O3 -funroll-loops -ffast-math $(PROFOPT) -fopenmp -I${MKLROOT}/include/intel64/<wbr>lp64 -m64 -I${MKLROOT}/include -I'/home/rcohen/CP2K/cp2k-6.1.<wbr>0/tools/toolchain/install/<wbr>mpich-3.2.1/include' -m64 -I/mnt/beegfs/intel/compilers_<wbr>and_libraries_2019.4.243/<wbr>linux/mkl/include -I'/home/rcohen/CP2K/cp2k-6.1.<wbr>0/tools/toolchain/install/<wbr>fftw-3.3.7/include' -I'/home/rcohen/CP2K/cp2k-6.1.<wbr>0/tools/toolchain/install/<wbr>libint-1.1.6/include' -I'/home/rcohen/CP2K/cp2k-6.1.<wbr>0/tools/toolchain/install/<wbr>libxc-4.0.3/include' -I'/home/rcohen/CP2K/cp2k-6.1.<wbr>0/tools/toolchain/install/<wbr>libxsmm-1.9.0/include' -I'/home/rcohen/gnu/include/<wbr>elpa_openmp-2018.11.001/<wbr>modules' -I'/home/rcohen/gnu/include/<wbr>elpa_openmp-2018.11.001/elpa' $(FCDEBFLAGS) $(WFLAGS) $(DFLAGS)<br>#<br>LDFLAGS = $(FCFLAGS) -Wl,--enable-new-dtags -L'/home/rcohen/CP2K/cp2k-6.1.<wbr>0/tools/toolchain/install/<wbr>mpich-3.2.1/lib' -Wl,-rpath='/home/rcohen/CP2K/<wbr>cp2k-6.1.0/tools/toolchain/<wbr>install/mpich-3.2.1/lib' -L'/home/rcohen/CP2K/cp2k-6.1.<wbr>0/tools/toolchain/install/<wbr>fftw-3.3.7/lib' -Wl,-rpath='/home/rcohen/CP2K/<wbr>cp2k-6.1.0/tools/toolchain/<wbr>install/fftw-3.3.7/lib' -L'/home/rcohen/CP2K/cp2k-6.1.<wbr>0/tools/toolchain/install/<wbr>libint-1.1.6/lib' -Wl,-rpath='/home/rcohen/CP2K/<wbr>cp2k-6.1.0/tools/toolchain/<wbr>install/libint-1.1.6/lib' -L'/home/rcohen/CP2K/cp2k-6.1.<wbr>0/tools/toolchain/install/<wbr>libxc-4.0.3/lib' -Wl,-rpath='/home/rcohen/CP2K/<wbr>cp2k-6.1.0/tools/toolchain/<wbr>install/libxc-4.0.3/lib' -L'/home/rcohen/CP2K/cp2k-6.1.<wbr>0/tools/toolchain/install/<wbr>libxsmm-1.9.0/lib' -Wl,-rpath='/home/rcohen/CP2K/<wbr>cp2k-6.1.0/tools/toolchain/<wbr>install/libxsmm-1.9.0/lib' -Wl,-rpath='/home/rcohen/gnu/<wbr>lib'<br>LIBS = -lelpa_openmp -lxsmmf -lxsmm -ldl -lpthread -lxcf03 -lxc -lderiv -lint -lfftw3 -lfftw3_omp -lmpi -lmpicxx ${MKLROOT}/lib/intel64/libmkl_<wbr>blas95_lp64.a ${MKLROOT}/lib/intel64/libmkl_<wbr>lapack95_lp64.a ${MKLROOT}/lib/intel64/libmkl_<wbr>scalapack_lp64.a -Wl,--start-group ${MKLROOT}/lib/intel64/libmkl_<wbr>gf_lp64.a ${MKLROOT}/lib/intel64/libmkl_<wbr>gnu_thread.a ${MKLROOT}/lib/intel64/libmkl_<wbr>core.a ${MKLROOT}/lib/intel64/libmkl_<wbr>blacs_intelmpi_lp64.a -Wl,--end-group -lgomp -lpthread -lm -ldl -lstdc++<br></div></div><div><br></div><div><br></div><div>I had cp2k built fine but my system migrated to centos and I had to rebuild. I had a similar problem in the past trying to use the intel toolchain.</div><div><br></div><div>Thanks for help!</div><div><br></div><div>Sincerely,</div><div><br></div><div>Ron</div><div><br></div></div><div><br></div> -- <br> You received this message because you are subscribed to a topic in the Google Groups "cp2k" group.<br> To unsubscribe from this topic, visit <a href="https://groups.google.com/d/topic/cp2k/vrgNXE8L0F4/unsubscribe" target="_blank" rel="nofollow" onmousedown="this.href='https://groups.google.com/d/topic/cp2k/vrgNXE8L0F4/unsubscribe';return true;" onclick="this.href='https://groups.google.com/d/topic/cp2k/vrgNXE8L0F4/unsubscribe';return true;">https://groups.google.com/d/<wbr>topic/cp2k/vrgNXE8L0F4/<wbr>unsubscribe</a>.<br> To unsubscribe from this group and all its topics, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="-c6JAqWQAQAJ" rel="nofollow" onmousedown="this.href='javascript:';return true;" onclick="this.href='javascript:';return true;">cp...@googlegroups.<wbr>com</a>.<br> To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/cp2k/cd7bcd1f-4250-4528-aab7-ff5dd656417d%40googlegroups.com?utm_medium=email&utm_source=footer" target="_blank" rel="nofollow" onmousedown="this.href='https://groups.google.com/d/msgid/cp2k/cd7bcd1f-4250-4528-aab7-ff5dd656417d%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter';return true;" onclick="this.href='https://groups.google.com/d/msgid/cp2k/cd7bcd1f-4250-4528-aab7-ff5dd656417d%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter';return true;">https://groups.google.com/d/<wbr>msgid/cp2k/cd7bcd1f-4250-4528-<wbr>aab7-ff5dd656417d%<wbr>40googlegroups.com</a>.<br> </div></blockquote></div><br></div></div></blockquote></div>