<div dir="ltr"><div>I am getting the following crashes reproducibly after several thousand time steps:</div><div><br></div><div> *******************************************************************************<br> *             MPI error 672742415 in mpi_cart_sub @ mp_cart_sub : Other MPI   *<br> *   ___         error, error stack:<br>PMPI_Cart_sub(228)..................:     *<br> *  /   \        MPI_Cart_sub(comm=0xc4040010, remain_dims=0x7f200ce27e80,     *<br> * [ABORT]                       comm_new=0x2b5549318b64)                      *<br> *  \___/              failed<br>PMPI_Cart_sub(167)..................:            *<br> *    |                   <br>MPIR_Comm_split_impl(253)...........:               *<br> *  O/|        <br>MPIR_Get_contextid_sparse_group(604): Too many communicators   *<br> * /| |                 (0/32768 free on this process; ignore_id=0)            *<br> * / \                                                    dbcsr_mpiwrap.F:1098 *<br> *******************************************************************************<br><br><br> ===== Routine Calling Stack ===== <br><br>           14 mp_cart_sub<br>           13 dbcsr_complete_redistribute<br>           12 copy_dbcsr_to_fm<br>           11 cp_dbcsr_sm_fm_multiply<br>           10 make_basis_sm<br>            9 reorthogonalize_vectors<br>            8 wfi_extrapolate<br>            7 scf_env_initial_rho_setup<br>            6 init_scf_run<br>            5 qs_energies<br>            4 qs_forces<br>            3 velocity_verlet<br>            2 qs_mol_dyn_low<br>            1 CP2K<br></div><div><br></div><div>It seems that there is a communicator leak--more communicators are created than destroyed. This is using the latest code for github . I am using 2019 intel compilers and mkl ,</div><div>but it seems to be a problem in cp2k dbscr. I am running psmp hybrid with 384 cores and OMP_NUM_THREADS=4 . I am stuck until I can get pas this.<br></div><div><br></div><div>Thanks!</div><div><br></div><div>Ron</div><div><br></div><div><br></div></div>