[CP2K:3237] status of hybrid OpenMP+MPI version

[Axel]

On Monday, May 9, 2011 4:50:22 PM UTC-4, Urban wrote:
>
> Hi Axel,
>
> On Mon, 2011-05-09 at 09:34 -0700, Axel wrote:
> > [...]
> > hmmm... interesting.
> > was this the very latest cvs code?
>
> Yes, this was today's CVS.
>

ok. i updated and tried again, but no luck so far.

> > it almost looked to me as if there were some 
> > mpi calls within a multi-threaded region that 
> > were messing things up...
>
> I believe we don't (or shouldn't) use such calls.  Also, you can
> explicitly make CP2K call mpi_init_thread() instead of mpi_init()
> (change .TRUE. to .FALSE. on line 479 of src/message_passing.F)--the
> current default is a workaround  for de facto MPI threading behavior but
> is technically wrong.  Also, I just committed a patch to declare the use
> of MPI funneled mode instead of general mode.
>
i tried that, but now cp2k quits right away with a confusing error:

 CP2K|  MPI error 0 in Inadequate level of thread support is provided. : 
MPI_SUCCESS: no errors
 CP2K| Abnormal program termination, stopped by process number 0
 CP2K|  MPI error 0 in Inadequate level of thread support is provided. : 
MPI_SUCCESS: no errors
 CP2K| Abnormal program termination, stopped by process number 1
 CP2K|  MPI error 0 in Inadequate level of thread support is provided. : 
MPI_SUCCESS: no errors
 CP2K| Abnormal program termination, stopped by process number 2
 CP2K|  MPI error 0 in Inadequate level of thread support is provided. : 
MPI_SUCCESS: no errors
 CP2K| Abnormal program termination, stopped by process number 3
 CP2K|  MPI error 0 in Inadequate level of thread support is provided. : 
MPI_SUCCESS: no errors
 CP2K| Abnormal program termination, stopped by process number 4
 CP2K|  MPI error 0 in Inadequate level of thread support is provided. : 
MPI_SUCCESS: no errors
 CP2K| Abnormal program termination, stopped by process number 5
 CP2K|  MPI error 0 in Inadequate level of thread support is provided. : 
MPI_SUCCESS: no errors
 CP2K| Abnormal program termination, stopped by process number 6
 CP2K|  MPI error 0 in Inadequate level of thread support is provided. : 
MPI_SUCCESS: no errors
 CP2K| Abnormal program termination, stopped by process number 7

> also, can you comment on what these two defines are for?
> >  -D__USE_CP2K_TRACE -D__HAS_smm_dnn 
>
> __USE_CP2K_TRACE uses CP2K's stack trace infrastructure on failed
> assertions in the DBCSR sparse matrix (sub)library.  To use it you must
> also add the timings_mp.o module to the LIBS variable (used to get
> around this and some other circular dependencies).  While using it or
> not should not affect behavior, there are OpenMP statements in stack
> tracing and timing.
>
i have so far tried without.

> __HAS_smm_dnn is for using the Small Matrix Multiply library (see
> tools/build_libsmm).  It is not threaded so skipping it should have no
> effect on the program's behavior.
>
ok. thanks for the info. this is a fairly new feature, right?
 

> > this one is a bit of a surprise, since already gfortran 4.3.x claims
> > to be OpenMP 3 compliant.
> > -D__HAS_NO_OMP_3
>
> I used gfortran 4.3 until recently (where psmp also worked last time I
> checked) and didn't update the defines...
>
> > as far as threaded libraries go. what about MKL?
> > last thing a remember is that it would not go multi-threaded
> > if it is called from within an OpenMP multi-threaded region,
> > but that may be wishful thinking...
>
> I haven't tried MKL recently so I can't say.  However, to rule out any
> library issues, I would try standard netlib BLAS and Lapack...
>
well, in my experience those can get as easily miscompiled
as using precompiled packages like MKL.

for good measure, i am linking with the serial version
now, but that didn't change the behavior.

it _does_ work fine in the sopt/popt binaries.

just make sure we're on the same page. the input(s) that
are giving me the problems are the water benchmark inputs.
in tests/QS/benchmark/

the cp2k.?smp binaries work fine for OMP_NUM_THREADS=1,
but as soon as i enable more than one thread with cp2k.psmp
and multiple MPI tasks, i get NaNs or random energies.
cp2k.ssmp appears to be working for both intel 11.1 and gfortran 4.4.4

thanks,
    axel.

Cheers,
> Urban.
>
> > 
> > in any case, i'll give it a try.
> > 
> > 
> > thanks,
> >      axel.
> > 
> > -- 
> > You received this message because you are subscribed to the Google
> > Groups "cp2k" group.
> > To post to this group, send email to cp... at googlegroups.com.
> > To unsubscribe from this group, send email to cp2k
> > +unsu... at googlegroups.com.
> > For more options, visit this group at
> > http://groups.google.com/group/cp2k?hl=en.
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.cp2k.org/archives/cp2k-user/attachments/20110509/6dbe5bd7/attachment.htm>