THE ODISSEY OF CP2K; THE LONG WAY BACK TO ITHACA

[Axel]

hi francesco,

[...]

my your words find entrance into the ears of the cp2k gods.

it is funny to see how cp2k divides the world. i am feeling a bit bad, 
though,
because some of your OpenMP troubles could have been avoided if
i had posted a summary based on the thread that i started recently.
at some point the discussion went off-list and thus there was no resolution
posted here. perhaps, i should make up for it now.

for the most part, it wasn't possible to reach an agreement what the 
(main) problem is, since it apparently cannot be reproduced on other
machines but the ones that i am using. ...and i refuse to use as ugly
a software as MPICH, if the much more elegant and maintainable 
OpenMPI provides the same API and only a faction of the headaches
for maintaining and using it. so i ended up with the following conclusions
as to how to compile for MPI+OpenMP (MPI-only had no problems
with either compiler provided a serial MKL was loaded)

- gfortran 4.4.x is OpenMP 3.0 capable according to cp2k, intel 11.1 is not.
  one can make the ssmp  version work with a define.

- OpenMPI needs to be compiled with MPI threads support (or another 
workaround
  define needs to be used), but then seems to work, even with the MKL 
provided
  BLACS/SCALAPACK (i have more trust in that being compiled correctly 
than myself).

- when compiling with OpenMP enabled, don't compile fftw3 with OpenMP 
support.
  also, don't link with a multi-threaded MKL for psmp. this is actually less 
of a problem
  than what might initially think, since most of the cases where you would 
_want_ to
  use OpenMP+MPI (i.e. when its performance is better than MPI alone), there 
is
  not so much data to parallelize FFTs or BLAS over to justify the overhead. 
one can 
  test this using FFTSG where the impact of parallel/non-parallel compile of 
the FFT
  alone can be tested. this is not really that much of a surprise, since on 
modern hardware
  memory bandwidth is the most valuable property and with multi-threaded 
FFTs in
  shared memory, you often are head back by the available bandwidth more 
than what
  you would gain from the additional CPU power.

with these settings, i can run the testjobs that people in our group need 
fairly reliable
and efficiently. Intel still seems to be off a bit with OpenMP+MPI, since 
occasionally
calculations get stuck. i cannot help, but suspect that there are some race 
conditions
hidden that just don't show if you have the right hardware/software on your 
local machine.

Anyway, these are my two pennies for improving the architecture file 
> archive, and, maybe, stimulate some discussion about the issue of 
> working versions and debug of the code. 
>

unfortunately, the cp2k code has developed into such an hard to read and
difficult to understand monster, that it is unlikely that problems will be 
resolved
unless they show up on somebody's machine that is used to digging into
the code deep enough to sort this out. for me this was the end of the road.

in some sense, i can understand the status of the arch files. you have
to change them for almost any machine all the time, and people _can_
look up in the corresponding documentation how to do this. if you don't
know how to do this, you probably shouldn't dare to compile cp2k in the
first place. turning this into some autoconf (or worse, cmake) style setup
will most likely make things even worse, by giving people a false sense
of know having to know exactly what to place on the compiler and linker
flag variable statements.

with that in mind, i am attaching the set of arch files that i am currently
using and that seemed to have worked at some point in time around 
mid may in 2011. 

ciao,
    axel.

p.s.: building/tuning libsmm on a 4-way opteron 6174 machine is fun. ;-)

 

>
>
> Cheers, 
> F. 
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.cp2k.org/archives/cp2k-user/attachments/20110528/c4dadd95/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cp2k-arch-ak.tar.gz
Type: application/octet-stream
Size: 1233 bytes
Desc: not available
URL: <https://lists.cp2k.org/archives/cp2k-user/attachments/20110528/c4dadd95/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: libsmm-config.tar.gz
Type: application/octet-stream
Size: 1542 bytes
Desc: not available
URL: <https://lists.cp2k.org/archives/cp2k-user/attachments/20110528/c4dadd95/attachment-0001.obj>