cp2k on POWER4 running Linux

[Eric Shamay]

The parallel compilation finally went through after tweaking the BLACS
and SCALAPACK just right. The problem with BLACS was definitely that
the 'make clean' command wasn't working as expected for the SRC/MPI
directory, and also setting the TRANSCOMM to -DUseMpich instead of -
DUseMpi. SCALAPACK had an issue with underscores not being set
correctly as guessed. I'll list the Bmake.inc and SLmake.inc below for
anyone searching the forums and using a similar platform.

There is another issue now that I've managed to compile all the
libraries and an executable cp2k.popt. When I run any test using the
standard 'mpirun -np 8 cp2k.popt traj.inp' (or similar) the program
dies right away. An example is when I run the cp2k/tests/QS/H2O-
md.inp.

Output is listed below:

**********************************

10:36:05  eshamay at p690 (~/cp2k/tests/QS) mpirun -np 8 /home/users/
eshamay/cp2k/exe/p690/cp2k.popt H2O-md.inp
 *
 *** 10:36:31 ERRORL2 in cp_parser_methods:parser_read_line err=12
An   ***
 *** input/output error occurred file:'/home/users/eshamay/cp2k/tests/
QS'  ***
 *** line:     1 col:     0(IOSTAT =
12)                               ***
 *

 Looking for words in the input similar to the unknown:
   'No info available yet, we will keep you informed.'

 CP2K| Abnormal program termination, stopped by process number 1
 CP2K| Abnormal program termination, stopped by process number 6
/usr/local/packages/mpich/mpich-1.2.7p1-p4_ibm-7.0-9.1/bin/
mpirun.ch_p4: line 243: 16932 Segmentation fault      "/home/users/
eshamay/cp2k/exe/p690/cp2k.popt" "H2O-md.inp" -p4pg "/home/users/
eshamay/cp2k/tests/QS/PI16692" -p4wd "/home/users/eshamay/cp2k/tests/
QS"
p1_16938:  p4_error: : 1
rm_l_1_16950: (11.592950) net_send: could not write to fd=5, errno =
32
[1] MPI Abort by user Aborting program !
[1] Aborting program!
p6_17024:  p4_error: : 1
rm_l_6_17036: (8.791662) net_send: could not write to fd=5, errno = 32
p6_17024: (8.793096) net_send: could not write to fd=5, errno = 32
[6] MPI Abort by user Aborting program !
[6] Aborting program!
10:36:39  eshamay at p690 (~/cp2k/tests/QS) p1_16938: (15.604253)
net_send: could not write to fd=5, errno = 32

***********************

Any idea what that could mean? An issue with my mpi setup or with the
way I've compiled cp2k? I'm totally lost at this point.

Thanks!
~Eric






*** system info ***

IBM POWER4 with 16 cores
ibmxlc/7.0.1
ibmxlf/9.1.1
mpi/mpich-1.2.7p1-p4_ibm-7.0-9.1
ATLAS blas and lapack routines, but adding in the full lapack library
from a separate compile.

**** SLmake.inc *****

SHELL         = /bin/sh
home          = $(HOME)/src/SCALAPACK
PLAT          = LINUX
BLACSDBGLVL   = 0
BLACSdir      = $(HOME)/src/BLACS
USEMPI        = -DUsingMpiBlacs
SMPLIB        =
BLACSFINIT    = $(BLACSdir)/LIB/blacsF77init_MPI-$(PLAT)-$
(BLACSDBGLVL).a
BLACSCINIT    = $(BLACSdir)/LIB/blacsCinit_MPI-$(PLAT)-$
(BLACSDBGLVL).a
BLACSLIB      = $(BLACSdir)/LIB/blacs_MPI-$(PLAT)-$(BLACSDBGLVL).a
TESTINGdir    = $(home)/TESTING
CBLACSLIB     = $(BLACSCINIT) $(BLACSLIB) $(BLACSCINIT)
FBLACSLIB     = $(BLACSFINIT) $(BLACSLIB) $(BLACSFINIT)
PBLASdir      = $(home)/PBLAS
SRCdir        = $(home)/SRC
TESTdir       = $(home)/TESTING
PBLASTSTdir   = $(TESTINGdir)
TOOLSdir      = $(home)/TOOLS
REDISTdir     = $(home)/REDIST
REDISTTSTdir  = $(TESTINGdir)
F77           = mpif77
CC            = mpicc
NOOPT         =
F77FLAGS      = $(NOOPT) -O2 -qarch=pwr4 -qcache=auto -qmaxmem=-1 -
qtune=pwr4
CCFLAGS       = -O3
SRCFLAG       =
F77LOADER     = $(F77)
CCLOADER      = $(CC)
F77LOADFLAGS  =
CCLOADFLAGS   =
CDEFS         = -DNoChange -DNO_IEEE $(USEMPI)
ARCH          = ar
ARCHFLAGS     = cr
RANLIB        = ranlib
SCALAPACKLIB  = $(home)/libscalapack.a
ATLASLIB	  = $(HOME)/src/ATLAS/lib/POWER4
BLASLIB       = $(ATLASLIB)/libf77blas.a $(ATLASLIB)/libcblas.a $
(ATLASLIB)/libatlas.a
LAPACKLIB     = $(ATLASLIB)/liblapack.a
PBLIBS        = $(SCALAPACKLIB) $(FBLACSLIB) $(LAPACKLIB) $(BLASLIB) $
(SMPLIB)
PRLIBS        = $(SCALAPACKLIB) $(CBLACSLIB) $(SMPLIB)
RLIBS         = $(SCALAPACKLIB) $(FBLACSLIB) $(CBLACSLIB) $(LAPACKLIB)
$(BLASLIB) $(SMPLIB)
LIBS          = $(PBLIBS)


**** Bmake.inc ****


   SHELL = /bin/sh
   BTOPdir = $(HOME)/src/BLACS
   COMMLIB = MPI
   PLAT = LINUX
   BLACSdir    = $(BTOPdir)/LIB
   BLACSDBGLVL = 0
   BLACSFINIT  = $(BLACSdir)/blacsF77init_$(COMMLIB)-$(PLAT)-$
(BLACSDBGLVL).a
   BLACSCINIT  = $(BLACSdir)/blacsCinit_$(COMMLIB)-$(PLAT)-$
(BLACSDBGLVL).a
   BLACSLIB    = $(BLACSdir)/blacs_$(COMMLIB)-$(PLAT)-$(BLACSDBGLVL).a
   MPIdir = $(MPI)
   MPIdev =
   MPIplat =
   MPILIBdir = $(MPIdir)/lib
   MPIINCdir = $(MPIdir)/include
   MPILIB = $(MPIdir)/lib/libmpich.a
   BTLIBS = $(BLACSFINIT) $(BLACSLIB) $(BLACSFINIT) $(MPILIB)
   INSTdir = $(BTOPdir)/INSTALL/EXE
   TESTdir = $(BTOPdir)/TESTING/EXE
   FTESTexe = $(TESTdir)/xFbtest_$(COMMLIB)-$(PLAT)-$(BLACSDBGLVL)
   CTESTexe = $(TESTdir)/xCbtest_$(COMMLIB)-$(PLAT)-$(BLACSDBGLVL)

   SYSINC = -I$(MPIINCdir)
   INTFACE = -DNoChange
   SENDIS =
   BUFF =
  TRANSCOMM = -DUseMpich
   WHATMPI =
   SYSERRORS =
   DEBUGLVL = -DBlacsDebugLvl=$(BLACSDBGLVL)
   DEFS1 = -DSYSINC $(SYSINC) $(INTFACE) $(DEFBSTOP) $(DEFCOMBTOP) $
(DEBUGLVL)
   BLACSDEFS = $(DEFS1) $(SENDIS) $(BUFF) $(TRANSCOMM) $(WHATMPI) $
(SYSERRORS)

   F77            = mpif77
   F77NO_OPTFLAGS =
   F77FLAGS       = $(F77NO_OPTFLAGS) -O2 -qarch=pwr4 -qcache=auto -
qmaxmem=-1 -qtune=pwr4
   F77LOADER      = $(F77)
   F77LOADFLAGS   =
   CC             = mpicc
   CCFLAGS        = -O2
   CCLOADER       = $(CC)
   CCLOADFLAGS    =
   ARCH      = ar
   ARCHFLAGS = r
   RANLIB    = ranlib