[CP2K:3741] Re: cp2k on IBM Blue Gene

[Wei]

Dear Iain, 

today I have tried to compile cp2k on Blue Gene/Q with the arch file you 
provided in the paper, I got some internal compiler error after some time, 

** topology_psf   === End of Compilation 1 ===
1501-510  Compilation successful for file topology_psf.f90.
** fist_environment_types   === End of Compilation 1 ===
1501-510  Compilation successful for file fist_environment_types.f90.
** qs_environment_types   === End of Compilation 1 ===
1585-100  Internal compiler error -- please contact your service 
representative. For more information visit: 
http://www.ibm.com/support/docview.wss?uid=swg21110810
1501-511  Compilation failed for file qs_environment_types.f90.
make[1]: *** [qs_environment_types.o] Error 255

Do you maybe have tried it already? or have you seen this error previously 
on BG/P?

I download the most recent version from the trunk, and my acrh file is 

CC = xlc
CPP = cpp -traditional -E
FC = mpixlf95_r -qsuffix=f=f90
LD = mpixlf95_r
AR = ar -r
DFLAGS = -D__HAS_NO_ISO_C_BINDING -D__AIX -D__ESSL -D__FFTSG -D__parallel \
-D__BLACS -D__SCALAPACK -D__FFTW3
FFTW3_BASE = /bgsys/local/fftw3/3.3.2/fftw_g/
CPPFLAGS = -C $(DFLAGS) -P -I$(FFTW3_BASE)/include
FCFLAGS = -O2 -qarch=450d -qtune=450 -qstrict -qessl
FCFLAGS2 = -O0 -qarch=450d -qtune=450 -qessl
LDFLAGS = $(FCFLAGS)
LIBS = /bgsys/local/scalapack/2.0.1_g/lib/libscalapack.a \
/bgsys/local/lapack/3.3.0_g/lib/liblapack.a \
$(FFTW3_BASE)/lib/libfftw3.a \
/opt/ibmmath/essl/5.1/lib64/libesslbg.a

OBJECTS_ARCHITECTURE = machine_aix.o
bibliography.o: bibliography.F
         $(FC) -c $(FCFLAGS2) $<

Since we don't need LIBINT, I just removed all the related library path to 
make the compilation easier.. Also I didn't include libblacsF77init.a and 
libblacs.a, I cannot find them on our system, but our admin said they are 
integrated in libscalapack.a.

Thanks a lot in advance!

Best

Wei

On Wednesday, March 7, 2012 10:51:13 AM UTC+1, IBethune wrote:
>
> Wei,
>
> In CP2K, since some data relating to the atoms in the system (topology, 
> neighbour lists etc.) are replicated, this implies that the system size is 
> ultimately limited by the amount of memory per node, and although in 
> practice this is only an issue for very large systems perhaps 100,000s of 
> atoms.  Using more CPU cores will help to an extent since other data 
> structures such as the sparse matrices will be distributed over more cores, 
> and take up less memory per core, but for such a large system size you will 
> likely need a large number of cores to accomplish your calculation in a 
> reasonable amount of time in any case.
>
> The scaling results reported in the PRACE white paper are for a 6144-atom 
> system (DFT with MOLOPT basis set), and good scaling (close to linear) was 
> achieved up to 8192 cores, which is the largest run we attempted.  In 
> general, one would expect better scalability on the BG/P due to the fact it 
> has slower cores and faster (lower latency, higher bandwidth) interconnect 
> than a standard linux cluster.  I don't have any specific results to 
> compare with however.
>
> Re: BG/Q, given that IBM provides a linux-like environment with standard 
> fortran Compilers I don't forsee huge issues with porting, but we have not 
> tested this (yet).
>
> Cheers
>
> - Iain
>
> --
>
> Iain Bethune
> Applications Consultant, EPCC
>
> Email: ibet... at epcc.ed.ac.uk
> Tel/Fax: +44 (0)131 650 5201/6555
> Mob: +44 (0)7598317015
> Addr: 2404 JCMB, The King's Buildings, Mayfield Road, Edinburgh, EH9 3JZ
>
> On 6 Mar 2012, at 16:12, Wei wrote:
>
> > Hi Iain,
> > 
> > thanks a lot for your fast reply!
> > 
> > I have read the document, and it is a huge surprise that you have
> > tried with 1 million atom system with DFT.. Now I have two further
> > questions:
> > 
> > 1. Does this newly compiled cp2k on Blue Gene has a lower limit of
> > atom size for a specific job (4000 cores)? Have you ever tried to run
> > md or relaxation for many steps for a smaller system size (i.e. 10,000
> > atom), and how big is the speedup factor compare to the calculation on
> > normal linux-cluster with intel cpus (let's say 200 cores with 3.06
> > GHz)?
> > 
> > 2. will the compilation be the same for Blue Gene/Q?
> > 
> > thanks a lot!
> > 
> > Best Regards,
> > 
> > Wei
> > 
> > 
> > On Mar 6, 3:04 pm, Iain Bethune <ibet... at epcc.ed.ac.uk> wrote:
> >> Hi Wei,
> >> 
> >> We ported CP2K to the BlueGene/P under PRACE last year.  See here :
> http://prace-portal.cscs.ch/uploads/tx_pracetmo/CP2K.pdffor details, 
> incuding ARCH files and compiler details.
> >> 
> >> There were some problems, particularly with the mixed-mode (psmp) 
> version of the code, but if you just want to do MPI it will be fine.
> >> 
> >> Cheers
> >> 
> >> - Iain
> >> 
> >> --
> >> 
> >> Iain Bethune
> >> Applications Consultant, EPCC
> >> 
> >> Email: ibet... at epcc.ed.ac.uk
> >> Tel/Fax: +44 (0)131 650 5201/6555
> >> Mob: +44 (0)7598317015
> >> Addr: 2404 JCMB, The King's Buildings, Mayfield Road, Edinburgh, EH9 3JZ
> >> 
> >> On 6 Mar 2012, at 14:02, Wei wrote:
> >> 
> >> 
> >> 
> >> 
> >> 
> >> 
> >> 
> >> 
> >> 
> >>> Dear all,
> >> 
> >>> I wonder if there is an efficient version of cp2k designed for IBM
> >>> Blue Gene? How could I find more information about it? Any suggestion
> >>> will be highly appreciated.
> >> 
> >>> Thanks in advance!
> >> 
> >>> Best Regards,
> >> 
> >>> Wei Zhang
> >> 
> >>> --
> >>> You received this message because you are subscribed to the Google 
> Groups "cp2k" group.
> >>> To post to this group, send email to cp... at googlegroups.com.
> >>> To unsubscribe from this group, send email to 
> cp2k+uns... at googlegroups.com.
> >>> For more options, visit this group athttp://
> groups.google.com/group/cp2k?hl=en.
> >> 
> >> --
> >> The University of Edinburgh is a charitable body, registered in
> >> Scotland, with registration number SC005336.
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups "cp2k" group.
> > To post to this group, send email to cp... at googlegroups.com.
> > To unsubscribe from this group, send email to 
> cp2k+uns... at googlegroups.com.
> > For more options, visit this group at 
> http://groups.google.com/group/cp2k?hl=en.
> > 
> > 
>
>
> -- 
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.cp2k.org/archives/cp2k-user/attachments/20120627/f0b4a9c1/attachment.htm>