[CP2K:3741] Re: cp2k on IBM Blue Gene

[Iain Bethune]

Wei,

In CP2K, since some data relating to the atoms in the system (topology, neighbour lists etc.) are replicated, this implies that the system size is ultimately limited by the amount of memory per node, and although in practice this is only an issue for very large systems perhaps 100,000s of atoms.  Using more CPU cores will help to an extent since other data structures such as the sparse matrices will be distributed over more cores, and take up less memory per core, but for such a large system size you will likely need a large number of cores to accomplish your calculation in a reasonable amount of time in any case.

The scaling results reported in the PRACE white paper are for a 6144-atom system (DFT with MOLOPT basis set), and good scaling (close to linear) was achieved up to 8192 cores, which is the largest run we attempted.  In general, one would expect better scalability on the BG/P due to the fact it has slower cores and faster (lower latency, higher bandwidth) interconnect than a standard linux cluster.  I don't have any specific results to compare with however.

Re: BG/Q, given that IBM provides a linux-like environment with standard fortran Compilers I don't forsee huge issues with porting, but we have not tested this (yet).

Cheers

- Iain

--

Iain Bethune
Applications Consultant, EPCC

Email: ibet... at epcc.ed.ac.uk
Tel/Fax: +44 (0)131 650 5201/6555
Mob: +44 (0)7598317015
Addr: 2404 JCMB, The King's Buildings, Mayfield Road, Edinburgh, EH9 3JZ





On 6 Mar 2012, at 16:12, Wei wrote:

> Hi Iain,
> 
> thanks a lot for your fast reply!
> 
> I have read the document, and it is a huge surprise that you have
> tried with 1 million atom system with DFT.. Now I have two further
> questions:
> 
> 1. Does this newly compiled cp2k on Blue Gene has a lower limit of
> atom size for a specific job (4000 cores)? Have you ever tried to run
> md or relaxation for many steps for a smaller system size (i.e. 10,000
> atom), and how big is the speedup factor compare to the calculation on
> normal linux-cluster with intel cpus (let's say 200 cores with 3.06
> GHz)?
> 
> 2. will the compilation be the same for Blue Gene/Q?
> 
> thanks a lot!
> 
> Best Regards,
> 
> Wei
> 
> 
> On Mar 6, 3:04 pm, Iain Bethune <ibet... at epcc.ed.ac.uk> wrote:
>> Hi Wei,
>> 
>> We ported CP2K to the BlueGene/P under PRACE last year.  See here :http://prace-portal.cscs.ch/uploads/tx_pracetmo/CP2K.pdffor details, incuding ARCH files and compiler details.
>> 
>> There were some problems, particularly with the mixed-mode (psmp) version of the code, but if you just want to do MPI it will be fine.
>> 
>> Cheers
>> 
>> - Iain
>> 
>> --
>> 
>> Iain Bethune
>> Applications Consultant, EPCC
>> 
>> Email: ibet... at epcc.ed.ac.uk
>> Tel/Fax: +44 (0)131 650 5201/6555
>> Mob: +44 (0)7598317015
>> Addr: 2404 JCMB, The King's Buildings, Mayfield Road, Edinburgh, EH9 3JZ
>> 
>> On 6 Mar 2012, at 14:02, Wei wrote:
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>>> Dear all,
>> 
>>> I wonder if there is an efficient version of cp2k designed for IBM
>>> Blue Gene? How could I find more information about it? Any suggestion
>>> will be highly appreciated.
>> 
>>> Thanks in advance!
>> 
>>> Best Regards,
>> 
>>> Wei Zhang
>> 
>>> --
>>> You received this message because you are subscribed to the Google Groups "cp2k" group.
>>> To post to this group, send email to cp... at googlegroups.com.
>>> To unsubscribe from this group, send email to cp2k+uns... at googlegroups.com.
>>> For more options, visit this group athttp://groups.google.com/group/cp2k?hl=en.
>> 
>> --
>> The University of Edinburgh is a charitable body, registered in
>> Scotland, with registration number SC005336.
> 
> -- 
> You received this message because you are subscribed to the Google Groups "cp2k" group.
> To post to this group, send email to cp... at googlegroups.com.
> To unsubscribe from this group, send email to cp2k+uns... at googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/cp2k?hl=en.
> 
> 


-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.