[CP2K-user] [CP2K:14653] how could the RAM for ADMM be negative number

Sun Geng gengs... at gmail.com
Fri Jan 29 16:39:18 UTC 2021
Previous message (by thread): [CP2K-user] [CP2K:14653] how could the RAM for ADMM be negative number
Next message (by thread): [CP2K-user] Error while attempting to read the Hessian file
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Hi Tiziano,
Thank you very much for your reply, now I understand the first question. 
For the second question (about the estimation of the ERI memory 
consumption),  which tag in the CP2K input file allows printing this 
estimated memory?
Best,
Geng


在2021年1月28日星期四 UTC-8 下午11:49:58<tiz... at chem.uzh.ch> 写道：

> Dear Geng,
>
> for 1) the memory should be the total over MPI ranks (but only the ERI 
> storage part). Since the HFX code is OpenMP-parallelized I would suggest 
> to use less MPI ranks per physical node and more threads. Each MPI rank 
> has to maintain the full density matrix locally, hence you would like to 
> avoid too many ranks per node. The ERIs on the other hand can be 
> compressed and even stored to disk if needed.
>
> for 2) much of the memory usage is known a-priori (before any ERI is 
> actually calculated), see 
>
> https://github.com/cp2k/cp2k/blob/d7da0214a5d90c06d8c33ae514e3a7d3c4a4888b/src/hfx_energy_potential.F#L611-L651, 
>
> but some part as you already wrote is dynamic. From that one could 
> probably derive a (pessimistic) upper limit.
>
> Best,
> Tiziano
>
> On 1/29/21 4:05 AM, Sun Geng wrote:
> > Dear Tiziano,
> > Thanks for your reply.
> > I applied the patch to the CP2K code (v8.1), and now it can print a 
> > reasonable number for the RAM (below).
> > May I take the advantage to ask two additional questions regarding the 
> > RAM in HFX? Thank you very much in advance.
> > 1) The line "  HFX_MEM_INFO| Total memory consumption ERI's RAM [MiB]: 
> >                540840" prints the total (including all MPI processors ) 
> > used memory, am I right?
> > 2) Is there a way to estimate the required memory for the HFX module 
> > before running the CP2k program? I thought no because the memory is 
> > allocated dynamically according to the occupation matrix, but I would 
> > like to have a double-check.
> > Thanks again.
> > Best Regards,
> > Geng
> > 
> > 
> >   HFX_MEM_INFO| Number of cart. primitive ERI's calculated: 
> >  32799680679415
> >   HFX_MEM_INFO| Number of sph. ERI's calculated: 
> >  1498990881919
> >   HFX_MEM_INFO| Number of sph. ERI's stored in-core: 
> > 683885389270
> >   HFX_MEM_INFO| Number of sph. ERI's stored on disk: 
> >        0
> >   HFX_MEM_INFO| Number of sph. ERI's calculated on the fly: 
> >         0
> >   HFX_MEM_INFO| Total memory consumption ERI's RAM [MiB]: 
> >    540840
> >   HFX_MEM_INFO| Whereof max-vals [MiB]: 
> >     10475
> >   HFX_MEM_INFO| Total compression factor ERI's RAM: 
> >      9.65
> >   HFX_MEM_INFO| Total memory consumption ERI's disk [MiB]: 
> >        0
> >   HFX_MEM_INFO| Total compression factor ERI's disk: 
> >     0.00
> >   HFX_MEM_INFO| Size of density/Fock matrix [MiB]: 
> >      408
> >   HFX_MEM_INFO| Size of buffers [MiB]: 
> >       25
> >   HFX_MEM_INFO| Number of periodic image cells considered: 
> >       33
> >   HFX_MEM_INFO| Est. max. program size after HFX  [MiB]: 
> >     7536
> > 
> > 
> > 在2021年1月28日星期四 UTC-8 上午8:09:03<tiz... at chem.uzh.ch> 写道：
> > 
> > Hi Geng,
> > 
> > can you please tell us which version of CP2K you are using and possibly
> > also submit an input file with which you can reproduce the issue?
> > 
> > But since that's quiet a number of ERIs (which seems a bit weird since
> > you're using ADMM), my first guess is an integer overflow here:
> > 
> > 
> https://github.com/cp2k/cp2k/blob/5b641945b68ab1861fea87544a697b03403b4443/src/hfx_energy_potential.F#L1776-L1777
> > <
> https://github.com/cp2k/cp2k/blob/5b641945b68ab1861fea87544a697b03403b4443/src/hfx_energy_potential.F#L1776-L1777
> >
> > 
> > 
> > ... while the target variable is an int8, the variables multiplied are
> > int4, and get multiplied first as such before converted to int8:
> > https://gcc.godbolt.org/z/x4hYsb <https://gcc.godbolt.org/z/x4hYsb>
> > If you have CP2K v8.1 you can try the attached patch and see whether it
> > fixes the issue (only compile-tested so far).
> > 
> > Best,
> > Tiziano
> > 
> > On 1/28/21 4:01 PM, Sun Geng wrote:
> > > Dear CP2K users,
> > > I am running jobs with CP2K/ADMM method, in which the CP2k prints
> > the
> > > following lines:
> > >
> > >   HFX_MEM_INFO| Number of cart. primitive ERI's calculated:
> > >  52451602395221
> > >   HFX_MEM_INFO| Number of sph. ERI's calculated:
> > >  2343013195064
> > >   HFX_MEM_INFO| Number of sph. ERI's stored in-core:
> > >  2247108266900
> > >   HFX_MEM_INFO| Number of sph. ERI's stored on disk:
> > >        0
> > >   HFX_MEM_INFO| Number of sph. ERI's calculated on the fly:
> > >         0
> > >   HFX_MEM_INFO| Total memory consumption ERI's RAM [MiB]:
> > >   -449594
> > >   HFX_MEM_INFO| Whereof max-vals [MiB]:
> > >     17729
> > >   HFX_MEM_INFO| Total compression factor ERI's RAM:
> > >    -38.13
> > >   HFX_MEM_INFO| Total memory consumption ERI's disk [MiB]:
> > >        0
> > >   HFX_MEM_INFO| Total compression factor ERI's disk:
> > >     0.00
> > >   HFX_MEM_INFO| Size of density/Fock matrix [MiB]:
> > >      408
> > >   HFX_MEM_INFO| Size of buffers [MiB]:
> > >       25
> > >   HFX_MEM_INFO| Number of periodic image cells considered:
> > >       33
> > >   HFX_MEM_INFO| Est. max. program size after HFX  [MiB]:
> > >    31121
> > >
> > > The total memory consumption ERI's RAM is a negative number.
> > > Why it is a negative number?
> > > How can I know the real RAM consumption in this job?
> > > Thank you very much in advance.
> > > Best
> > > Geng
> > >
> > >
> > > --
> > > You received this message because you are subscribed to the Google
> > > Groups "cp2k" group.
> > > To unsubscribe from this group and stop receiving emails from it,
> > send
> > > an email to cp... at googlegroups.com
> > > <mailto:cp... at googlegroups.com>.
> > > To view this discussion on the web visit
> > >
> > 
> https://groups.google.com/d/msgid/cp2k/02d0eebb-898e-4536-913f-548448e11648n%40googlegroups.com
> > <
> https://groups.google.com/d/msgid/cp2k/02d0eebb-898e-4536-913f-548448e11648n%40googlegroups.com
> >
> > 
> > >
> > <
> https://groups.google.com/d/msgid/cp2k/02d0eebb-898e-4536-913f-548448e11648n%40googlegroups.com?utm_medium=email&utm_source=footer
> > <
> https://groups.google.com/d/msgid/cp2k/02d0eebb-898e-4536-913f-548448e11648n%40googlegroups.com?utm_medium=email&utm_source=footer
> >>.
> > 
> > 
> > -- 
> > Tiziano Müller
> > University of Zurich
> > Department of Chemistry
> > Winterthurerstrasse 190
> > CH-8057 Zürich
> > 
> > Tel: +41 44 63 54234
> > www.chem.uzh.ch <http://www.chem.uzh.ch>
> > tiz... at chem.uzh.ch
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> > Groups "cp2k" group.
> > To unsubscribe from this group and stop receiving emails from it, send 
> > an email to cp... at googlegroups.com 
> > <mailto:cp... at googlegroups.com>.
> > To view this discussion on the web visit 
> > 
> https://groups.google.com/d/msgid/cp2k/b579bdce-d28d-4551-b0b9-fb497a5efa3dn%40googlegroups.com 
> > <
> https://groups.google.com/d/msgid/cp2k/b579bdce-d28d-4551-b0b9-fb497a5efa3dn%40googlegroups.com?utm_medium=email&utm_source=footer
> >.
>
> -- 
> Tiziano Müller
> University of Zurich
> Department of Chemistry
> Winterthurerstrasse 190
> CH-8057 Zürich
>
> Tel: +41 44 63 54234
> www.chem.uzh.ch
> tiz... at chem.uzh.ch
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.cp2k.org/archives/cp2k-user/attachments/20210129/25aa575b/attachment.htm>
Previous message (by thread): [CP2K-user] [CP2K:14653] how could the RAM for ADMM be negative number
Next message (by thread): [CP2K-user] Error while attempting to read the Hessian file
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
More information about the CP2K-user mailing list