[CP2K-user] Memory usage in hybrid DFT calculations

[Maxime Van den Bossche]

Dear CP2K developers,

I have started to benchmark CP2K on our clusters, and there are a few 
things that I don't
fully understand about the memory consumption of hybrid DFT calculations.

1. What needs to be considered in setting MAX_MEMORY 
<https://manual.cp2k.org/trunk/CP2K_INPUT/FORCE_EVAL/DFT/XC/HF/MEMORY.html#MAX_MEMORY>, 
so as to use as much RAM as possible?
MAX_MEMORY being the maximal amount of memory per MPI rank, in MiB, that 
the HFX module
is allowed to use.

At first, I thought it  would simply involve dividing the amount of RAM per 
(multi-core)
compute node by the number of MPI ranks per node, and then subtracting a 
modest amount
that will be needed by the rest of the program.

In the attached down-scaled version of the LiH-HFX benchmark, for example, 
the GGA prerun
on a 28-core node utilizes circa 250 MiB per MPI rank. In a subsequent HFX 
calculation
on the same node with MAX_MEMORY = 1500 MiB, the HFX module uses all of its 
allowed memory,
which is fine. But the job's total memory usage amounts to around 3000 MiB 
per MPI rank,
as given by CP2K's "max memory usage/rank" output and a check with "htop".

This is quite larger than the 1500+250=1750 MiB per MPI rank that I naively 
expected.
So is there a more sophisticated "rule of thumb" for predicting the total 
memory usage?
Or is this not even how CP2K is supposed to behave?


2. What should I expect from using multiple OpenMP threads per MPI rank in 
terms of memory
consumption? Suppose that MAX_MEMORY is such that 50% of the ERIs cannot be 
stored and
need to be (re)calculated on-the-fly, for OMP_NUM_THREADS=1. Suppose then 
that I set
OMP_NUM_THREADS=2, use half the number of MPI ranks, and double the 
MAX_MEMORY,
to arrive at the same memory usage by the HFX module. Should this then 
allow all ERIs to be
stored in memory?


Best,
Maxime
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.cp2k.org/archives/cp2k-user/attachments/20200602/a01c7142/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: inputs.zip
Type: application/zip
Size: 5419481 bytes
Desc: not available
URL: <https://lists.cp2k.org/archives/cp2k-user/attachments/20200602/a01c7142/attachment.zip>