Memory requirements report: what is the maximum amount of memory needed for the simulation?

cluster
memory

#1

I’m a little confused what the results of the memory report mean, and importantly how much memory I have to allocate to run the simulation. To clarify, I’m running this on a cluster, and I want to ask for an accurate amount of memory. So what is the maximum amount of memory that a simulation will need? Is it the largest of the bullet points, or is it the sum? For example if the (fake) memory report gives

Initialization and mesh: 1 GB
Running simulation: 2 GB
Data collection: 4 GB
Monitor data saved to fsp file: 3 GB

Then does it need 4 GB or does it need 10 GB?

Thanks,


#2

Hi @bpi,

This is a very good question. Definitely, don’t use the sum of the memory estimates because these correspond to different stages of the simulation.

The short answer to your question is that the largest of the four numbers is the maximum amount of memory that will be required. However, there are a couple of things you should consider when you are using multiple processes, especially if you are in a cluster (where the memory distribution is important):

  • These are estimates for total memory usage. Memory for each sub-process will be a fraction of this.
  • For the first two stages (initialization & mesh and running simulation) the memory will be distributed between all of the sub-processes.
  • In the data collection stage the master process collects the monitor data from all the other sub-processes. Therefore, to give a safe estimate, the memory reported is roughly double the actual amount of monitor data: one copy distributed between all processes and one full copy in the master process. In practice you probably don’t end up using that much memory so this is more of an overestimate.

One additional comment:
The first three estimates (initialization & mesh, running simulation and data collection) correspond to RAM requirements for each of those stages. The last one (monitor data saved to fsp file) corresponds to hard disk space.