How to reduce memory usage of the MathKernel.exe?

[Janneke Baetens]

Hi all,

Currently, I'm using Mathematica 6.0 on a Windows XP machine which has 2 Gb of RAM to perform massive numerical computations. It all works great but sometimes the Kernel needs to shut because it's out of memory. I can follow this memory depletion in the Windows task manager.

First, I thought this would be caused be wanting to store too much variables consisting of millions of elements. Because of that I tried to clear the memory after each individual computation by using the command ClearSystemCache, though this doesn't seem to affect the memory needed by the MathKernel. Even if I used that command and clear all variables the kernel keeps on needing the same amount of memory as I can see in the Windows Task Manager.

Only by killing the kernel and afterwards starting a new one it's possible to execute a new massive computation. I think there must be a more subtle manner to reduce the memory use by the Kernel but so far I didn't find any.

I wonder whether anyone on this list has a good suggestion.

Best regards,

Jan Baetens
Ghent University

[Jens-Peer Kuska]

Hi,

a) you should set th global variable $HistoryLength=0
b) you should call Share[] periodical, but not to often
c) it depends on your computation if ClearSystemCache[] helps, in
numerical calculations I would not expect an huge effect.

Regards
Jens

Post by Janneke Baetens
Hi all,
Currently, I'm using Mathematica 6.0 on a Windows XP machine which has 2 Gb of RAM to perform massive numerical computations. It all works great but sometimes the Kernel needs to shut because it's out of memory. I can follow this memory depletion in the Windows task manager.
First, I thought this would be caused be wanting to store too much variables consisting of millions of elements. Because of that I tried to clear the memory after each individual computation by using the command ClearSystemCache, though this doesn't seem to affect the memory needed by the MathKernel. Even if I used that command and clear all variables the kernel keeps on needing the same amount of memory as I can see in the Windows Task Manager.
Only by killing the kernel and afterwards starting a new one it's possible to execute a new massive computation. I think there must be a more subtle manner to reduce the memory use by the Kernel but so far I didn't find any.
I wonder whether anyone on this list has a good suggestion.
Best regards,
Jan Baetens
Ghent University

[Bill Rowe]

Post by Janneke Baetens
Currently, I'm using Mathematica 6.0 on a Windows XP machine which
has 2 Gb of RAM to perform massive numerical computations. It all
works great but sometimes the Kernel needs to shut because it's out
of memory. I can follow this memory depletion in the Windows task
manager.
First, I thought this would be caused be wanting to store too much
variables consisting of millions of elements. Because of that I
tried to clear the memory after each individual computation by using
the command ClearSystemCache, though this doesn't seem to affect the
memory needed by the MathKernel. Even if I used that command and
clear all variables the kernel keeps on needing the same amount of
memory as I can see in the Windows Task Manager.

If you look at the documentation for ClearSystemCache you will
find it is not intended to minimize memory usage. Specifically,
it clears internal caches of data. I doubt it removes those
caches or frees memory for other usage. Most likely, Mathematica
sets up cache sizes as one of the start up actions.

To free memory associated with a given variable, use Clear or
Remove. But not while Remove[expr] will delete that expression
it may not completely free all of the memory associated with it.
That is because it likely is referenced as part of the session
history. To minimize memory usage it is essential to do

$HistoryLength = 0

By default, this is set to infinity which tracks all
computations made, making it possible to re-do any when needed.

Other things to look at are Share and tutorial/MemoryManagement.

If none of the above is sufficient, then likely you will need to
re-work you computation so as to do only part of it at a time.