Heap manager tuning

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Heap manager tuning

denisgolovan
Hi all

I am trying to debug an issue that seems to be related to FPC heap manager.
Test machine - Linux x64.

It looks like really large memory "overhead" compared to cmem manager.
Test load profile - 50 heavy objects with around 25 large (1Mb - 10Mb) dynamic arrays each.
Those arrays are created in threads.

That allocation takes 10Gb+ under FPC heap manager. Further allocation becomes hardly possible (I have 16Gb on-board).
Meanwhile, cmem allocates around 3Gb. So I can operate >3 times more object set.

I am pretty sure I don't have any memory leaks (heaptr is enabled all the time).

Any suggestions?
I am willing to sacrifice some allocation performance to decrease total memory consumption if it is necessary.

--
Regards,
Denis Golovan
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Heap manager tuning

Jonas Maebe-2
On 09/08/14 23:01, denisgolovan wrote:
> That allocation takes 10Gb+ under FPC heap manager. Further allocation becomes hardly possible (I have 16Gb on-board).
> Meanwhile, cmem allocates around 3Gb. So I can operate >3 times more object set.
>
> I am pretty sure I don't have any memory leaks (heaptr is enabled all the time).
>
> Any suggestions?

Try avoiding growing memory blocks small amounts at a time via
reallocmem or setlength, and instead grow them in bigger chuncks. This
will both increase the performance of your program and reduce the
internal memory fragmentation.


Jonas
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Heap manager tuning

denisgolovan
In reply to this post by denisgolovan
> On 09/08/14 23:01, denisgolovan wrote:
>
> Try avoiding growing memory blocks small amounts at a time via
> reallocmem or setlength, and instead grow them in bigger chuncks. This
> will both increase the performance of your program and reduce the
> internal memory fragmentation.
>
> Jonas

I don't have that allocation pattern here. Just a lot of large arrays allocating/deallocating (including local arrays for temporary results).
Any Java-like knobs to play with? :)

--
Regards,
Denis Golovan
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal