FPC + valgrind massif problems

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

FPC + valgrind massif problems

Michalis Kamburelis-3
Hi,

I wanted to debug where my program uses the most memory. (There are no
memory leaks, but I want to optimize memory usage on some large inputs.
As the code is quite large, simply guessing which part is responsible
becomes quite hard :) In the past, I happily used valgrind's massif tool
for such job, and it worked like a charm. But now, FPC 2.6.0 with
valgrind 3.7.0, it seems valgrind/massif has problems:

$ fpc -gv trivial_alloc.pas
$ valgrind --tool=massif ./trivial_alloc
....
--7286-- VALGRIND INTERNAL ERROR: Valgrind received a signal 11
(SIGSEGV) - exiting
--7286-- si_code=1;  Faulting address: 0xFFFFFFF7;  sp: 0x6298914c
valgrind: the 'impossible' happened:
    Killed by fatal signal

(Full output and trivial_alloc.pas attached.)

The code is compiled with -gv option. It works very fine with valgrind's
callgrind tool. So I suspect this is a problem with valgrind,
specifically massif, not related to FPC. Also, it seems to fail the same
way with older/newer FPC versions (I tried 2.2.4, 2.4.4, 2.6.0, and
2.7.1 from today 2012-05-11). Probably, some valgrind version broke
massif for FPC binaries.

So, questions:

1. I want to report this as massif's bug. Any ideas what more
information should I provide to valgrind's developers? I don't know if
they are familiar with FPC. And simple C examples work fine, so it's
probably something specific in FPC binaries.

- Maybe this problem is already reported?

- Maybe there's a workaround? IOW, can someone successfully use massif
with FPC programs?

2. Is there any other reliable memory profiler for FPC? As you can swap
memory manager in Pascal, and FPC can give you a backtrace, I suppose
it's quite possible to just write a unit that works between your program
and standard memory manager, and tracks memory usage this way. This will
only measure memory use of Pascal code (external libs don't go through
FPC memory manager), but that would be very fine for me --- I know
(well, I'm 99% sure) that my Pascal code is the main memory eater here.

I didn't find any (good and maintained) memory profiler mentioned on
http://wiki.lazarus.freepascal.org/Profiling#Memory_Profiling . (I tried
FpProfiler, but it depends on inserting instrumentation code into Pascal
sources. That's actually a very nice idea, but it means it's closely
tied to what fcl-passrc can handle. And current fcl-passrc still lacks a
couple of features (reported). Also FpProfiler seems to use an older,
local fcl-passrc version.)

Thanks,
Michalis

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal

massif_output.txt (2K) Download Attachment
trivial_alloc.pas (55 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: FPC + valgrind massif problems

Sven Barth-2
Am 11.05.2012 02:40, schrieb Michalis Kamburelis:

> 2. Is there any other reliable memory profiler for FPC? As you can swap
> memory manager in Pascal, and FPC can give you a backtrace, I suppose
> it's quite possible to just write a unit that works between your program
> and standard memory manager, and tracks memory usage this way. This will
> only measure memory use of Pascal code (external libs don't go through
> FPC memory manager), but that would be very fine for me --- I know
> (well, I'm 99% sure) that my Pascal code is the main memory eater here.
>
> I didn't find any (good and maintained) memory profiler mentioned on
> http://wiki.lazarus.freepascal.org/Profiling#Memory_Profiling . (I tried
> FpProfiler, but it depends on inserting instrumentation code into Pascal
> sources. That's actually a very nice idea, but it means it's closely
> tied to what fcl-passrc can handle. And current fcl-passrc still lacks a
> couple of features (reported). Also FpProfiler seems to use an older,
> local fcl-passrc version.)

You could try whether heaptrc is of any help for you (see here:
http://www.freepascal.org/docs-html/rtl/heaptrc/usage.html ). If not it
might at least provide a starting point if you should decide to write a
custom memory profiler. Otherwise I don't know of any FPC based memory
profilers.

Regards,
Sven
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

RE : [fpc-pascal] FPC + valgrind massif problems

Ludo Brands
In reply to this post by Michalis Kamburelis-3
> Hi,
>
> I wanted to debug where my program uses the most memory.
> (There are no
> memory leaks, but I want to optimize memory usage on some
> large inputs.
> As the code is quite large, simply guessing which part is responsible
> becomes quite hard :) In the past, I happily used valgrind's
> massif tool
> for such job, and it worked like a charm. But now, FPC 2.6.0 with
> valgrind 3.7.0, it seems valgrind/massif has problems:
>
> $ fpc -gv trivial_alloc.pas
> $ valgrind --tool=massif ./trivial_alloc
> ....
> --7286-- VALGRIND INTERNAL ERROR: Valgrind received a signal 11
> (SIGSEGV) - exiting
> --7286-- si_code=1;  Faulting address: 0xFFFFFFF7;  sp: 0x6298914c
> valgrind: the 'impossible' happened:
>     Killed by fatal signal
>
> (Full output and trivial_alloc.pas attached.)
>
> The code is compiled with -gv option. It works very fine with
> valgrind's
> callgrind tool. So I suspect this is a problem with valgrind,
> specifically massif, not related to FPC. Also, it seems to
> fail the same
> way with older/newer FPC versions (I tried 2.2.4, 2.4.4, 2.6.0, and
> 2.7.1 from today 2012-05-11). Probably, some valgrind version broke
> massif for FPC binaries.
>
> So, questions:
>
> 1. I want to report this as massif's bug. Any ideas what more
> information should I provide to valgrind's developers? I
> don't know if
> they are familiar with FPC. And simple C examples work fine, so it's
> probably something specific in FPC binaries.
>
> - Maybe this problem is already reported?
>
> - Maybe there's a workaround? IOW, can someone successfully
> use massif
> with FPC programs?
>

This is known problem not related to fpc. Run

valgrind --tool=massif --run-libc-freeres=no  ./trivial_alloc

Ludo

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: FPC + valgrind massif problems

Michalis Kamburelis-3
In reply to this post by Sven Barth-2
Sven Barth wrote:
> You could try whether heaptrc is of any help for you (see here:
> http://www.freepascal.org/docs-html/rtl/heaptrc/usage.html ). If not it
> might at least provide a starting point if you should decide to write a
> custom memory profiler. Otherwise I don't know of any FPC based memory
> profilers.
>

Thanks. I'm using HeapTrc all the time, that's how I know that my
program has no memory leaks and that's how I know FPC memory manager can
be reliably switched and a backtrace for each allocation can be captured :)

My needs are a little larger than what HeapTrc currently does, I want a
memory profiler --- where you can see at a glance what is the largest
memory consumer inside your code and such. Things like
http://lazarusroad.blogspot.com/2007/09/zlibar-memory-behavior-compressing-many.html 
. Like calling HeapTrc.DumpHeap all the time, and tracking where the
large increases happened. Valgrind+massif used to be very helpful for this.

Michalis
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: RE : [fpc-pascal] FPC + valgrind massif problems

Michalis Kamburelis-3
In reply to this post by Ludo Brands
Ludo Brands wrote:
>> - Maybe there's a workaround? IOW, can someone successfully
>> use massif
>> with FPC programs?
>>
>
> This is known problem not related to fpc. Run
>
> valgrind --tool=massif --run-libc-freeres=no  ./trivial_alloc
>

Fantastic, many thanks, it works like a charm now! :) Both on trivial
tests, as well as my large 3D viewer (view3dscene). Thank you! Oh, and I
see this hint was just added to the
http://wiki.lazarus.freepascal.org/Profiling too.

Michalis
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal