Error: Local variables size exceeds supported limit

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Error: Local variables size exceeds supported limit

Fabio Luis Girardi
Hi All!

I'm trying to use the FPC 3.2.0 beta + Lazarus 2.0.1 with fpCEF3. But when I try compile this package with FPC 3.2.0 I got this error:

Error: Local variables size exceeds supported limit

But the same package compiles fine with FPC 3.0.4 and 3.0.5.

Should I fill a bug report?

Ahh, in time:

FPC 3.2.0 beta buit with sources updated today (07 - mar - 2019) 64 bits
Linux Mint 19.1 x86_64
Lazarus 2.0.1 
fpCEF3 v 3029.1

Sorry if I forget to include some information.

--
The best regards,

Fabio Luis Girardi
PascalSCADA Project
http://sourceforge.net/projects/pascalscada
http://www.pascalscada.com

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Error: Local variables size exceeds supported limit

Yuriy Sydorov
On 07.03.2019 22:32, Fabio Luis Girardi wrote:
> Hi All!
>
> I'm trying to use the FPC 3.2.0 beta + Lazarus 2.0.1 with fpCEF3. But when I try compile this package with FPC 3.2.0 I
> got this error:
>
> Error: Local variables size exceeds supported limit

It would be helpful if you provide a code snippet of a function where the error is triggered. Especially the part with
local vars declarations, including all referenced custom types.

Yuriy.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Error: Local variables size exceeds supported limit

Jonas Maebe-3
In reply to this post by Fabio Luis Girardi
On 07/03/2019 21:32, Fabio Luis Girardi wrote:

> I'm trying to use the FPC 3.2.0 beta + Lazarus 2.0.1 with fpCEF3. But
> when I try compile this package with FPC 3.2.0 I got this error:
>
> Error: Local variables size exceeds supported limit
>
> But the same package compiles fine with FPC 3.0.4 and 3.0.5.

FPC 3.2.0 limits the total size of local variables and parameter space
for a single procedure/function to 2GB on 32 bit and 64 bit targets. On
FPC 3.0.4 the limit was the maximum size supported by the address space.

The previous size check for the locals was not working correctly
according to the commit message, but I don't know why this was changed
like that for 64 bit targets.


Jonas
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Error: Local variables size exceeds supported limit

Fabio Luis Girardi
Hi Jonas!

Thanks by your feedback. Do you think that should I fill a bug report?

The datatype that throws this error is this (cef3types.pas:2416):

TCefCompositionUnderlineArray = array[0..(High(Integer) div SizeOf(TCefCompositionUnderline)) - 1] of TCefCompositionUnderline;   

The TCefCompositionUnderline is a record with 20 bytes of size (5x32 bits fields/vars into this record). So only this var exceeds this limit on 32 bits systems.

The error is throw on this procedure, in cef3ref.pas:1574

procedure TCefBrowserHostRef.ImeSetComposition(const text: ustring; underlinesCount: TSize;
  underlines: TCefCompositionUnderlineArray; const replacementRange, selectionRange: TCefRange);

I think that 2GB on 64 bits a very conservative limit.

Jonas, do you remember the revision number where this was changed?



Em sáb, 9 de mar de 2019 às 07:59, Jonas Maebe <[hidden email]> escreveu:
On 07/03/2019 21:32, Fabio Luis Girardi wrote:

> I'm trying to use the FPC 3.2.0 beta + Lazarus 2.0.1 with fpCEF3. But
> when I try compile this package with FPC 3.2.0 I got this error:
>
> Error: Local variables size exceeds supported limit
>
> But the same package compiles fine with FPC 3.0.4 and 3.0.5.

FPC 3.2.0 limits the total size of local variables and parameter space
for a single procedure/function to 2GB on 32 bit and 64 bit targets. On
FPC 3.0.4 the limit was the maximum size supported by the address space.

The previous size check for the locals was not working correctly
according to the commit message, but I don't know why this was changed
like that for 64 bit targets.


Jonas
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal


--
The best regards,

Fabio Luis Girardi
PascalSCADA Project
http://sourceforge.net/projects/pascalscada
http://www.pascalscada.com

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Error: Local variables size exceeds supported limit

Jonas Maebe-3
On 09/03/2019 13:06, Fabio Luis Girardi wrote:
> Thanks by your feedback. Do you think that should I fill a bug report?

That's up to you.

> I think that 2GB on 64 bits a very conservative limit.

2GB stack space on is very large on any platform. Normally the heap is
used for large allocations.

This probably mostly stems from 32 (and 16) bit platforms, where the
issue was that it was hard to have large contiguous address ranges in
the virtual memory space. OTOH,
* several systems still support limiting the stack size separately from
the rest of the program's memory to more quickly catch infinite
recursion errors. Thread stacks are also often limited in size.
* The offsets you can encode in a single instruction are always limited.
E.g. on x86-64, offsets relative to the stack/frame pointer are limited
to 2GB. So if you allocate more stack space than that, accessing local
variables that are outside this addressable range will become slower
because you'll need extra instructions (while local variables are often
accessed a lot)

> Jonas, do you remember the revision number where this was changed?

r39916


Jonas

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Error: Local variables size exceeds supported limit

Fabio Luis Girardi


Em Sáb, 9 de mar de 2019 09:34, Jonas Maebe <[hidden email]> escreveu:


r39916

Thanks. How fpcef3 isn't my project and I don't understood how it works (I don't understand why a very large array is needed), I'll switch back to a previous revision number and wait the FPC 3.2 to be released. After that I'll see if I report a bug to FPC or fpCEF3 team.

The best regards,

Fabio

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Error: Local variables size exceeds supported limit

Yuriy Sydorov
In reply to this post by Jonas Maebe-3
On 09.03.2019 14:33, Jonas Maebe wrote:

> On 09/03/2019 13:06, Fabio Luis Girardi wrote:
>> Thanks by your feedback. Do you think that should I fill a bug report?
>
> That's up to you.
>
>> I think that 2GB on 64 bits a very conservative limit.
>
> 2GB stack space on is very large on any platform. Normally the heap is used for large allocations.
>
> This probably mostly stems from 32 (and 16) bit platforms, where the issue was that it was hard to have large contiguous
> address ranges in the virtual memory space. OTOH,
> * several systems still support limiting the stack size separately from the rest of the program's memory to more quickly
> catch infinite recursion errors. Thread stacks are also often limited in size.
> * The offsets you can encode in a single instruction are always limited. E.g. on x86-64, offsets relative to the
> stack/frame pointer are limited to 2GB. So if you allocate more stack space than that, accessing local variables that
> are outside this addressable range will become slower because you'll need extra instructions (while local variables are
> often accessed a lot)

FYI on 64-bit Windows the stack size limit is hard coded to 1GB. The same 1GB limit applies on 32-bit Windows. Probably
Linux can be configured for larger stack.

But I agree with Jonas, that using huge local vars is pointless and inefficient.

So it is not the FPC issue, but the library design issue.

Yuriy.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Error: Local variables size exceeds supported limit

Yuriy Sydorov
In reply to this post by Fabio Luis Girardi
On 09.03.2019 14:06, Fabio Luis Girardi wrote:

> The datatype that throws this error is this (cef3types.pas:2416):
>
> TCefCompositionUnderlineArray = array[0..(High(Integer) div SizeOf(TCefCompositionUnderline)) - 1] of
> TCefCompositionUnderline;
>
> The TCefCompositionUnderline is a record with 20 bytes of size (5x32 bits fields/vars into this record). So only this
> var exceeds this limit on 32 bits systems.
>
> The error is throw on this procedure, in cef3ref.pas:1574
>
> procedure TCefBrowserHostRef.ImeSetComposition(const text: ustring; underlinesCount: TSize;
>    underlines: TCefCompositionUnderlineArray; const replacementRange, selectionRange: TCefRange);
>
> I think that 2GB on 64 bits a very conservative limit.

Passing a 2GB array _by value_ as the "underlines" parameter for the procedure "ImeSetComposition" is totally wrong.
A pointer to an array of "underlinesCount" size should be passed here.

You should file a bug report for this library.

Yuriy.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Error: Local variables size exceeds supported limit

Jonas Maebe-3
In reply to this post by Fabio Luis Girardi
On 2019-03-10 10:22, Yuriy Sydorov wrote:
> FYI on 64-bit Windows the stack size limit is hard coded to 1GB. The
> same 1GB limit applies on 32-bit Windows. Probably Linux can be
> configured for larger stack.

Linux needs to be specially configured to forbid a larger stack.

> But I agree with Jonas, that using huge local vars is pointless and
> inefficient.
>
> So it is not the FPC issue, but the library design issue.

I disagree. We should not break things unless it is required. On Win64
we should probably give an error when the stack allocation size is
larger than 1GB, but on other 64 bit platforms we should allow up to
high(int64).


Jonas
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Error: Local variables size exceeds supported limit

Yuriy Sydorov
On 10.03.2019 12:07, Jonas Maebe wrote:

> On 2019-03-10 10:22, Yuriy Sydorov wrote:
>> FYI on 64-bit Windows the stack size limit is hard coded to 1GB. The
>> same 1GB limit applies on 32-bit Windows. Probably Linux can be
>> configured for larger stack.
>
> Linux needs to be specially configured to forbid a larger stack.
>
>> But I agree with Jonas, that using huge local vars is pointless and inefficient.
>>
>> So it is not the FPC issue, but the library design issue.
>
> I disagree. We should not break things unless it is required. On Win64 we should probably give an error when the stack
> allocation size is larger than 1GB, but on other 64 bit platforms we should allow up to high(int64).

If I recall correctly, currently the compiler uses longints as stack offsets internally in several places. That's why
the limit is 2GB even for 64-bit targets.

The issue in this tread is caused by the incorrect declaration of procedure's params. See my other answer. In this case
2GB of stack is not needed at all.
I suppose there are no real world apps which require 2GB of stack for local vars or parameters. Even if we consider
future apps it is useless to support 64-bit stack offsets for local vars of a single procedure.

Yuriy.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Error: Local variables size exceeds supported limit

Jonas Maebe-3
On 10/03/2019 11:40, Yuriy Sydorov wrote:
> If I recall correctly, currently the compiler uses longints as stack
> offsets internally in several places. That's why the limit is 2GB even
> for 64-bit targets.

That would be compiler bugs that need to be fixed. Ideally, the commit
message would also have mentioned that :)

> The issue in this tread is caused by the incorrect declaration of
> procedure's params. See my other answer. In this case 2GB of stack is
> not needed at all.

Indeed.

> I suppose there are no real world apps which require 2GB of stack for
> local vars or parameters. Even if we consider future apps it is useless
> to support 64-bit stack offsets for local vars of a single procedure.

If the architecture and OS support it, I see no reason to forbid it.
It's always possible to add a hint about efficiency in case the size of
the locals grows beyond offsets that can be embedded in a single
instruction, although this hint would trigger already at 32KB of locals
on e.g. PowerPC.


Jonas
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Error: Local variables size exceeds supported limit

Karoly Balogh (Charlie/SGR)
Hi,

On Sun, 10 Mar 2019, Jonas Maebe wrote:

> It's always possible to add a hint about efficiency in case the size of
> the locals grows beyond offsets that can be embedded in a single
> instruction, although this hint would trigger already at 32KB of locals
> on e.g. PowerPC.

I'm all for such a hint in any case, even if that limit is quite low on
many archs. It's useful also on m68k, where the "efficiency limit" is also
32K. One who ports code across many CPU archs, especially slower ones
(embedded anyone?) such hint could be important to discover inefficiency
hotspots.

Charlie
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Error: Local variables size exceeds supported limit

Yuriy Sydorov
In reply to this post by Jonas Maebe-3
On 10.03.2019 13:48, Jonas Maebe wrote:
> On 10/03/2019 11:40, Yuriy Sydorov wrote:
>> If I recall correctly, currently the compiler uses longints as stack offsets internally in several places. That's why
>> the limit is 2GB even for 64-bit targets.
>
> That would be compiler bugs that need to be fixed. Ideally, the commit message would also have mentioned that :)

Sure :)
I've checked it right now. Indeed the compiler uses longint values as the stack offsets here and there. It need to be
fixed in order to support 64-bit offsets. Earlier versions of FPC simply does not contain the proper checks and does not
throw en error for local vars exceeding 2GB. But the resulting generated code is broken anyway. So no regression here.

>> I suppose there are no real world apps which require 2GB of stack for local vars or parameters. Even if we consider
>> future apps it is useless to support 64-bit stack offsets for local vars of a single procedure.
>
> If the architecture and OS support it, I see no reason to forbid it. It's always possible to add a hint about efficiency
> in case the size of the locals grows beyond offsets that can be embedded in a single instruction, although this hint
> would trigger already at 32KB of locals on e.g. PowerPC.

Sure, if someone would volunteer to implement this :)

ARM also requires several instructions for offsets larger than 4KB and this is normal. Therefore the hint should be
issued only for huge offsets related to the bitness of a target CPU. The warning would be even better, since in most
cases usage of huge local vars/params means a bug or not desired effect, such as discussed in this topic.

Yuriy.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal