For ..in GetEnumerator Allocation

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

For ..in GetEnumerator Allocation

Ryan Joseph
As I understand the for..in loop GetEnumerator method is expected to create a new object each time it’s called and FPC destroys it later when the loos is finished. Can I retain the enumerator and just reset it in-between calls? I’d like to remove all these alloc/deallocs so I can use for..in more efficiently in tight loops.

Regards,
        Ryan Joseph

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: For ..in GetEnumerator Allocation

Michael Van Canneyt


On Wed, 4 Oct 2017, Ryan Joseph wrote:

> As I understand the for..in loop GetEnumerator method is expected to create a new object each time it’s called
> and FPC destroys it later when the loos is finished. Can I retain the enumerator and just reset it in-between calls?
> I’d like to remove all these alloc/deallocs so I can use for..in more efficiently in tight loops.

You can do so by overriding the newinstance and it's sister method of
your enumerator class.

As an alternative you can create an object enumeator.
It's simply allocated on the stack, and you can reset it in the enumerator
operator.

See the example in http://wiki.freepascal.org/for-in_loop
section: Using any identifiers instead of builtin MoveNext and Current

Michael.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: For ..in GetEnumerator Allocation

Marco van de Voort
In our previous episode, Michael Van Canneyt said:
> As an alternative you can create an object enumeator.
> It's simply allocated on the stack, and you can reset it in the enumerator
> operator.

Yup, or a record.  See e.g. http://www.stack.nl/~marcov/lightcontainers.zip

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: For ..in GetEnumerator Allocation

Graeme Geldenhuys-6
In reply to this post by Ryan Joseph
On 2017-10-04 09:41, Ryan Joseph wrote:
> I’d like to remove all these alloc/deallocs so I can use for..in more efficiently in tight loops.

I've had the same requirement, and also needed that functionality before
the for..in syntax existed in FPC. Take a look at my Iterator interface
and implementation. It allows you to move forward, backwards, reset,
filter data etc. You can hold on to the instance reference as long as
you like.

This code lives in the tiOPF project, but can be used outside of the
tiOPF project too (I do that often) - simply delete the iterator
implementations for TtiObjectList.


https://github.com/graemeg/tiopf/blob/tiopf2/Options/tiIteratorIntf.pas

https://github.com/graemeg/tiopf/blob/tiopf2/Options/tiIteratorImpl.pas

Some years ago I wrote a article for a magazine about this, and that is
when I implemented the code. You can still find the "Iterator" article
in the link below - and the accompanied source code too (though the
tiOPF code is newer).

   http://geldenhuys.co.uk/articles/


Regards,
   Graeme

--
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://fpgui.sourceforge.net/

My public PGP key:  http://tinyurl.com/graeme-pgp
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: For ..in GetEnumerator Allocation

Martok
In reply to this post by Michael Van Canneyt
Am 04.10.2017 um 11:26 schrieb Michael Van Canneyt:
> As an alternative you can create an object enumeator.
> It's simply allocated on the stack, and you can reset it in the enumerator
> operator.
That is by far the easiest solution (records need $modeswitch advancedrecords,
but are otherwise equivalent).

As an example, here's how I iterate over at TNodeSet returned by fcl-xml's XPath
engine: <https://pastebin.com/Zj2CLRbX>

The object is allocated on the stack and simply cleared/reset every time the
operator is executed.

--
Martok
Ceterum censeo b32079 esse sanandam.

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: For ..in GetEnumerator Allocation

Ryan Joseph
In reply to this post by Marco van de Voort

> On Oct 4, 2017, at 5:19 PM, Marco van de Voort <[hidden email]> wrote:
>
> Yup, or a record.  See e.g. http://www.stack.nl/~marcov/lightcontainers.zip

This seems like the simplest more efficient method. Does FPC just know it’s a record internally and not try to dealloc it? GetEnumerator seems like a magic method so I’m not sure how the memory is being managed but I assume the stack space for the method return value is reserved and I just fill in the values of the record to that location.

Regards,
        Ryan Joseph

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: For ..in GetEnumerator Allocation

Ryan Joseph
In reply to this post by Michael Van Canneyt

> On Oct 4, 2017, at 4:26 PM, Michael Van Canneyt <[hidden email]> wrote:
>
> You can do so by overriding the newinstance and it's sister method of
> your enumerator class.

Can you explain how this works or give an example? Not sure how these gets around the problem of alloc/dealloc for each iterator.

Regards,
        Ryan Joseph

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: For ..in GetEnumerator Allocation

Michael Van Canneyt


On Wed, 4 Oct 2017, Ryan Joseph wrote:

>
>> On Oct 4, 2017, at 4:26 PM, Michael Van Canneyt <[hidden email]> wrote:
>>
>> You can do so by overriding the newinstance and it's sister method of
>> your enumerator class.
>
> Can you explain how this works or give an example? Not sure how these gets around the problem of alloc/dealloc for each iterator.

Careful, the iterator is instantiated only once per loop ?

Newinstance allocates the memory for a new instance of the class.
By default this is GetMem(instanceSize).

What you can do is allocate somewhere a block of the correct size,
once, and always return this single block.

Of course, you need to know for sure that only 1 instance of your enumerator
will be active at any given point in your program.
If there are multiple, you could allocate a static array and allocate from that.

The FreeInstance procedure does a freemem, by default, but you can change
that to do nothing. (in case of an array, you mark the element unused)


Michael.

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: For ..in GetEnumerator Allocation

Ryan Joseph

> On Oct 4, 2017, at 10:03 PM, Michael Van Canneyt <[hidden email]> wrote:
>
> Newinstance allocates the memory for a new instance of the class.
> By default this is GetMem(instanceSize).

So you override the class method Newinstance in the enumerator class and return the same block of memory? Then when I override FreeInstance and do nothing I need to manually free the memory in the calling class I guess.

That’s an interesting solution and I didn’t know TObject did that even but using records still seems like a better solution and about as efficient (perhaps). I’ll try this tomorrow also to make sure I understand it. Thanks.

Regards,
        Ryan Joseph

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: For ..in GetEnumerator Allocation

Mattias Gaertner
In reply to this post by Ryan Joseph
On Wed, 4 Oct 2017 15:41:27 +0700
Ryan Joseph <[hidden email]> wrote:

> As I understand the for..in loop GetEnumerator method is expected to create a new object each time it’s called and FPC destroys it later when the loos is finished. Can I retain the enumerator and just reset it in-between calls? I’d like to remove all these alloc/deallocs so I can use for..in more efficiently in tight loops.

Here is an example how to use a global enumerator object:

type
  TMyEnumerator = class
  private
    FOwner: TComponent;
    FCurrent: TComponent;
  public
    procedure Init(Owner: TComponent);
    function MoveNext: boolean;
    property Current: TComponent read FCurrent;
    function GetEnumerator: TMyEnumerator;
  end;

  TForm1 = class(TForm)
  public
    function GetEnumerator: TMyEnumerator;
  end;

var
  Form2: TForm2;
  MyEnumerator: TMyEnumerator;

implementation

procedure TMyEnumerator.Init(Owner: TComponent);
begin
  FOwner:=Owner;
  FCurrent:=nil;
end;

function TMyEnumerator.MoveNext: boolean;
var
  i: Integer;
begin
  if FCurrent=nil then begin
    if FOwner.ComponentCount=0 then exit(false);
    FCurrent:=FOwner.Components[0];
  end else begin
    i:=FCurrent.ComponentIndex+1;
    if i>=FCurrent.Owner.ComponentCount then exit(false);
    FCurrent:=FCurrent.Owner.Components[i];
  end;
  Result:=true;
end;

function TMyEnumerator.GetEnumerator: TMyEnumerator;
begin
  Result:=Self;
end;

function TForm1.GetEnumerator: TMyEnumerator;
begin
  if MyEnumerator=nil then
    MyEnumerator:=TMyEnumerator.Create;
  MyEnumerator.Init(Self);
end;

finalization
  MyEnumerator.Free;
end.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal