Feature announcement: Dynamic array extensions

classic Classic list List threaded Threaded
94 messages Options
12345
Reply | Threaded
Open this post in threaded view
|

Re: Feature announcement: Dynamic array extensions

Mark Morgan Lloyd-5
On 02/06/18 13:00, Mattias Gaertner wrote:
>> Sven Barth via fpc-pascal <[hidden email]> hat am 2. Juni 2018 um 09:42 geschrieben: > > denisgolovan < [hidden email]> schrieb am Sa., 2. Juni 2018, 09:18: > > @Sven > >  Please reconsider "+" operator for arrays if your changes really forbid to overload operators for arrays now.> >  > It wasn't me who implemented that part. I personally had planned to do it with a warning for existing overloads, but Florian beat me to it and implemented it this way. Though when asked by me he did say that we'll wait and see if people complain... So *maybe* we'll change this.

> +1 for the possibility to overload this operator.

It would be reasonable to assume that the predefined + might be
substantially more efficient than a programmer-defined one could be.

--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Feature announcement: Dynamic array extensions

Mattias Gaertner
On Sun, 3 Jun 2018 11:34:33 +0000
Mark Morgan Lloyd <[hidden email]> wrote:

> On 02/06/18 13:00, Mattias Gaertner wrote:
> >> Sven Barth via fpc-pascal <[hidden email]> hat am 2. Juni 2018 um 09:42 geschrieben: > > denisgolovan < [hidden email]> schrieb am Sa., 2. Juni 2018, 09:18: > > @Sven > >  Please reconsider "+" operator for arrays if your changes really forbid to overload operators for arrays now.> >  > It wasn't me who implemented that part. I personally had planned to do it with a warning for existing overloads, but Florian beat me to it and implemented it this way. Though when asked by me he did say that we'll wait and see if people complain... So *maybe* we'll change this.  
>
> > +1 for the possibility to overload this operator.  
>
> It would be reasonable to assume that the predefined + might be
> substantially more efficient than a programmer-defined one could be.

Yes, that's one of the reasons I vote for keeping the new feature
and allow to overload the operator.

Mattias
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Feature announcement: Dynamic array extensions

Florian Klämpfl
Am 03.06.2018 um 14:27 schrieb Mattias Gaertner:

> On Sun, 3 Jun 2018 11:34:33 +0000
> Mark Morgan Lloyd <[hidden email]> wrote:
>
>> On 02/06/18 13:00, Mattias Gaertner wrote:
>>>> Sven Barth via fpc-pascal <[hidden email]> hat am 2. Juni 2018 um 09:42 geschrieben: > > denisgolovan < [hidden email]> schrieb am Sa., 2. Juni 2018, 09:18: > > @Sven > >  Please reconsider "+" operator for arrays if your changes really forbid to overload operators for arrays now.> >  > It wasn't me who implemented that part. I personally had planned to do it with a warning for existing overloads, but Florian beat me to it and implemented it this way. Though when asked by me he did say that we'll wait and see if people complain... So *maybe* we'll change this.
>>
>>> +1 for the possibility to overload this operator.
>>
>> It would be reasonable to assume that the predefined + might be
>> substantially more efficient than a programmer-defined one could be.
>
> Yes, that's one of the reasons I vote for keeping the new feature
> and allow to overload the operator.

There is still concat which provides exactly the same functionality.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Feature announcement: Dynamic array extensions

Mattias Gaertner
On Sun, 3 Jun 2018 14:59:46 +0200
Florian Klämpfl <[hidden email]> wrote:

>[...]
> > Yes, that's one of the reasons I vote for keeping the new feature
> > and allow to overload the operator.  
>
> There is still concat which provides exactly the same functionality.

True.
Same as in Delphi.

Is there a problem having the intrinsic '+' and custom overloads for
arrays?

Mattias
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Feature announcement: Dynamic array extensions

Florian Klämpfl
Am 03.06.2018 um 15:15 schrieb Mattias Gaertner:

> On Sun, 3 Jun 2018 14:59:46 +0200
> Florian Klämpfl <[hidden email]> wrote:
>
>> [...]
>>> Yes, that's one of the reasons I vote for keeping the new feature
>>> and allow to overload the operator.
>>
>> There is still concat which provides exactly the same functionality.
>
> True.
> Same as in Delphi.
>
> Is there a problem having the intrinsic '+' and custom overloads for
> arrays?
>

This is so far not allowed: overriding intrinsic operators with custom overloads, so it would be another language exception.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Feature announcement: Dynamic array extensions

Marco van de Voort
In reply to this post by Ryan Joseph
In our previous episode, Ryan Joseph said:
> > but IMO it is strange to use + for string concatenation;
> > I always have bad feelings about this.
>
> It?s actually pretty common to use the phrase ?adding two things together?
> in English.  For example: ?I?m going to add some more strawberries to my
> milkshake?, or, ?add this to my list of things to do?.  Maybe it?s a
> language thing but adding is certainly pretty typical for the idea of
> concatenation.

But nobody + an item to their item list. The operator is "+" not "add".
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Feature announcement: Dynamic array extensions

Ryan Joseph


> On Jun 3, 2018, at 9:07 PM, Marco van de Voort <[hidden email]> wrote:
>
> But nobody + an item to their item list. The operator is "+" not "add".

What other operators could be used then? There’s only so many characters on the keyboard after all.

My only interest is because it’s fast to type and looks nice. Swift allows you to overload any operator in the unicode set so we could make up our own crazy stuff. :)

list • 1; // list.Add(1);

Regards,
        Ryan Joseph

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Feature announcement: Dynamic array extensions

Michael Van Canneyt


On Sun, 3 Jun 2018, Ryan Joseph wrote:

>
>
>> On Jun 3, 2018, at 9:07 PM, Marco van de Voort <[hidden email]> wrote:
>>
>> But nobody + an item to their item list. The operator is "+" not "add".
>
> What other operators could be used then? There’s only so many characters on the keyboard after all.
>
> My only interest is because it’s fast to type and looks nice. Swift allows you to overload any operator in the unicode set so we could make up our own crazy stuff. :)
>
> list • 1; // list.Add(1);
That is a horrible idea.

Think of it:
You will read someone else's code, and have absolutely no clue what it does.

Michael.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Feature announcement: Dynamic array extensions

Ryan Joseph


> On Jun 3, 2018, at 10:36 PM, Michael Van Canneyt <[hidden email]> wrote:
>
> That is a horrible idea.
>
> Think of it: You will read someone else's code, and have absolutely no clue what it does.

For sure. Apple even showed how you can use emojis as operators. :) The best I can say is some interesting operators could be discovered and standardized if given enough time and experimentation. Math people probably has some fun ideas that could work out in the end.

Regards,
        Ryan Joseph

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Feature announcement: Dynamic array extensions

Free Pascal - General mailing list
In reply to this post by Bernd Oppolzer
Bernd Oppolzer <[hidden email]> schrieb am So., 3. Juni 2018, 11:56:



Am 02.06.2018 um 15:14 schrieb Sven Barth via fpc-pascal:
Mark Morgan Lloyd <[hidden email]> schrieb am Sa., 2. Juni 2018, 10:53:
However as Dennis points out + is also essential for vector operations.
Perhaps either leaving it to the programmer to define what's needed
would be the best approach, or alternatively splitting dynamic arrays
into mathematical vectors and non-mathematical collections. Or relaxing
the requirement that only predefined operators can be redefined, so that
something like _ could be used for concatenation.

That needlessly complicates the parser as the compiler still needs to know them and they also need to be part of its operator precedence rules. Don't complicate the language for nothing! And in the end operator overloads are one of the best examples for syntactic sugar as you can easily achieve the same result with functions and methods. 

Regards, 
Sven 


This is somehow off topic of course,
but IMO it is strange to use + for string concatenation;
I always have bad feelings about this. This whole thread would
not exist, if FreePascal had gone another direction like PL/1, for example,
where the string concatenation operator is ||
(and DB2, and - probably - other SQL dialects).

FPC inherited the +-operator for concatenation from the base language: Pascal. So there simply was no other route to take (not that anyone would have thought to take a different route). 


Where does this + for string concat come from?

Ask Wirth, he is the one who invented Pascal... 

Regards, 
Sven 

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Feature announcement: Dynamic array extensions

Free Pascal - General mailing list
In reply to this post by Ryan Joseph
Ryan Joseph <[hidden email]> schrieb am So., 3. Juni 2018, 10:22:


> On Jun 3, 2018, at 3:15 PM, Sven Barth via fpc-pascal <[hidden email]> wrote:
>
> Because operator overloads are static methods not normal methods.

I don’t understand. Why aren’t both those variants possible? They’re both static I believe. The first is mutating the left side value and the second is a clone (the most common operation for += )

class operator + (var left: TIntArray; const right: T);
class operator + (constref left: TIntArray; const value: T): TIntArray;

Let me make one thing perfectly clear: the C-operators (+=, -=, *=, /=) do not exist as explicit operators in FPC. They are merely syntactic sugar for the more expressive variants and converted to those as soon as possible. 

Regards,
Sven 

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Feature announcement: Dynamic array extensions

Mark Morgan Lloyd-5
In reply to this post by Ryan Joseph
On 03/06/18 14:30, Ryan Joseph wrote:
>> On Jun 3, 2018, at 9:07 PM, Marco van de Voort <[hidden email]> wrote:> > But nobody + an item to their item list. The operator is "+" not "add".
> What other operators could be used then? There’s only so many characters on the keyboard after all.
> My only interest is because it’s fast to type and looks nice. Swift allows you to overload any operator in the unicode set so we could make up our own crazy stuff. :)
> list • 1; // list.Add(1);

All of the predefined ones have the problem of clashing with arithmetic
use in numeric vectors. I'd note that Perl uses . but my preference
would be _ (followed by whitespace, or at least not by something that
would make it look like a variable name).

People with deeply masochistic tendencies might appreciate the link
below, if unavailable Google might have a cache.

http://www.dlugosz.com/Perl6/web/APL.html

--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]


http://webcache.googleusercontent.com/search?q=cache:ag72S4uAW30J:http://www.dlugosz.com/Perl6/web/APL.html%2B%22provide+SIMD+processing+without+writing+explicit+loops,+and+APL+also+has+the+feature%22&client=firefox-b&hl=en&ct=clnk

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Feature announcement: Dynamic array extensions

denisgolovan
Having worked with APL for almost 6 years and continuing with Q/KDB, I would say APL syntax is much more readable than Perl 6.

Though being able to modify language like that is really impressive.

BR,
Denis
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Feature announcement: Dynamic array extensions

Marco van de Voort
In reply to this post by Free Pascal - General mailing list
In our previous episode, Sven Barth via fpc-pascal said:
> > not exist, if FreePascal had gone another direction like PL/1, for
> > example,
> > where the string concatenation operator is ||
> > (and DB2, and - probably - other SQL dialects).
> >
>
> FPC inherited the +-operator for concatenation from the base language:
> Pascal. So there simply was no other route to take (not that anyone would
> have thought to take a different route).

Well, + is also addition in pascal. It depends on what is considered more
important, the fact that array of integer has a base type that suppers
adding, or the simularity with strings(that are also dynamic array like)
that uses it for concat.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Feature announcement: Dynamic array extensions

Mark Morgan Lloyd-5
In reply to this post by denisgolovan
On 03/06/18 16:30, denisgolovan wrote:
> Having worked with APL for almost 6 years and continuing with Q/KDB, I would say APL syntax is much more readable than Perl 6.

Horses for courses. I think the impressive thing about APL is that the
necessary operations were worked out (and used for Blackboard
demonstrations) /before/ it was converted into a computer language, and
by and large weren't added to. However the functional nature of the
language was vastly overrused, and students who thought they were being
smart would on occasion find themselves with mainframe runtimes of
/months/ because they'd created an enormous array instead of using a
simple control structure.

Nod here to Vector Pascal as well. And as a bit of history, the first
computer implementation of Iverson's notation was done at Stanford under
the watchful eye of Niklaus Wirth.

> Though being able to modify language like that is really impressive.

I agree, which is the main reason I posted the link.

--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Feature announcement: Dynamic array extensions

denisgolovan


> Horses for courses. I think the impressive thing about APL is that the
> necessary operations were worked out (and used for Blackboard
> demonstrations) /before/ it was converted into a computer language, and
> by and large weren't added to. However the functional nature of the
> language was vastly overrused, and students who thought they were being
> smart would on occasion find themselves with mainframe runtimes of
> /months/ because they'd created an enormous array instead of using a
> simple control structure.

Yes, sure.
But things slowly change in compiler/interpreter world.
Now we have loop/streams fusion techniques and other devectorization tricks, so I think smart APL interpreter would be of real help here.
 
> Nod here to Vector Pascal as well. And as a bit of history, the first
> computer implementation of Iverson's notation was done at Stanford under
> the watchful eye of Niklaus Wirth.

Hm, I didn't know that. Thanks for sharing.
Looks like computer scientist community was really small those days.

Vector Pascal is much closer to our current discussion though :)

--
Regards,
Denis Golovan
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Feature announcement: Dynamic array extensions

Bernd Oppolzer
In reply to this post by Free Pascal - General mailing list

Am 03.06.2018 um 18:12 schrieb Sven Barth via fpc-pascal:
Bernd Oppolzer <[hidden email]> schrieb am So., 3. Juni 2018, 11:56:



Am 02.06.2018 um 15:14 schrieb Sven Barth via fpc-pascal:
Mark Morgan Lloyd <[hidden email]> schrieb am Sa., 2. Juni 2018, 10:53:
However as Dennis points out + is also essential for vector operations.
Perhaps either leaving it to the programmer to define what's needed
would be the best approach, or alternatively splitting dynamic arrays
into mathematical vectors and non-mathematical collections. Or relaxing
the requirement that only predefined operators can be redefined, so that
something like _ could be used for concatenation.

That needlessly complicates the parser as the compiler still needs to know them and they also need to be part of its operator precedence rules. Don't complicate the language for nothing! And in the end operator overloads are one of the best examples for syntactic sugar as you can easily achieve the same result with functions and methods. 

Regards, 
Sven 


This is somehow off topic of course,
but IMO it is strange to use + for string concatenation;
I always have bad feelings about this. This whole thread would
not exist, if FreePascal had gone another direction like PL/1, for example,
where the string concatenation operator is ||
(and DB2, and - probably - other SQL dialects).

FPC inherited the +-operator for concatenation from the base language: Pascal. So there simply was no other route to take (not that anyone would have thought to take a different route). 


Where does this + for string concat come from?

Ask Wirth, he is the one who invented Pascal... 


AFAIK, the Pascal Standard (ISO) does not tell anything about strings
and concatenation. Wirth did not include varying length strings into
the language in the 197x years. Every Pascal compiler which contains
strings and concatenation does this by extending the Pascal Standard
and can choose its own way to do it.

IBMs Pascal/VS uses || for string concatenation (inspired by PL/1, probably),
and that's what I implemented in the New Stanford Pascal compiler, too.

The + for concatenation must be an invention of Turbo Pascal or UCSD Pascal
or something like that.

Regards, 
Sven 



_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Feature announcement: Dynamic array extensions

Nitorami
>> It would be reasonable to assume that the predefined + might be
>> substantially more efficient than a programmer-defined one could be.

>Yes, that's one of the reasons I vote for keeping the new feature
>and allow to overload the operator.

I don't think that argument holds water. Concatenation of dynamic structures
is a slow function in the first place. I am not a core developer, but I
consider that regardless how you do it, it will require a call to setlength,
which in turn will call the operating system to allocate memory for the now
larger structure. That takes much more time than a few assembly instructions
for a normal pascal procedure call.



--
Sent from: http://free-pascal-general.1045716.n5.nabble.com/
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Feature announcement: Dynamic array extensions

Mark Morgan Lloyd-5
On 04/06/18 12:00, Nitorami wrote:
>>> It would be reasonable to assume that the predefined + might be>> substantially more efficient than a programmer-defined one could be.
>> Yes, that's one of the reasons I vote for keeping the new feature>and allow to overload the operator.
> I don't think that argument holds water. Concatenation of dynamic structuresis a slow function in the first place. I am not a core developer, but Iconsider that regardless how you do it, it will require a call to setlength,which in turn will call the operating system to allocate memory for the nowlarger structure. That takes much more time than a few assembly instructionsfor a normal pascal procedure call.

The obvious workaround is to preallocate a dynamic array to a nominal
size and then to trim it using SetLength(0), I'd assumed that it would
retain ownership of the memory but I've just checked.

What actually happens is that the memory is released back to the heap
(but *not* to the OS, at least on Linux), with the result that
concatenating elements will introduce a substantial hit particularly if
space for a new element allocated from the heap isn't contiguous.

--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Feature announcement: Dynamic array extensions

Martok
> What actually happens is that the memory is released back to the heap
> (but *not* to the OS, at least on Linux), with the result that
> concatenating elements will introduce a substantial hit particularly if
> space for a new element allocated from the heap isn't contiguous.
Writing a preallocating wrapper *where needed* for heavily grown arrays is
fairly simple.

The thing is: it's rarely worth it ;-)
I recently did that for a wavefront mesh loader that appends vertices one-by-one
to an array, and for a scene with some 400k vertices, the difference was just
some milliseconds out of several seconds overall.
Turns out the allocator usually finds a spot where the array doesn't need to
actually be copied around for a while, and the pure bookkeeping of realloc is
very cheap.

--
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
12345