Better usage of "with"

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

Better usage of "with"

Ryan Joseph
I was just watching a video from Jonathan Blow (the developer of a really popular game named Braid) on a new language he’s working on to get around all the cruft and slowness in c++ for making games. One the problems he identifies is poor memory layout using object oriented inheritance and arrays of structs.

His proposal was to make it easier to build linked lists in structs that can access tightly packed arrays of continuous memory for fields that are iterated often and susceptible to cache misses. The solution he came up with is really brilliant in my opinion and even though he used the keyword “using” it’s basically like “with” in Pascal.

I never use “with” normally but this is an extremely interesting idea and I wonder if it would be compatible with how “with” works in FPC now. Is any of this functionality possible using “with"? Curious what anyone thinks about this, providing my explanation makes any sense.

type
        TEntity = record
                position: TVec2;
        end;
        TEntityPtr = ^TEntity;

type
        TDoor = record
                with entity: TEntityPtr; // import entity namespace for entire TDoor scope
                state: boolean;
        end;

var
        door: TDoor;
begin
        door.entity := GetAvailableEntity; // pop an entity from storage
        door.position := V2(1, 1); // entity is “with” so we get access to its members (door.entity^.position)
       
// works with function parameters also.
// this is almost like a class helper or at very least mimics “self” in methods.

procedure OpenDoor(with var door: TDoor);
begin
        state := true; // with imports door namespace into entire function scope
end;


Regards,
        Ryan Joseph

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Better usage of "with"

Marco van de Voort
In our previous episode, Ryan Joseph said:
> // works with function parameters also.
> // this is almost like a class helper or at very least mimics ?self? in methods.
>
> procedure OpenDoor(with var door: TDoor);
> begin
> state := true; // with imports door namespace into entire function scope
> end;

you set a runtime variable to add a compiletime scope ? I don't understand.

Is your reference language an interpreter?
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Better usage of "with"

Ryan Joseph


> On Jun 14, 2018, at 3:23 PM, Marco van de Voort <[hidden email]> wrote:
>
>> // works with function parameters also.
>> // this is almost like a class helper or at very least mimics ?self? in methods.
>>
>> procedure OpenDoor(with var door: TDoor);
>> begin
>> state := true; // with imports door namespace into entire function scope
>> end;
>
> you set a runtime variable to add a compiletime scope ? I don't understand.
>
> Is your reference language an interpreter?


the “with” in the parameter is like a “with door do” block inside the entire function scope. I really liked this idea of his because it’s basically something Pascal already does but it’s automatic with this syntax.

procedure OpenDoor(var door: TDoor);
begin
  with door do
   begin
     state := true; // with imports door namespace into entire function scope
   end;
end;


Regards,
        Ryan Joseph

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Better usage of "with"

Free Pascal - General mailing list
In reply to this post by Ryan Joseph
Ryan Joseph <[hidden email]> schrieb am Do., 14. Juni 2018, 10:03:
type
        TEntity = record
                position: TVec2;
        end;
        TEntityPtr = ^TEntity;

type
        TDoor = record
                with entity: TEntityPtr; // import entity namespace for entire TDoor scope
                state: boolean;
        end;

var
        door: TDoor;
begin
        door.entity := GetAvailableEntity;      // pop an entity from storage
        door.position := V2(1, 1);              // entity is “with” so we get access to its members (door.entity^.position)

Something like that only leads to confusion. Pascal is a rather explicit language so manually declaring a Position property inside TDoor would be more in the spirit of the language. 

A possible alternative would be generic type helpers, like this (just an example, not working code):

=== code beging ===

type
  generic TEntityUserHelper<T> = type helper for T
  private
    function GetPosition: TVec2D; inline;
    procedure SetPosition(constref aValue: TVec2D); inline;
  public
    property Position: TVec2D read GetPosition write SetPosition;
  end;

function TEntityUserHelper.GetPosition: TVec2D;
begin
  Result := Self.Entity^.Position;
end;

procedure TEntityUserHelper.SetPosition(constref aValue: TVec2D);
begin
  Self.Entity^.Position := aValue;
end;

type
  TDoor = record
    Entity: PEntity;
    State: Boolean;
  end;
  TDoorHelper = specialize TEntityUserHelper<TDoor>;

var
  door: TDoor;
begin
  // init door
  door.Position := Vec2D(21, 42);
end;

=== code end ===



// works with function parameters also.
// this is almost like a class helper or at very least mimics “self” in methods.

procedure OpenDoor(with var door: TDoor);
begin
        state := true; // with imports door namespace into entire function scope
end;

The declaration of the function (in the interface section) would need to contain the "with" as all parameters have to match (and its only that parts are removed from the definition (default parameters, modifiers), but not the other way around) and thus the declaration would "spill" information about the function while not needing to. 
And again I think that as an expressive language the explicit usage of a "with" block is better than something like this. Please also not that there are people out there that advocate *against* the usage of "with" at all as it is likely to introduce bugs with it. 

Regards, 
Sven 

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Better usage of "with"

Ryan Joseph


> On Jun 14, 2018, at 4:25 PM, Sven Barth via fpc-pascal <[hidden email]> wrote:
>
> omething like that only leads to confusion. Pascal is a rather explicit language so manually declaring a Position property inside TDoor would be more in the spirit of the language.
>
> A possible alternative would be generic type helpers, like this (just an example, not working code):

That’s probably the best available solution but more verbose than “with” in records. JB also hinted at how templates could be used to hack up something and I think this is what had in mind. :) The idea of “with” inside records is so interesting because it basically mimics inheritance without going full class. Now that I’ve seen it it feels like a natural extension of how data should be able to be laid out.

Something I tried was using properties like this but it didn’t work. Is that something properties could do? It feels like they should be able to reach into records fields that since everything is known to the property as time of declaration.

property Position: TVec2D read entity^.position;

>
> === code beging ===
>
> type
>   generic TEntityUserHelper<T> = type helper for T
>   private
>     function GetPosition: TVec2D; inline;
>     procedure SetPosition(constref aValue: TVec2D); inline;
>   public
>     property Position: TVec2D read GetPosition write SetPosition;
>   end;
>
> function TEntityUserHelper.GetPosition: TVec2D;
> begin
>   Result := Self.Entity^.Position;
> end;
>
> procedure TEntityUserHelper.SetPosition(constref aValue: TVec2D);
> begin
>   Self.Entity^.Position := aValue;
> end;
>
> type
>   TDoor = record
>     Entity: PEntity;
>     State: Boolean;
>   end;
>   TDoorHelper = specialize TEntityUserHelper<TDoor>;
>
> var
>   door: TDoor;
> begin
>   // init door
>   door.Position := Vec2D(21, 42);
> end;
>
> === code end ===
>
>
>
> // works with function parameters also.
> // this is almost like a class helper or at very least mimics “self” in methods.
>
> procedure OpenDoor(with var door: TDoor);
> begin
>         state := true; // with imports door namespace into entire function scope
> end;
>
> The declaration of the function (in the interface section) would need to contain the "with" as all parameters have to match (and its only that parts are removed from the definition (default parameters, modifiers), but not the other way around) and thus the declaration would "spill" information about the function while not needing to.
> And again I think that as an expressive language the explicit usage of a "with" block is better than something like this. Please also not that there are people out there that advocate *against* the usage of "with" at all as it is likely to introduce bugs with it.

I don’t follow the syntax issue in the first paragraph or understand why it’s a problem to have with in both the interface and implementation (var is that way after all).

A with block does the same thing but putting in the parameters basically mimics what classes do with methods, i.e. a hidden “self” which is the first parameter and hidden in the scope. It’s such a common pattern for procedural API’s where the first param is a pointer to the member data that it makes sense to be part of the language imo.

The reason Jonathan Blow suggested this is to get out of OOP designs when really all you want is the syntactic sugar of not typing door^.xxx all the time, which is reason enough to make a class in some people minds. He’s using c++ though so he has no fallback like “with" in Pascal and even so wrapping entire functions in with .. do is not as nice as his idea.

Regards,
        Ryan Joseph

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Better usage of "with"

Ryan Joseph


> On Jun 14, 2018, at 4:44 PM, Ryan Joseph <[hidden email]> wrote:
>
> Something I tried was using properties like this but it didn’t work. Is that something properties could do? It feels like they should be able to reach into records fields that since everything is known to the property as time of declaration.
>
> property Position: TVec2D read entity^.position;

I just did a test and found out property does indeed access record fields (that’s great to know) but when I made it a pointer that’s when it broke. Why not make properties do this?

type
        TEntity = record
                position: TVec2;
        end;
        TEntityPtr = ^TEntity;

type
        TDoor = record
                entity: TEntityPtr;
                state: boolean;
                property position: TVec2 read entity^.position;
        end;


Regards,
        Ryan Joseph

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Better usage of "with"

Mattias Gaertner
In reply to this post by Ryan Joseph
On Thu, 14 Jun 2018 16:44:40 +0700
Ryan Joseph <[hidden email]> wrote:

>[...]
> The idea of “with” inside records is so interesting because it basically mimics inheritance without going full class.

If you only want that, why not use objects?

Mattias
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Better usage of "with"

Ryan Joseph


> On Jun 14, 2018, at 5:18 PM, Mattias Gaertner <[hidden email]> wrote:
>
> If you only want that, why not use objects?

The real reason Jon Blow proposed this syntax is because it allows you keep continuous arrays of memory which you can index into using struts. The fact it mimics multiple inheritance (something Pascal doesn’t have anyways) is just a bonus.

His premise for the proposal is that programming languages should help the programmer to make code which is better read by the CPU instead of high-level constructs for abstract concepts. It’s very specific to a particular task but very important for low-languages like c++ and Pascal which are close to the CPU.

Regards,
        Ryan Joseph

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Better usage of "with"

Martin Friebe
On 14/06/2018 13:43, Ryan Joseph wrote:
>
>> On Jun 14, 2018, at 5:18 PM, Mattias Gaertner <[hidden email]> wrote:
>>
>> If you only want that, why not use objects?
> The real reason Jon Blow proposed this syntax is because it allows you keep continuous arrays of memory which you can index into using struts.
> The fact it mimics multiple inheritance (something Pascal doesn’t have anyways) is just a bonus.


You can do that without the "with"
record TFoo
   Bar: TOtherRecord; // has a field abc
end;

The only thing is that you need to access the field by its fully
qualified name Foo.Bar.abc, instead of Foo.abc.
Which is good, because you can have

record TFoo
   Bar: TOtherRecord; // has a field abc
   Other: TOtherRecord; // has a field abc
end;

and as with multiple inheritance "Foo.abc" would be ambiguous.

And as already was mentioned you can have properties to shortcut this.
But then the question came up why not:

record TFoo
   Bar: POtherRecord; // pointer
   property abc: xxx read Bar^.abc
end;

Well because abc may be nil. But then it is no longer "continuous memory"
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Better usage of "with"

Free Pascal - General mailing list
In reply to this post by Ryan Joseph
Ryan Joseph <[hidden email]> schrieb am Do., 14. Juni 2018, 11:59:


> On Jun 14, 2018, at 4:44 PM, Ryan Joseph <[hidden email]> wrote:
>
> Something I tried was using properties like this but it didn’t work. Is that something properties could do? It feels like they should be able to reach into records fields that since everything is known to the property as time of declaration.
>
> property Position: TVec2D read entity^.position;

I just did a test and found out property does indeed access record fields (that’s great to know) but when I made it a pointer that’s when it broke. Why not make properties do this?

type
        TEntity = record
                position: TVec2;
        end;
        TEntityPtr = ^TEntity;

type
        TDoor = record
                entity: TEntityPtr;
                state: boolean;
                property position: TVec2 read entity^.position;
        end;

Because one needs to be able to represent the properties in the RTTI. Without pointers all that is needed to access the property from code is an offset. With pointers you'd need to add where dereferences are required especially as the "path" to the final field could be more complex (e.g. Field1^.Field3.Field1.Field5^.FinalField).
For this the solution is a getter function that you can also declare as "inline". 

Regards, 
Sven 

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Better usage of "with"

Marcos Douglas B. Santos
In reply to this post by Ryan Joseph
On Thu, Jun 14, 2018 at 8:43 AM, Ryan Joseph <[hidden email]> wrote:
>
[...]
> His premise for the proposal is that programming languages
> should help the programmer to make code which is better read
> by the CPU instead of high-level constructs for abstract concepts.
> It’s very specific to a particular task but very important for
> low-languages like c++ and Pascal which are close to the CPU.

It is the opposite: programming languages should help programmers
to make code which is better read by human being, not CPU.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Better usage of "with"

Ryan Joseph
In reply to this post by Martin Friebe


> On Jun 14, 2018, at 7:19 PM, Martin <[hidden email]> wrote:
>
> You can do that without the "with"
> record TFoo
>   Bar: TOtherRecord; // has a field abc
> end;
>
> The only thing is that you need to access the field by its fully qualified name Foo.Bar.abc, instead of Foo.abc.
> Which is good, because you can have
>
> record TFoo
>   Bar: TOtherRecord; // has a field abc
>   Other: TOtherRecord; // has a field abc
> end;
>
> and as with multiple inheritance "Foo.abc" would be ambiguous.

yeah so you wouldn’t design your data like that. :) There’s always a potential for naming conflicts but some how we manage them.

>
> And as already was mentioned you can have properties to shortcut this.
> But then the question came up why not:
>
> record TFoo
>   Bar: POtherRecord; // pointer
>   property abc: xxx read Bar^.abc
> end;
>
> Well because abc may be nil. But then it is no longer "continuous memory"

That’s not what you’d do though. Like my response above this assumes you actually know what you’re doing and you aren’t going to trash memory.


Regards,
        Ryan Joseph

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Better usage of "with"

Ryan Joseph
In reply to this post by Free Pascal - General mailing list


> On Jun 14, 2018, at 7:26 PM, Sven Barth via fpc-pascal <[hidden email]> wrote:
>
> Because one needs to be able to represent the properties in the RTTI. Without pointers all that is needed to access the property from code is an offset. With pointers you'd need to add where dereferences are required especially as the "path" to the final field could be more complex (e.g. Field1^.Field3.Field1.Field5^.FinalField).

I guess I don’t understand because I thought an offset to a pointer you dereference is as good as an offset to anything else. The property contains the ^ symbol so it should know to dereference the pointer at the offset right?

It would be so much nicer to do this with properties then making tons of boiler plate code that fills up the implementation and requires maintaining if you change names etc… If you could somehow do it inside the record (like c++) then it would be easier to maintain but in Pascal and separate implementation section it becomes a bit of a chore.

Regards,
        Ryan Joseph

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Better usage of "with"

Ryan Joseph
In reply to this post by Marcos Douglas B. Santos


> On Jun 14, 2018, at 7:42 PM, Marcos Douglas B. Santos <[hidden email]> wrote:
>
> It is the opposite: programming languages should help programmers
> to make code which is better read by human being, not CPU.

Jonathan Blow disagrees with you then. :) None of this matter for typical desktop applications. This is strictly for real time graphics, games etc… where getting tons of cache misses on every frame is an actual problem.

That aside I think the language can keep the code in basically the same state but let us rearrange thins in a more efficient manner at the same time.

Regards,
        Ryan Joseph

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Better usage of "with"

Ryan Joseph


> On Jun 14, 2018, at 7:51 PM, Ryan Joseph <[hidden email]> wrote:
>
> Jonathan Blow disagrees with you then. :) None of this matter for typical desktop applications. This is strictly for real time graphics, games etc… where getting tons of cache misses on every frame is an actual problem.

Btw here’s the link to his language test. The first 10 minutes he goes well into the problem and he plans to solve it. I would think Pascal programmers would find this extremely interesting. I did at least. :)

https://www.youtube.com/watch?v=ZHqFrNyLlpA

Regards,
        Ryan Joseph

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Better usage of "with"

Free Pascal - General mailing list
In reply to this post by Ryan Joseph
Ryan Joseph <[hidden email]> schrieb am Do., 14. Juni 2018, 14:48:


> On Jun 14, 2018, at 7:26 PM, Sven Barth via fpc-pascal <[hidden email]> wrote:
>
> Because one needs to be able to represent the properties in the RTTI. Without pointers all that is needed to access the property from code is an offset. With pointers you'd need to add where dereferences are required especially as the "path" to the final field could be more complex (e.g. Field1^.Field3.Field1.Field5^.FinalField).

I guess I don’t understand because I thought an offset to a pointer you dereference is as good as an offset to anything else. The property contains the ^ symbol so it should know to dereference the pointer at the offset right?

The RTTI for a property contains *only* the offset to the final field inside the record, object or class instance. With a dereference in between this does not work anymore as the field could now be anywhere.

Regards, 
Sven 

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal