Proper preprocessor?

classic Classic list List threaded Threaded
77 messages Options
1234
Reply | Threaded
Open this post in threaded view
|

Re: Proper preprocessor?

Martin Schreiber-2
On Wednesday 20 June 2018 16:06:13 Michael Van Canneyt wrote:
>
> Please stop calling it 'dogma'.
>
> As with all features, it is a trade-off between the burden this places on
> the compiler (and the people maintaining it) and the expected gain.
>
And the damage it causes on readability of code. Every new language feature
will be used, every new language feature forces all programmers which have to
read code from others to learn the new features. This is especially important
for languages which are established in open source world.
Macros are the worst code obfuscating feature ever.

Martin
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Proper preprocessor?

Michael Van Canneyt


On Wed, 20 Jun 2018, Martin Schreiber wrote:

> On Wednesday 20 June 2018 16:06:13 Michael Van Canneyt wrote:
>>
>> Please stop calling it 'dogma'.
>>
>> As with all features, it is a trade-off between the burden this places on
>> the compiler (and the people maintaining it) and the expected gain.
>>
> And the damage it causes on readability of code. Every new language feature
> will be used, every new language feature forces all programmers which have to
> read code from others to learn the new features. This is especially important
> for languages which are established in open source world.
> Macros are the worst code obfuscating feature ever.

Exactly...

Michael.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Proper preprocessor?

Marc Santhoff-2
In reply to this post by Michael Van Canneyt
On Wed, 2018-06-20 at 15:09 +0200, Michael Van Canneyt wrote:

>
> On Wed, 20 Jun 2018, Mark Morgan Lloyd wrote:
>
> > The other alternative would be break the compiler in such a way that it
> > was usable from a standard makefile, but since there isn't separate
> > compilation of definition and implementation parts this would probably
> > impact on type safety. I believe that this too has been debated in the
> > past, and has attracted even less enthusiasm than a hook for an extrnal
> > preprocessor preprocessor.
>
> Nothing stops people from preprocessing their code if they need really
> advanced preprocessing: The toolchain can handle it already.
>
> But there is no need to integrate it in the compiler and thus needlessly
> complicating it even more. The consequences of such a step are far-reaching.
>
> And till now, no-one has presented the really pressing use cases that would
> warrant such a step.

The only use case for me would be macros having more than one parameter,
needed when translating C-code.

But I speak up for another reason:

Long ago, at the time of fpc 1.9.x or 2.0.x I did some digging in compiler
source code, the lexer and parser part. IIRC there were some hooks for calling
a proprocessor in the code at that time. If they are still there, wouldn't it
be a solution for both sides?

The compiler programmers only have to activate (or complete) a way to call a
preprocessor, solving the problem of mangled error messages.

The preprocessor user could implement whatever is needed on his or her side?

My 2 cent,
Marc

--
Marc Santhoff <[hidden email]>
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Proper preprocessor?

Ryan Joseph
In reply to this post by Michael Van Canneyt


> On Jun 20, 2018, at 9:06 PM, Michael Van Canneyt <[hidden email]> wrote:
>
> So, it really is not dogma, but a simple weighing of pros and cons.

Thanks for your input guys. If it’s going to mess up PPUs and break things then that I’m satisfied that’s a good enough reason not to include it.

My impression was it was trivial extension to the current macros system but the compiler team was adverse to the idea because it creates “bad code” and all the other (very reasonable) reasons to not use macros in code (I read some old threads to this effect). It’s hard to understand why the current macros can’t be extended so I suspect this issue will just keep coming up over and over again.

Regards,
        Ryan Joseph

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Proper preprocessor?

Ryan Joseph
In reply to this post by Martin Schreiber-2


> On Jun 20, 2018, at 9:21 PM, Martin Schreiber <[hidden email]> wrote:
>
> Macros are the worst code obfuscating feature ever.

Ironically everyone agrees but back to my original point that’s just dogma (sorry I said it!). If I was doing C I wouldn’t refuse to use the macros to solve simple problems because “there the worst code obfuscating feature ever”.

At the end of the day I just want to be happy and productive programming. If a macro helps me do that then so be it. Life’s too short.

Regards,
        Ryan Joseph

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Proper preprocessor?

Ryan Joseph
In reply to this post by Marc Santhoff-2


> On Jun 20, 2018, at 9:26 PM, Marc Santhoff <[hidden email]> wrote:
>
> Long ago, at the time of fpc 1.9.x or 2.0.x I did some digging in compiler
> source code, the lexer and parser part. IIRC there were some hooks for calling
> a proprocessor in the code at that time. If they are still there, wouldn't it
> be a solution for both sides?

I hope that’s not true. :) Marco says otherwise but it’s hard to imagine why the current $define:= couldn’t take an argument. I’m sorry to say for the compiler team this issue is just going to come up over and over because programmers are going to be confused as to why the current macros don’t take parameters.

Regards,
        Ryan Joseph

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Proper preprocessor?

Martin Schreiber-2
In reply to this post by Ryan Joseph
On Wednesday 20 June 2018 16:38:10 Ryan Joseph wrote:

> > On Jun 20, 2018, at 9:21 PM, Martin Schreiber <[hidden email]> wrote:
> >
> > Macros are the worst code obfuscating feature ever.
>
> Ironically everyone agrees but back to my original point that’s just dogma
> (sorry I said it!). If I was doing C I wouldn’t refuse to use the macros to
> solve simple problems because “there the worst code obfuscating feature
> ever”.
>
> At the end of the day I just want to be happy and productive programming.
> If a macro helps me do that then so be it. Life’s too short.
>
I daily read code from many people. Beleave me, “there the worst code
obfuscating feature ever” is one of the most proven in use dogma ever. ;-)
That you will not misuse it does not mean nobody will misuse it. The C-example
which has been provided looks like a misuse for me. Macros replace text in
code with something other which can be anything. Isn't this obfuscation by
definition?

Martin
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Proper preprocessor?

Michael Van Canneyt
In reply to this post by Ryan Joseph


On Wed, 20 Jun 2018, Ryan Joseph wrote:

>
>
>> On Jun 20, 2018, at 9:26 PM, Marc Santhoff <[hidden email]> wrote:
>>
>> Long ago, at the time of fpc 1.9.x or 2.0.x I did some digging in compiler
>> source code, the lexer and parser part. IIRC there were some hooks for calling
>> a proprocessor in the code at that time. If they are still there, wouldn't it
>> be a solution for both sides?
>
> I hope that’s not true.  :) Marco says otherwise but it’s hard to imagine
> why the current $define:= couldn’t take an argument.
Because it is a simple textual token replacement at present.
Supporting arguments would mean parsing the macro, parsing whatever comes after
it, matching the arguments etc. A whole added layer of complexity.

> I’m sorry to say for
> the compiler team this issue is just going to come up over and over
> because programmers are going to be confused as to why the current macros
> don’t take parameters.

To put things in perspective: in 25 years it came up only a handful of times.
So I'm not inclined to perceive it as a pressing problem...

Michael.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Proper preprocessor?

Ryan Joseph
In reply to this post by Mark Morgan Lloyd-5


> On Jun 20, 2018, at 9:16 PM, Mark Morgan Lloyd <[hidden email]> wrote:
>
>> How can you integrate a preprocessor without misaligning error messages and debugging information?
>
> I forget the detail but some language implementations have pragmata which tell subsequent processing steps that at this point the source should be considered to have come from such-and-such a line number in such-and-such a file.

I think you’re right but that’s internal to the compiler and not really something we can replicate ourselves. In fact that’s what the $include directive does right? Again correct me if I’m wrong but I think $define:= does the same thing otherwise column numbers (at least) in error messages would be misaligned.

If macros took parameters it’s the same as before it just includes some extra text. I keep saying it but it’s really difficult to  understand why this is compiler breaking stuff.

{$define foo(xx) := DoFoo(xx)}

Regards,
        Ryan Joseph

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Proper preprocessor?

Ryan Joseph
In reply to this post by Martin Schreiber-2


> On Jun 20, 2018, at 9:54 PM, Martin Schreiber <[hidden email]> wrote:
>
> Isn't this obfuscation by
> definition?

Indeed it is. I really do agree 100% but strangely enough it doesn’t matter. :) Keep in mind often we’re writing code that only ourselves will ever see and we don’t need any justification to do something non-standard or otherwise very stupid.

Regards,
        Ryan Joseph

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Proper preprocessor?

Ryan Joseph
In reply to this post by Michael Van Canneyt


> On Jun 20, 2018, at 10:02 PM, Michael Van Canneyt <[hidden email]> wrote:
>
> Because it is a simple textual token replacement at present. Supporting arguments would mean parsing the macro, parsing whatever comes after it, matching the arguments etc. A whole added layer of complexity.

It’s like a simplified regular expression. Capture anything inside () on the left side and replace occurrences on the right side.

{$define hello(me) := writeln(‘hello me')}

hello(world) becomes writeln(‘hello world’)

Regards,
        Ryan Joseph

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Proper preprocessor?

Marc Santhoff-2
In reply to this post by Ryan Joseph
On Wed, 2018-06-20 at 21:34 +0700, Ryan Joseph wrote:

> My impression was it was trivial extension to the current macros system but
> the compiler team was adverse to the idea because it creates “bad code” and
> all the other (very reasonable) reasons to not use macros in code (I read
> some old threads to this effect). It’s hard to understand why the current
> macros can’t be extended so I suspect this issue will just keep coming up
> over and over again.

If it's that important to you, do it yourself. The compiler is open source,
pure, readable pascal.

The spots where resolving single parameter macros is done are pretty easy to
find. Parsing the macro text and replacement will be the hardest part, as
Michael wrote. A bit of housekeeping for parameter-type lists, etc...

--
Marc Santhoff <[hidden email]>
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Proper preprocessor?

Ryan Joseph


> On Jun 20, 2018, at 10:20 PM, Marc Santhoff <[hidden email]> wrote:
>
> The spots where resolving single parameter macros is done are pretty easy to
> find. Parsing the macro text and replacement will be the hardest part, as
> Michael wrote. A bit of housekeeping for parameter-type lists, etc...

Easy? I’ve wanted to contribute to the compiler for years but it’s so daunting finding anything I give up. I’m curious how the parser works so please show me where the parser does the $defines. Maybe I can figure it out this time.

Regards,
        Ryan Joseph

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Proper preprocessor?

Marc Santhoff-2
On Wed, 2018-06-20 at 22:45 +0700, Ryan Joseph wrote:

> > On Jun 20, 2018, at 10:20 PM, Marc Santhoff <[hidden email]> wrote:
> >
> > The spots where resolving single parameter macros is done are pretty easy
> > to
> > find. Parsing the macro text and replacement will be the hardest part, as
> > Michael wrote. A bit of housekeeping for parameter-type lists, etc...
>
> Easy? I’ve wanted to contribute to the compiler for years but it’s so
> daunting finding anything I give up. I’m curious how the parser works so
> please show me where the parser does the $defines. Maybe I can figure it out
> this time.

When I looked around it was in

scanner.pas
symsym.pas

Just grep for "macro".

If there is more or I'm wrong hopefully one of the "compiler guys" will help
out here, please. ;)

--
Marc Santhoff <[hidden email]>
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Proper preprocessor?

Mark Morgan Lloyd-5
In reply to this post by Marc Santhoff-2
On 20/06/18 14:45, Marc Santhoff wrote:
> But I speak up for another reason:
> Long ago, at the time of fpc 1.9.x or 2.0.x I did some digging in compilersource code, the lexer and parser part. IIRC there were some hooks for callinga proprocessor in the code at that time. If they are still there, wouldn't itbe a solution for both sides?
> The compiler programmers only have to activate (or complete) a way to call apreprocessor, solving the problem of mangled error messages.
> The preprocessor user could implement whatever is needed on his or her side?
> My 2 cent,Marc

Is that what the PREPROCWRITE define is for?

My 1½d :-)

--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Proper preprocessor?

Mark Morgan Lloyd-5
In reply to this post by Marc Santhoff-2
On 20/06/18 16:00, Marc Santhoff wrote:
> On Wed, 2018-06-20 at 22:45 +0700, Ryan Joseph wrote:> > On Jun 20, 2018, at 10:20 PM, Marc Santhoff <[hidden email]> wrote:> > > > The spots where resolving single parameter macros is done are pretty easy> > to> > find. Parsing the macro text and replacement will be the hardest part, as> > Michael wrote. A bit of housekeeping for parameter-type lists, etc...> > Easy? I’ve wanted to contribute to the compiler for years but it’s so> daunting finding anything I give up. I’m curious how the parser works so> please show me where the parser does the $defines. Maybe I can figure it out> this time.
> When I looked around it was in
> scanner.passymsym.pas
> Just grep for "macro".
> If there is more or I'm wrong hopefully one of the "compiler guys" will helpout here, please. ;)

Word of caution: IIRC macros only work properly in comments delimited by
braces {}

--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Proper preprocessor?

Free Pascal - General mailing list
In reply to this post by Mark Morgan Lloyd-5
Mark Morgan Lloyd <[hidden email]> schrieb am Mi., 20. Juni 2018, 18:38:
On 20/06/18 14:45, Marc Santhoff wrote:
> But I speak up for another reason:
> Long ago, at the time of fpc 1.9.x or 2.0.x I did some digging in compilersource code, the lexer and parser part. IIRC there were some hooks for callinga proprocessor in the code at that time. If they are still there, wouldn't itbe a solution for both sides?
> The compiler programmers only have to activate (or complete) a way to call apreprocessor, solving the problem of mangled error messages.
> The preprocessor user could implement whatever is needed on his or her side?
> My 2 cent,Marc

Is that what the PREPROCWRITE define is for?

That is merely for the ability to output a preprocessed source file after all the compiler directives and macros have been handled. AFAIK it hasn't worked for a long time... 

Regards, 
Sven 

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Proper preprocessor?

Free Pascal - General mailing list
In reply to this post by Ryan Joseph
Ryan Joseph <[hidden email]> schrieb am Mi., 20. Juni 2018, 17:41:


> On Jun 20, 2018, at 10:02 PM, Michael Van Canneyt <[hidden email]> wrote:
>
> Because it is a simple textual token replacement at present. Supporting arguments would mean parsing the macro, parsing whatever comes after it, matching the arguments etc. A whole added layer of complexity.

It’s like a simplified regular expression. Capture anything inside () on the left side and replace occurrences on the right side.

{$define hello(me) := writeln(‘hello me')}

hello(world) becomes writeln(‘hello world’)

In Pascal you'd use inline functions for this one though of course with strong typing and not some identifier that both a human and the IDE will have to guess what it is. 

Regards, 
Sven 

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Proper preprocessor?

Marco van de Voort
In reply to this post by Ryan Joseph
In our previous episode, Ryan Joseph said:
> > On Jun 20, 2018, at 10:02 PM, Michael Van Canneyt <[hidden email]> wrote:
> >
> > Because it is a simple textual token replacement at present. Supporting arguments would mean parsing the macro, parsing whatever comes after it, matching the arguments etc. A whole added layer of complexity.
>
> It?s like a simplified regular expression. Capture anything inside () on the left side and replace occurrences on the right side.
>
> {$define hello(me) := writeln(?hello me')}
>
> hello(world) becomes writeln(?hello world?)

That would be C incompatible, which I thought was the main reason to have
it?  It would also replace me in identifiers (like 'varwithme'), which C
afaik wouldn't without ##


_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Proper preprocessor?

Mark Morgan Lloyd-5
On 20/06/18 17:30, Marco van de Voort wrote:

> That would be C incompatible, which I thought was the main reason to haveit?

I don't believe Ryan said that (and I certainly didn't). It's the
functionality that counts, not slavish adherence to any particular syntax.

--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
1234