TProcess vs RunProcess()

classic Classic list List threaded Threaded
24 messages Options
12
Reply | Threaded
Open this post in threaded view
|

TProcess vs RunProcess()

Graeme Geldenhuys-6
Hi,

Whats the differences or pros and cons between using TProcess vs
RunProcess().


Regards,
  Graeme

--
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://fpgui.sourceforge.net/

My public PGP key:  http://tinyurl.com/graeme-pgp
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: TProcess vs RunProcess()

Michael Van Canneyt


On Thu, 26 Jan 2017, Graeme Geldenhuys wrote:

> Hi,
>
> Whats the differences or pros and cons between using TProcess vs
> RunProcess().

TProcess gives you full access to the process. You can kill it,
pause it, write to stdin, read from stdout.

These things cannot be done with RunProcess.
That's one-shot and wait till it exits.

Michael.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: TProcess vs RunProcess()

Graeme Geldenhuys-6
On 2017-01-26 14:28, Michael Van Canneyt wrote:
> These things cannot be done with RunProcess.
> That's one-shot and wait till it exits.


Perfect and very handy to know the difference. Thanks for the explanation.



Regards,
  Graeme
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: TProcess vs RunProcess()

Graeme Geldenhuys-6
In reply to this post by Michael Van Canneyt
On 2017-01-26 14:28, Michael Van Canneyt wrote:
> TProcess gives you full access to the process. You can kill it,
> pause it, write to stdin, read from stdout.
>
> These things cannot be done with RunProcess.
> That's one-shot and wait till it exits.

Another difference.... simply so others might learn from this discussion
(like I did).

If you have a huge list of parameters, RunProcess() - at least under
Windows - will most likely hit a 260 character limit, and then fail to
actually execute. I don't actually know where that 260 limit originates
from (Windows API, Command Prompt, FPC etc).

Whereas if you use TProcess and the Parameters (TStrings) property, you
don't have such a limitation.

This was what I experienced.

Regards,
  Graeme

--
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://fpgui.sourceforge.net/

My public PGP key:  http://tinyurl.com/graeme-pgp
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: TProcess vs RunProcess()

Marco van de Voort
In our previous episode, Graeme Geldenhuys said:
> > pause it, write to stdin, read from stdout.
> >
> > These things cannot be done with RunProcess.
> > That's one-shot and wait till it exits.
>
> Another difference.... simply so others might learn from this discussion
> (like I did).

> If you have a huge list of parameters, RunProcess() - at least under
> Windows - will most likely hit a 260 character limit, and then fail to
> actually execute. I don't actually know where that 260 limit originates
> from (Windows API, Command Prompt, FPC etc).

Which runprocess are you talking about? I know a process.runcommand and a
sysutils.executeprocess, but not a runprocess.

> Whereas if you use TProcess and the Parameters (TStrings) property, you
> don't have such a limitation.
>
> This was what I experienced.

if you are talking about executeprocess, test again with trunk.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: TProcess vs RunProcess()

Graeme Geldenhuys-6
On 2017-01-27 17:43, Marco van de Voort wrote:
>
> Which runprocess are you talking about? I know a process.runcommand and a
> sysutils.executeprocess, but not a runprocess.

Apologies, our code is RunProcess(), which internally calls
SysUtils.ExecuteProcess()


>
> if you are talking about executeprocess, test again with trunk.

At the moment we are using FPC 2.6.4 and might move to FPC 3.0.2 when
that is final. Not using Trunk I'm afraid.

Was the 260 char limit removed/fixed in FPC trunk?


Regards,
  Graeme

--
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://fpgui.sourceforge.net/

My public PGP key:  http://tinyurl.com/graeme-pgp
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: TProcess vs RunProcess()

Marco van de Voort
In our previous episode, Graeme Geldenhuys said:
> >
> > if you are talking about executeprocess, test again with trunk.
>
> At the moment we are using FPC 2.6.4 and might move to FPC 3.0.2 when
> that is final. Not using Trunk I'm afraid.
>
> Was the 260 char limit removed/fixed in FPC trunk?

First, I'm not aware of a different between the two, so if you have tests
for that I'm interested. (at least if you are testing 3.0 or newer, but that
I can easily do myself if you have simple reproduction case).

The strange thing is that all variants use CreateProcess, but afaik only
executeprocess in trunk uses the -w variant. I started upgrading tprocess,
but it got out of hand, and I shelved the work.

Afaik by prefixing with \\.\ you can use larger paths with -w variants.

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: TProcess vs RunProcess()

noreply
In reply to this post by Michael Van Canneyt
On Thu, January 26, 2017 7:28 am, Michael Van Canneyt wrote:
>

>
> On Thu, 26 Jan 2017, Graeme Geldenhuys wrote:
>
>
>> Hi,
>>
>>
>> Whats the differences or pros and cons between using TProcess vs
>> RunProcess().
>>
>
> TProcess gives you full access to the process. You can kill it,
> pause it, write to stdin, read from stdout.
>
> These things cannot be done with RunProcess.
> That's one-shot and wait till it exits.
>

Similar to FpSystem() and ExecuteProcess, and then there is AssignStream
but assignstream/fpsystem unix only

http://www.freepascal.org/docs-html/rtl/sysutils/executeprocess.html

http://www.freepascal.org/docs-html/rtl/unix/assignstream.html

http://www.freepascal.org/docs-html/rtl/unix/fpsystem.html

I like fpsystem because it does not require absolute paths to commands,
too bad fpsystem not available on Windows.

All the above is used in webcmd (part of powtils) and also fpcRun ;-)

But I may try RunProcess/TProcess soon!
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: TProcess vs RunProcess()

noreply
In reply to this post by Graeme Geldenhuys-6
On Fri, January 27, 2017 10:24 am, Graeme Geldenhuys wrote:

> On 2017-01-26 14:28, Michael Van Canneyt wrote:
>
>> TProcess gives you full access to the process. You can kill it,
>> pause it, write to stdin, read from stdout.
>>
>> These things cannot be done with RunProcess.
>> That's one-shot and wait till it exits.
>>
>
> Another difference.... simply so others might learn from this discussion
> (like I did).
>
>
> If you have a huge list of parameters, RunProcess() - at least under
> Windows - will most likely hit a 260 character limit, and then fail to
> actually execute. I don't actually know where that 260 limit originates
> from (Windows API, Command Prompt, FPC etc).
>

One issue I discovered with ExecuteProcess, and fpSystem, is I cannot
figure out a way to capture StdErr... TProcess likely has more control
over this. Tried

Assign(StdErr, 'somefile.txt')
Rewrite(StdErr)

And didn't seem to work

So if you run a process and nothing prints to stdout, and there is some
data printed to stderr, I do not know how to capture it. Which makes me
want to try TProcess instead, but if I could do it with FpSystem and
ExecuteProcess that would be nice
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: TProcess vs RunProcess()

Marco van de Voort
In our previous episode, Lars said:
> Assign(StdErr, 'somefile.txt')
> Rewrite(StdErr)
>
> And didn't seem to work
>
> So if you run a process and nothing prints to stdout, and there is some
> data printed to stderr, I do not know how to capture it. Which makes me
> want to try TProcess instead, but if I could do it with FpSystem and
> ExecuteProcess that would be nice

Executeprocess doesn't support piping. It orignally was mainly meant as a
portable dos.exec with parameters passed separately and without string
length limits, implementable on RTL level.

If I were you I would simply look at tprocess sources how it pipes stderr.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: TProcess vs RunProcess()

Graeme Geldenhuys-6
In reply to this post by noreply
On 2017-01-29 07:59, Lars wrote:
> So if you run a process and nothing prints to stdout, and there is some
> data printed to stderr, I do not know how to capture it. Which makes me
> want to try TProcess instead


Indeed, that is one of the big differences between TProcess and
ExecuteProcess. I use TProcess often to execute something and then
capture the output results.  fpGUI's example IDE (Maximus) shows how it
can be done with TProcess.

Regards,
  Graeme

--
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://fpgui.sourceforge.net/

My public PGP key:  http://tinyurl.com/graeme-pgp
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: TProcess vs RunProcess()

Santiago A.
In reply to this post by Graeme Geldenhuys-6
El 27/01/17 a las 18:24, Graeme Geldenhuys escribió:
> If you have a huge list of parameters, RunProcess() - at least under
> Windows - will most likely hit a 260 character limit, and then fail to
> actually execute. I don't actually know where that 260 limit originates
> from (Windows API, Command Prompt, FPC etc).
it is easy to use more that 260 if you pass file names with long paths,
and people nowadays love to name directories and files with strings that
instead of titles look like abstracts.

In delphi and other languages I've hit may times the 260 character limit
in windows running  a Shell or using ShellApi (I supposed the limit was
255, not 260, there are always new things to learn) , I have also hit
that limit using the command line. Nevertheless using windows API
createprocess, there is not such limit (but the call is less handy and
more verbose), or maybe I haven't hit it because parameters and call
function are separated strings , so they are shorter strings. On the
other hand, CreateProcess must call an executable file, can't execute
internal commands like 'dir' or 'del', not a big deal but there it is. I
think that the limit is in the command line interpreters command.exe and
cmd.exe.

As far as I see, ExecuteProcess finally calls to an external symbol
named 'FPC_SYSCALLx'. And TProcess finally calls an external symbol
'FPC_SYSC_FORK'. Maybe ExecuteProcess is calling some kind of shell,
Freepascal developers will tell.

In my delphi days we implemented a procedure ("RunCommand" was it's
name, If memory serves), that was a warp to a complex call to
createprocess to avoid these problems. Maybe executeProcess could be
implemented this way. Don't know what are the drawbacks (slower, more
memory..?)

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: TProcess vs RunProcess()

Marco van de Voort
In our previous episode, Santi said:
> and people nowadays love to name directories and files with strings that
> instead of titles look like abstracts.
>
> In delphi and other languages I've hit may times the 260 character limit
> in windows running  a Shell or using ShellApi (I supposed the limit was
> 255, not 260, there are always new things to learn) , I have also hit
> that limit using the command line. Nevertheless using windows API
> createprocess, there is not such limit (but the call is less handy and
> more verbose)

There is a lot of confusion about this. (only -W calls can do larger
versions, and only if \\.\, but there are also reports that it works with -A
as long as the system is capable)

> or maybe I haven't hit it because parameters and call function are
> separated strings , so they are shorter strings.  On the other hand,
> CreateProcess must call an executable file, can't execute internal
> commands like 'dir' or 'del', not a big deal but there it is.  I think
> that the limit is in the command line interpreters command.exe and
> cmd.exe.

Internal commands are not internal to the OS, but internal to the shell, so
you need to execute a shell for that. On both Linux and Windows.
 
> As far as I see, ExecuteProcess finally calls to an external symbol
> named 'FPC_SYSCALLx'. And TProcess finally calls an external symbol
> 'FPC_SYSC_FORK'. Maybe ExecuteProcess is calling some kind of shell,
> Freepascal developers will tell.

On *nix, both will call fork followed by execve. On Windows it will call
CreateProcess, though executeprocess will call -W in trunk, and afaik
TProcess still -A.

> In my delphi days we implemented a procedure ("RunCommand" was it's
> name, If memory serves), that was a warp to a complex call to
> createprocess to avoid these problems. Maybe executeProcess could be
> implemented this way. Don't know what are the drawbacks (slower, more
> memory..?)

ExecuteProcess is to have some portable execution functionality in the base
RTL. For more complex cases there is the TProcess class that also supports
piping etc.

Some likely scenarios (e.g. run and capture stdout) have been prepared in
RunCommand, also in unit Process.

The main motivation for adding these is that 90+% of the TProcess and
CreateProcess usage in the forums was faulty (specially for larger outputs
or when the process also printed a lot on stderr (thanks Pierre!)).

Abstracting them in RunCommand reduces the amount of faulty code in play.

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: TProcess vs RunProcess()

noreply
In reply to this post by Santiago A.
On Sun, January 29, 2017 1:36 pm, Santi wrote:

>  Maybe ExecuteProcess is calling some kind of shell,
> Freepascal developers will tell.
>


To call the shell you send in cmd /C

FpSystem does this, but it's on unix

Calling the shell is useful for not requiring absolute paths to
commands... which is why I like fpSystem...

And to call a dos command you need a shell like cmd /C because calling a
dos command won't do anything, it's not an exe (there is no DIR.exe file
for example AFAIK).. so to call DIR (same as LS on unix) you need to
invoke a shell.


_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: TProcess vs RunProcess()

noreply
In reply to this post by Marco van de Voort
On Sun, January 29, 2017 2:41 pm, Marco van de Voort wrote:
> Some likely scenarios (e.g. run and capture stdout) have been prepared in
>  RunCommand, also in unit Process.
>
>
> The main motivation for adding these is that 90+% of the TProcess and
> CreateProcess usage in the forums was faulty (specially for larger outputs
>  or when the process also printed a lot on stderr (thanks Pierre!)).
>

So people were having issues with stderr going crazy (lots of prints to
stderr)?

And this is the problem I have with executeprocess: I want the results
from stderr, and need them, but can't figure out how to access them.. only
stdout is what I retrieve writes from.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: TProcess vs RunProcess()

noreply
In reply to this post by Graeme Geldenhuys-6
On Sun, January 29, 2017 10:46 am, Graeme Geldenhuys wrote:

> On 2017-01-29 07:59, Lars wrote:
>
>> So if you run a process and nothing prints to stdout, and there is some
>>  data printed to stderr, I do not know how to capture it. Which makes
>> me want to try TProcess instead
>
>
> Indeed, that is one of the big differences between TProcess and
> ExecuteProcess. I use TProcess often to execute something and then
> capture the output results.  fpGUI's example IDE (Maximus) shows how it
> can be done with TProcess.

Well I may modify executeprocess/fpsystem to deal with stderr if that is a
missing feature (may not be possible, haven't looked into it much).

Or, get assignstream working on windows and not just unix, but that will
be a bit of a big undertaking, and again may not be possible.

mseide/msegui may also have some tips in its source code as it captures a
lot of stuff and puts it into the ide in its custom widgets that trap
outputs, actually kind of a bit like a Blackbox ComponentPascal IDE from
switzerland, but still different ;-)

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: TProcess vs RunProcess()

noreply
In reply to this post by Marco van de Voort
On Sun, January 29, 2017 3:18 am, Marco van de Voort wrote:

> In our previous episode, Lars said:
>
>> Assign(StdErr, 'somefile.txt')
>> Rewrite(StdErr)
>>
>>
>> And didn't seem to work
>>
>>
>> So if you run a process and nothing prints to stdout, and there is some
>>  data printed to stderr, I do not know how to capture it. Which makes
>> me want to try TProcess instead, but if I could do it with FpSystem and
>> ExecuteProcess that would be nice
>>
>
> Executeprocess doesn't support piping. It orignally was mainly meant as a
>  portable dos.exec with parameters passed separately and without string
> length limits, implementable on RTL level.

But doesn't all processes report to stderr?
Ones that are created in a program do not report to stderr?

The OS makes it optional if you want to report to stderr?  Interesting..

Another idea is to modify AssignStream to work on windows as AFAIK it only
works on unix from the unix units of rtl..

Quote:
function AssignStream(var StreamIn, StreamOut, StreamErr: Text; const
prog: String): LongInt;
{
  Starts the program in 'prog' and makes its input, output and error output
  the other end of three pipes, which are the stdin, stdout and stderr of a
  program specified in 'prog'.

^^^ very useful function!

Reason I want to avoid TProcess is simply because cgi programs being
around 30-100K are easier to upload for people with limited internet
connection speeds. As soon as I pull in large units that involve lots of
classes the cgi programs become over 200-500K.

However I will use tprocess if it's a time save, which indeed it is
instead of rewriting assignstream for windows.

Plus, golang cgi exe's are about 2MB in size, so golang ain't got anything
on fpc at this moment (fpc still creates small programs compared to GoLang
runtime programs!)
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: TProcess vs RunProcess()

Jonas Maebe-3

Lars wrote:
> But doesn't all processes report to stderr?
> Ones that are created in a program do not report to stderr?
>
> The OS makes it optional if you want to report to stderr?  Interesting..

Every process (on Windows: every non-GUI process) on maintstream OSes
(and on many non-mainstream OSes too) has by default two standard file
handles available to which it can write output: stdout and stderr.


Jonas
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: TProcess vs RunProcess()

Jonas Maebe-3
In reply to this post by noreply
Lars wrote:
> Calling the shell is useful for not requiring absolute paths to
> commands... which is why I like fpSystem...

Never ship a program that uses fpSystem for that purpose, because it's
one of the most basic security holes you can have:
https://www.securecoding.cert.org/confluence/pages/viewpage.action?pageId=2130132


Jonas
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: TProcess vs RunProcess()

Marco van de Voort
In reply to this post by noreply
In our previous episode, Lars said:
> > Indeed, that is one of the big differences between TProcess and
> > ExecuteProcess. I use TProcess often to execute something and then
> > capture the output results.  fpGUI's example IDE (Maximus) shows how it
> > can be done with TProcess.
>
> Well I may modify executeprocess/fpsystem to deal with stderr if that is a
> missing feature (may not be possible, haven't looked into it much).

Not without effectively changing it into TProcess.
 
> Or, get assignstream working on windows and not just unix, but that will
> be a bit of a big undertaking, and again may not be possible.

Again, duplicating TProcess.

> mseide/msegui may also have some tips in its source code as it captures a
> lot of stuff and puts it into the ide in its custom widgets that trap
> outputs, actually kind of a bit like a Blackbox ComponentPascal IDE from
> switzerland, but still different ;-)

So thus Lazarus. It uses.... TProcess.

Hint: use TProcess :_)
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
12