Is there some example of an FPC program for use in svn hook calls (email)?

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Is there some example of an FPC program for use in svn hook calls (email)?

Bo Berglund
I am looking at switching from CVS to SVN as version control at my
company. We have used CVS(nt) since 16 years back, but it is no longer
working properly on new Windows versions (server side) so I am
planning to go to subversion instead.
For CVS I wrote a Delphi mailer program (CVSMailer) a long time ago,
which has been used to send commit, tag and import emails to the
developers from the script hooks in CVS.

Now I am looking at the same problem for svn and this time I need it
to be cross-platform, hence using FPC.
Before I start on it myself I wonder if someone here has already
written such a program for svn and if so, if it is available as open
source?
It should work from the post-commit hook in subversion.

Thanks.

--
Bo Berglund
Developer in Sweden

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Is there some example of an FPC program for use in svn hook calls (email)?

Adriaan van Os-2
Bo Berglund wrote:

> Now I am looking at the same problem for svn and this time I need it
> to be cross-platform, hence using FPC.
> Before I start on it myself I wonder if someone here has already
> written such a program for svn and if so, if it is available as open
> source?

I use sendmail <https://en.wikipedia.org/wiki/Sendmail> from FPC, using AssignStream, fpgeterrno
and PClose. Haven't tried that on Windows though.

Regards,

Adriaan van Os

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Is there some example of an FPC program for use in svn hook calls (email)?

Mark Morgan Lloyd-5
On 14/12/17 10:00, Adriaan van Os wrote:
> Bo Berglund wrote:
>> Now I am looking at the same problem for svn and this time I need it>
>> to be cross-platform, hence using FPC.> Before I start on it myself I
>> wonder if someone here has already> written such a program for svn and
>> if so, if it is available as open> source?
> I use sendmail <https://en.wikipedia.org/wiki/Sendmail> from FPC, using
> AssignStream, fpgeterrno and PClose. Haven't tried that on Windows though.

With the caveat that while Sendmail is a well-respected MTA, many unix
systems also provide a sendmail command as a generic mailer. On Windows
Blat is an alternative.

--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Is there some example of an FPC program for use in svn hook calls (email)?

Bo Berglund
In reply to this post by Adriaan van Os-2
On Thu, 14 Dec 2017 10:49:32 +0100, Adriaan van Os
<[hidden email]> wrote:

>Bo Berglund wrote:
>
>> Now I am looking at the same problem for svn and this time I need it
>> to be cross-platform, hence using FPC.
>> Before I start on it myself I wonder if someone here has already
>> written such a program for svn and if so, if it is available as open
>> source?
>
>I use sendmail <https://en.wikipedia.org/wiki/Sendmail> from FPC, using AssignStream, fpgeterrno
>and PClose. Haven't tried that on Windows though.
>

OK, I am looking for a program which will make a nice looking report
email to send. In CVSMailer I am parsing a lot of data from the
scripts in CVS and then build a html email showing the relevant info
like who made the commit, when and on which project as well as the
involved files with revisions and links to view them via VieVC.
But since subversion is so different I wanted to check for an already
existing solution before I start creating a program by myself.

BTW CVSMailer uses Indy for the network related stuff.


--
Bo Berglund
Developer in Sweden

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Is there some example of an FPC program for use in svn hook calls (email)?

Bo Berglund
In reply to this post by Mark Morgan Lloyd-5
On Thu, 14 Dec 2017 10:26:22 +0000, Mark Morgan Lloyd
<[hidden email]> wrote:

>On 14/12/17 10:00, Adriaan van Os wrote:
>> Bo Berglund wrote:
>>> Now I am looking at the same problem for svn and this time I need it>
>>> to be cross-platform, hence using FPC.> Before I start on it myself I
>>> wonder if someone here has already> written such a program for svn and
>>> if so, if it is available as open> source?
>> I use sendmail <https://en.wikipedia.org/wiki/Sendmail> from FPC, using
>> AssignStream, fpgeterrno and PClose. Haven't tried that on Windows though.
>
>With the caveat that while Sendmail is a well-respected MTA, many unix
>systems also provide a sendmail command as a generic mailer. On Windows
>Blat is an alternative.
But *sending email* is not the problem, I have done this in many
applications using Indy TIdSMTP. Instead it is getting data out oof
svn and formatting these in a friendly looking email to be sent....

Anyway I started on my own svn data collector and got stuck in a very
basic function...

I used the information in the wiki page below on how to execute a
command line program and retrieving its output data for processing.
Look at the section towards the bottom with subject "Reading large
output":
http://wiki.freepascal.org/Executing_External_Programs

To get the subversion commit data one has to use svnlook, which is a
command line program so I need to do this.

But when I use this approach I get no data at all, what could have
gone wrong here?

When I reach the end of the repeat loop there is no data at all to
process, i = 0!
But when I run the command on the command line manually I do get svn
data back.
So what did I do wrong when I used the wiki solution?

var
  //Command line parameters are:
  REPOS_PATH,         //Full path to repository
  REV,                //Revision number
  TXN_NAME:  string;  //Transaction name
  DataBuf,
  Parms:   TStringList;
  svncmd: string;
  SCD: TSvnCommitData;


function GetSvnData(Command: string; Parameters: TStringList; var
SvnData: TStringList): boolean;
{Command is the full path to the executable to run (<path>\svnlook.exe
or <path>/svnlook
Parameters contain the command line parameters. A typical parameter
set could be
 info
 -r
 2121
 <repository path on server>

 The result from the command will be stored into the SvnData
stringlist
 }
const
  BUF_SIZE = 2048; // Buffer size for reading the output in chunks
var
  AProcess     : TProcess;
  OutputStream : TStream;
  BytesRead    : longint;
  i: integer;
  Buffer       : array[1..BUF_SIZE] of byte;
begin
  Result := false;
  // Set up the process
  AProcess := TProcess.Create(nil);
  // Create a stream object to store the generated output in.
  OutputStream := TMemoryStream.Create;
  //Create the data buffer
  try
    AProcess.Executable := Command;

    // Process option poUsePipes has to be used so the output can be
captured.
    AProcess.Options := [poUsePipes];

    for i := 0 to Parameters.Count -1 do
      AProcess.Parameters.Add(Parameters[i]);

    i := 0;
    // Start the process (run the command)
    AProcess.Execute;

    // All generated output from AProcess is read in a loop until no
more data is available
    repeat
      // Get the new data from the process to a maximum of the buffer
size that was allocated.
      // Note that all read(...) calls will block except for the last
one, which returns 0 (zero).
      BytesRead := AProcess.Output.Read(Buffer, BUF_SIZE);

      // Add the bytes that were read to the stream for later usage
      OutputStream.Write(Buffer, BytesRead);
      Inc(i, BytesRead);
    until BytesRead = 0;  // Stop if no more data is available

    OutputStream.Position := 0; // Required to make sure all data is
copied from the start
    SvnData.LoadFromStream(OutputStream);
    Result := (i > 0); //(SvnData.Count > 0);
  finally
    OutputStream.Free;;
    AProcess.Free;
  end;

end;

begin
  //Check for the call parameters:
  if ParamCount() < 3 then
  begin
    Writeln('Call parameters missing!');
    exit;
  end;
  REPOS_PATH := ParamStr(1);
  REV := ParamStr(2);
  TXN_NAME := ParamStr(3);
  Parms := TStringList.Create;
  DataBuf := TStringList.Create;
  SCD := TSvnCommitData.Create;
  try
    // The commands for Windows and *nix are different hence the
$IFDEFs
    {$IFDEF Windows}
      svncmd := 'C:\Programs\Subversion\svnlook.exe';
    {$ENDIF Windows}
    {$IFDEF Unix}
      svncmd := '/usr/bin/svnlook';
    {$ENDIF Unix}
    //Build the call for svnlook
    {1. Get the starting information on the operation
        svnlook info -r <REV> <REPOS_PATH>
    }
    Parms.Add('info');
    Parms.Add('-r');
    Parms.Add('REV');
    Parms.Add('REPOS_PATH');
    if GetSvnData(svncmd, Parms, DataBuf) then
    begin
      SCD.UserName := DataBuf[0];
      SCD.CommitTime := DataBuf[1];
      SCD.Logmessage := DataBuf[3];
    end;
  finally
    SCD.Free;
    DataBuf.Free;
  end;
end.



--
Bo Berglund
Developer in Sweden

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Is there some example of an FPC program for use in svn hook calls (email)?

Paul Breneman
On 12/14/2017 12:02 PM, Bo Berglund wrote:

> On Thu, 14 Dec 2017 10:26:22 +0000, Mark Morgan Lloyd
> <[hidden email]> wrote:
>
>> On 14/12/17 10:00, Adriaan van Os wrote:
>>> Bo Berglund wrote:
>>>> Now I am looking at the same problem for svn and this time I need it>
>>>> to be cross-platform, hence using FPC.> Before I start on it myself I
>>>> wonder if someone here has already> written such a program for svn and
>>>> if so, if it is available as open> source?
>>> I use sendmail <https://en.wikipedia.org/wiki/Sendmail> from FPC, using
>>> AssignStream, fpgeterrno and PClose. Haven't tried that on Windows though.
>>
>> With the caveat that while Sendmail is a well-respected MTA, many unix
>> systems also provide a sendmail command as a generic mailer. On Windows
>> Blat is an alternative.
> But *sending email* is not the problem, I have done this in many
> applications using Indy TIdSMTP. Instead it is getting data out oof
> svn and formatting these in a friendly looking email to be sent....
>
> Anyway I started on my own svn data collector and got stuck in a very
> basic function...
>
> I used the information in the wiki page below on how to execute a
> command line program and retrieving its output data for processing.
> Look at the section towards the bottom with subject "Reading large
> output":
> http://wiki.freepascal.org/Executing_External_Programs
>
> To get the subversion commit data one has to use svnlook, which is a
> command line program so I need to do this.
>
> But when I use this approach I get no data at all, what could have
> gone wrong here?
>
> When I reach the end of the repeat loop there is no data at all to
> process, i = 0!
> But when I run the command on the command line manually I do get svn
> data back.
> So what did I do wrong when I used the wiki solution?

I recently used the same code, so I tried to execute your code (below).
I'd suggest adding the two lines (shown below) to troubleshoot. I didn't
execute svnlook but ppcx64 with parameter -h and that works okay with
your code.

>
> var
>    //Command line parameters are:
>    REPOS_PATH,         //Full path to repository
>    REV,                //Revision number
>    TXN_NAME:  string;  //Transaction name
>    DataBuf,
>    Parms:   TStringList;
>    svncmd: string;
>    SCD: TSvnCommitData;
>
>
> function GetSvnData(Command: string; Parameters: TStringList; var
> SvnData: TStringList): boolean;
> {Command is the full path to the executable to run (<path>\svnlook.exe
> or <path>/svnlook
> Parameters contain the command line parameters. A typical parameter
> set could be
>   info
>   -r
>   2121
>   <repository path on server>
>
>   The result from the command will be stored into the SvnData
> stringlist
>   }
> const
>    BUF_SIZE = 2048; // Buffer size for reading the output in chunks
> var
>    AProcess     : TProcess;
>    OutputStream : TStream;
>    BytesRead    : longint;
>    i: integer;
>    Buffer       : array[1..BUF_SIZE] of byte;
> begin
>    Result := false;
>    // Set up the process
>    AProcess := TProcess.Create(nil);
>    // Create a stream object to store the generated output in.
>    OutputStream := TMemoryStream.Create;
>    //Create the data buffer
>    try
>      AProcess.Executable := Command;
>
>      // Process option poUsePipes has to be used so the output can be
> captured.
>      AProcess.Options := [poUsePipes];
>
>      for i := 0 to Parameters.Count -1 do
>        AProcess.Parameters.Add(Parameters[i]);
>
>      i := 0;
>      // Start the process (run the command)
>      AProcess.Execute;
>
>      // All generated output from AProcess is read in a loop until no
> more data is available
>      repeat
>        // Get the new data from the process to a maximum of the buffer
> size that was allocated.
>        // Note that all read(...) calls will block except for the last
> one, which returns 0 (zero).
>        BytesRead := AProcess.Output.Read(Buffer, BUF_SIZE);
>
>        // Add the bytes that were read to the stream for later usage
>        OutputStream.Write(Buffer, BytesRead);
>        Inc(i, BytesRead);
>      until BytesRead = 0;  // Stop if no more data is available
>
>      OutputStream.Position := 0; // Required to make sure all data is
> copied from the start
>      SvnData.LoadFromStream(OutputStream);
>      Result := (i > 0); //(SvnData.Count > 0);

// I'd suggest adding these two lines to troubleshoot:
     writeln(SvnData.Text);
     writeln('Number of bytes = ', i);

>    finally
>      OutputStream.Free;;
>      AProcess.Free;
>    end;
>
> end;
>
> begin
>    //Check for the call parameters:
>    if ParamCount() < 3 then
>    begin
>      Writeln('Call parameters missing!');
>      exit;
>    end;
>    REPOS_PATH := ParamStr(1);
>    REV := ParamStr(2);
>    TXN_NAME := ParamStr(3);
>    Parms := TStringList.Create;
>    DataBuf := TStringList.Create;
>    SCD := TSvnCommitData.Create;
>    try
>      // The commands for Windows and *nix are different hence the
> $IFDEFs
>      {$IFDEF Windows}
>        svncmd := 'C:\Programs\Subversion\svnlook.exe';
>      {$ENDIF Windows}
>      {$IFDEF Unix}
>        svncmd := '/usr/bin/svnlook';
>      {$ENDIF Unix}
>      //Build the call for svnlook
>      {1. Get the starting information on the operation
>          svnlook info -r <REV> <REPOS_PATH>
>      }
>      Parms.Add('info');
>      Parms.Add('-r');
>      Parms.Add('REV');
>      Parms.Add('REPOS_PATH');
>      if GetSvnData(svncmd, Parms, DataBuf) then
>      begin
>        SCD.UserName := DataBuf[0];
>        SCD.CommitTime := DataBuf[1];
>        SCD.Logmessage := DataBuf[3];
>      end;
>    finally
>      SCD.Free;
>      DataBuf.Free;
>    end;
> end.
>
>
>

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Is there some example of an FPC program for use in svn hook calls (email)?

Marco van de Voort
In reply to this post by Bo Berglund
In our previous episode, Bo Berglund said:
> But *sending email* is not the problem, I have done this in many
> applications using Indy TIdSMTP. Instead it is getting data out oof
> svn and formatting these in a friendly looking email to be sent....

Actually process.runcommand was developed to get the output of SVN commands.

It is used to create the mergelogs http://www.stack.nl/~marcov/mergelogs26/
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Is there some example of an FPC program for use in svn hook calls (email)?

Bo Berglund
On Thu, 14 Dec 2017 19:37:55 +0100 (CET),
[hidden email] (Marco van de Voort) wrote:

>In our previous episode, Bo Berglund said:
>> But *sending email* is not the problem, I have done this in many
>> applications using Indy TIdSMTP. Instead it is getting data out oof
>> svn and formatting these in a friendly looking email to be sent....
>
>Actually process.runcommand was developed to get the output of SVN commands.
>
>It is used to create the mergelogs http://www.stack.nl/~marcov/mergelogs26/

Thanks for your suggestion!

Do you mean replace the current code:

    AProcess.Executable := Command;
    AProcess.Options := [poUsePipes];
    for i := 0 to Parameters.Count -1 do
      AProcess.Parameters.Add(Parameters[i]);

    AProcess.Execute;
    repeat
      BytesRead := AProcess.Output.Read(Buffer, BUF_SIZE);
      OutputStream.Write(Buffer, BytesRead);
    until BytesRead = 0;  // Stop if no more data is available

    OutputStream.Position := 0;
    SvnData.LoadFromStream(OutputStream);

With something like this:

   if AProcess.RunCommand(Command, Parameters.Strings, sTmp) then
   begin
     SvnData.Text := sTmp);
     Result := true;
   end;

Are all of those preliminaries unnecessary and could be discarded
then?

BTW:
How can I view what is actually happening in the command window when
debugging the code?
When I started this (in Lazarus 1.6) I selected a new "Program" rather
than a "Console application".


--
Bo Berglund
Developer in Sweden

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Is there some example of an FPC program for use in svn hook calls (email)?

Giuliano Colla
In reply to this post by Bo Berglund

Il 14/12/2017 18:02, Bo Berglund ha scritto:

But *sending email* is not the problem, I have done this in many
applications using Indy TIdSMTP. Instead it is getting data out oof
svn and formatting these in a friendly looking email to be sent....

You might find useful guidelines in the fpcup and fpcupdeluxe programs which extensively use svn to automatically update the fpc and lazarus versions from svn:

https://github.com/LongDirtyAnimAlf/Reiniero-fpcup

https://github.com/newpascal/fpcupdeluxe

(btw they provide also a comfortable way to keep different fpc and lazarus versions on the same platform, to set them up for crosscompiling, etc.)

Giuliano


_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Is there some example of an FPC program for use in svn hook calls (email)?

Bo Berglund
In reply to this post by Bo Berglund
On Thu, 14 Dec 2017 19:54:49 +0100, Bo Berglund
<[hidden email]> wrote:

>   if AProcess.RunCommand(Command, Parameters.Strings, sTmp) then
>   begin
>     SvnData.Text := sTmp);
>     Result := true;
>   end;
>

It works when this is changed to:
var
  Params: array of string;
  ...
begin
....
    SetLength(Params, Parameters.Count);
    for i := 0 to Parameters.Count-1 do
      Params[i] := Parameters[i];
    if RunCommand(Command, Params, sTmp) then
    begin
      SvnData.Text := sTmp;
      Result := true;
      writeln(SvnData.Text); <= Just for testing
    end;

But first I had to fix the real bug:

*** WHAT A STUPID MISTAKE! BANGING MY HEAD! ***
In the caller:

    Parms.Add('info');
    Parms.Add('-r');
    Parms.Add('REV');
    Parms.Add('REPOS_PATH');
    if GetSvnData(svncmd, Parms, DataBuf) then

The last two Parms.Add's are stupidly wrong. The quotes are leftovers
from when I had literals there as a first test. REV and REPOS_PATH are
*variables*! Removing the quotes is what it took to make it work....

But a good thing came out of this anyway because I was steered towards
the use of RunCommand(), which works just fine and simplifies the code
enormously!
Thanks for that!

--
Bo Berglund
Developer in Sweden

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal