fphttclient, no way to specify a connect timeout

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

fphttclient, no way to specify a connect timeout

Luca Olivetti-2
Hello,

the TFpHttpClient has an IoTimeout property, which in turn sets the
IOTimeout of its FSocket (TInetSocket), however there's no way to set a
connection timeout.
I see that even overriding the SocketHandler wouldn't change that, since
it's connect is only called *after* the fpconnect call succeeds.
Also, I see no way to interrupt a pending connect from another thread.

My problem is that I'm using it in a thread, and, when I terminate the
application, in case the host is not responding/not available, I have to
wait the roughly 30 seconds that it takes for the connect to timeout.

I had the same problem with synapse, but there it has been solved
several years ago by introducing a connection timeout (besides, I can
abort a socket operation from another thread, though it only works under
windows).

Bye
--
Luca

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: fphttclient, no way to specify a connect timeout

Michael Van Canneyt


On Fri, 23 Feb 2018, Luca Olivetti wrote:

> Hello,
>
> the TFpHttpClient has an IoTimeout property, which in turn sets the
> IOTimeout of its FSocket (TInetSocket), however there's no way to set a
> connection timeout.
> I see that even overriding the SocketHandler wouldn't change that, since
> it's connect is only called *after* the fpconnect call succeeds.
> Also, I see no way to interrupt a pending connect from another thread.
>
> My problem is that I'm using it in a thread, and, when I terminate the
> application, in case the host is not responding/not available, I have to
> wait the roughly 30 seconds that it takes for the connect to timeout.
>
> I had the same problem with synapse, but there it has been solved
> several years ago by introducing a connection timeout (besides, I can
> abort a socket operation from another thread, though it only works under
> windows).

This is on my todo list. Unfortunately, the way sockets work, this means
putting the socket in non-blocking mode, call connect, and then use select
or some other means to wait till the socket is done with the connect, and
then put the socket again in blocking mode.

So not something to be quickly done in 10 minutes...

Michael.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: fphttclient, no way to specify a connect timeout

Free Pascal - General mailing list
Hi,

Στις 23/2/2018 5:17 μμ, ο Michael Van Canneyt έγραψε:


On Fri, 23 Feb 2018, Luca Olivetti wrote:

Hello,

the TFpHttpClient has an IoTimeout property, which in turn sets the IOTimeout of its FSocket (TInetSocket), however there's no way to set a connection timeout.
I see that even overriding the SocketHandler wouldn't change that, since it's connect is only called *after* the fpconnect call succeeds.
Also, I see no way to interrupt a pending connect from another thread.

< snip >
This is on my todo list. Unfortunately, the way sockets work, this means
putting the socket in non-blocking mode, call connect, and then use select
or some other means to wait till the socket is done with the connect, and
then put the socket again in blocking mode.
try this :

  copy the ssockets.pp from <fpc sources>\packages\fcl-net\src dir to your project dir and add the following line

SetIOTimeout(FIOTimeout);

to Procedure TInetSocket.Connect; like this

Procedure TInetSocket.Connect;

Var
  A : THostAddr;
  addr: TInetSockAddr;
  Res : Integer;

begin
  A := StrToHostAddr(FHost);
  if A.s_bytes[1] = 0 then
    With THostResolver.Create(Nil) do
      try
        If Not NameLookup(FHost) then
          raise ESocketError.Create(seHostNotFound, [FHost]);
        A:=HostAddress;
      finally
        free;
      end;
  addr.sin_family := AF_INET;
  addr.sin_port := ShortHostToNet(FPort);
  addr.sin_addr.s_addr := HostToNet(a.s_addr);
  SetIOTimeout(FIOTimeout);
  {$ifdef unix}
  <a class="moz-txt-link-freetext" href="Res:=ESysEINTR">Res:=ESysEINTR;
    While (Res=ESysEINTR) do
  {$endif}
      <a class="moz-txt-link-freetext" href="Res:=fpConnect%28Handle">Res:=fpConnect(Handle, @addr, sizeof(addr));
  If Not (Res<0) then
    if not FHandler.Connect then
      begin
      <a class="moz-txt-link-freetext" href="Res:=-1">Res:=-1;
      CloseSocket(Handle);
      end;
  If (Res<0) then
    Raise ESocketError.Create(seConnectFailed, [Format('%s:%d',[FHost, FPort])]);
end;

It works for me ....

regards,

-- 
Dimitrios Chr. Ioannidis
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: fphttclient, no way to specify a connect timeout

Luca Olivetti-2
El 23/02/18 a les 18:51, Dimitrios Chr. Ioannidis via fpc-pascal ha escrit:

>
>    copy the ssockets.pp from <fpc sources>\packages\fcl-net\src dir to
> your project dir and add the following line
>
> SetIOTimeout(FIOTimeout);
>
> to Procedure TInetSocket.Connect; like this


fphttpclient already does that before calling connect and it doesn't work.

As Michael said, the socket must be put in non-blocking mode during the
connection. This is how synapse does it (but I see that ssockets has no
provision for toggling between blocking/non-blocking mode):

     if FConnectionTimeout > 0 then
     begin
       // connect in non-blocking mode
       b := NonBlockMode;
       NonBlockMode := true;
       SockCheck(synsock.Connect(FSocket, Sin));
       if (FLastError = WSAEINPROGRESS) OR (FLastError = WSAEWOULDBLOCK)
then
         if not CanWrite(FConnectionTimeout) then
           FLastError := WSAETIMEDOUT;
       NonBlockMode := b;
     end
     else
       SockCheck(synsock.Connect(FSocket, Sin));


Bye
--
Luca

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: fphttclient, no way to specify a connect timeout

Free Pascal - General mailing list
Hi,


Στις 23/2/2018 8:24 μμ, ο Luca Olivetti έγραψε:

> El 23/02/18 a les 18:51, Dimitrios Chr. Ioannidis via fpc-pascal ha
> escrit:
>
>>
>>    copy the ssockets.pp from <fpc sources>\packages\fcl-net\src dir
>> to your project dir and add the following line
>>
>> SetIOTimeout(FIOTimeout);
>>
>> to Procedure TInetSocket.Connect; like this
>
>
> fphttpclient already does that before calling connect and it doesn't
> work.
>

emm it doesn't do that ...

did you tried it ... ;)

regards,

--
Dimitrios Chr. Ioannidis
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: fphttclient, no way to specify a connect timeout

Luca Olivetti-2
El 23/02/18 a les 19:34, Dimitrios Chr. Ioannidis via fpc-pascal ha escrit:

> Hi,
>
>
> Στις 23/2/2018 8:24 μμ, ο Luca Olivetti έγραψε:
>> El 23/02/18 a les 18:51, Dimitrios Chr. Ioannidis via fpc-pascal ha
>> escrit:
>>
>>>
>>>    copy the ssockets.pp from <fpc sources>\packages\fcl-net\src dir
>>> to your project dir and add the following line
>>>
>>> SetIOTimeout(FIOTimeout);
>>>
>>> to Procedure TInetSocket.Connect; like this
>>
>>
>> fphttpclient already does that before calling connect and it doesn't
>> work.
>>
>
> emm it doesn't do that ...

yes, it does (in procedure TFPCustomHTTPClient.ConnectToServer)

   FSocket:=TInetSocket.Create(AHost,APort,G);
   try
     if FIOTimeout<>0 then
       FSocket.IOTimeout:=FIOTimeout; <-----
     FSocket.Connect;
   except
     FreeAndNil(FSocket);
     Raise;
   end;

Bye
--
Luca
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: fphttclient, no way to specify a connect timeout

Free Pascal - General mailing list
Hi

Στις 23/2/2018 11:14 μμ, ο Luca Olivetti έγραψε:

<snip>

> yes, it does (in procedure TFPCustomHTTPClient.ConnectToServer)
>
>   FSocket:=TInetSocket.Create(AHost,APort,G);
>   try
>     if FIOTimeout<>0 then
>       FSocket.IOTimeout:=FIOTimeout; <-----
>     FSocket.Connect;
>   except
>     FreeAndNil(FSocket);
>     Raise;
>   end;

please follow the call stack.

The TFPCustomHTTPClient calls TInetSocket.Create constructor which it
calls the inherited constructor from TSocketStream.

So just before the FSocket.Connect, the above code assigns the internal
TSocketStream's FIOTimeout field ( as the TInetSocket is a descendant )
the value from the TFPCustomHTTPClient's FIOTimeout field.

Now the folowing addition in the procedure  TInetSocket.Connect uses the
parents (TSocketStream) private procedure SetIOTimeout and FIOTimeout
field to setup the socket timeout option.

ssockets.pp line 979

   addr.sin_family := AF_INET;
   addr.sin_port := ShortHostToNet(FPort);
   addr.sin_addr.s_addr := HostToNet(a.s_addr);
   SetIOTimeout(FIOTimeout); <------------------------------------
   {$ifdef unix}
   Res:=ESysEINTR;
     While (Res=ESysEINTR) do
   {$endif}
       Res:=fpConnect(Handle, @addr, sizeof(addr));


It works because the option is set after the socket is created and
before the socket connect call. Of course this is a hack/workaround
because if you need different timeouts for connect and receive then you
need in the onconnect event handler to change the TFPCustomHTTPClient's
FIOTimeout field again for the receive timeout .

AFAIK I'm sure Michael will solved it in a more elegant way than my 30
min code review ....

Just try it and tell me if it works.

regards,

--
Dimitrios Chr. Ioannidis
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: fphttclient, no way to specify a connect timeout

Luca Olivetti-2
El 23/02/18 a les 22:57, Dimitrios Chr. Ioannidis via fpc-pascal ha escrit:

> Hi
>
> Στις 23/2/2018 11:14 μμ, ο Luca Olivetti έγραψε:
>
> <snip>
>> yes, it does (in procedure TFPCustomHTTPClient.ConnectToServer)
>>
>>   FSocket:=TInetSocket.Create(AHost,APort,G);
>>   try
>>     if FIOTimeout<>0 then
>>       FSocket.IOTimeout:=FIOTimeout; <-----
>>     FSocket.Connect;
>>   except
>>     FreeAndNil(FSocket);
>>     Raise;
>>   end;
>
> please follow the call stack.
>
> The TFPCustomHTTPClient calls TInetSocket.Create constructor which it
> calls the inherited constructor from TSocketStream.

Which, since TFPHttpClient assigns a socket handler in G, it won't call
connect.
So the timeout set above is really done before the connect call.

Bye
--
Luca
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: fphttclient, no way to specify a connect timeout

Free Pascal - General mailing list
Hi,

On 2018-02-24 03:42, Luca Olivetti wrote:
> El 23/02/18 a les 22:57, Dimitrios Chr. Ioannidis via fpc-pascal ha
> escrit:
>> Hi
>>
>> Στις 23/2/2018 11:14 μμ, ο Luca Olivetti έγραψε:
<snip>
>> The TFPCustomHTTPClient calls TInetSocket.Create constructor which it
>> calls the inherited constructor from TSocketStream.
>
> Which, since TFPHttpClient assigns a socket handler in G, it won't call
> connect.
> So the timeout set above is really done before the connect call.

OK. I got it. Point taken.

You don't want to try to see if it works. Instead we exchange
unnecessary emails.

My bad. Apologies for pushing you.


regards,

PS: I'm sure Michael at least he'll take a look.

--
Dimitrios Chr. Ioannidis
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: fphttclient, no way to specify a connect timeout

Luca Olivetti-2
El 24/02/18 a les 10:19, Dimitrios Chr. Ioannidis via fpc-pascal ha escrit:

> Hi,
>
> On 2018-02-24 03:42, Luca Olivetti wrote:
>> El 23/02/18 a les 22:57, Dimitrios Chr. Ioannidis via fpc-pascal ha
>> escrit:
>>> Hi
>>>
>>> Στις 23/2/2018 11:14 μμ, ο Luca Olivetti έγραψε:
> <snip>
>>> The TFPCustomHTTPClient calls TInetSocket.Create constructor which it
>>> calls the inherited constructor from TSocketStream.
>>
>> Which, since TFPHttpClient assigns a socket handler in G, it won't
>> call connect.
>> So the timeout set above is really done before the connect call.
>
> OK. I got it. Point taken.

I don't think you did: what you are proposing is what TFpHttpClient
already does, I traced it, it really does the call to SetIoTimeout
before the connect call, and it doesn't work.
The only reliable way to enforce a connect timeout is to try to connect
in non-blocking mode.
Sure, with some combination of kernel/winsock version it could possibly
work (with mine it doesn't), but then it would be just enough to set the
IoTimeout in the TFpHttpClient (which I'm already doing), no need to
modify ssockets.pp.

Bye
--
Luca
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: fphttclient, no way to specify a connect timeout

Luca Olivetti-2
El 24/02/18 a les 13:00, Luca Olivetti ha escrit:

> Sure, with some combination of kernel/winsock version it could possibly
> work (with mine it doesn't), but then it would be just enough to set the
> IoTimeout in the TFpHttpClient (which I'm already doing), no need to
> modify ssockets.pp.

Correction: it works in linux (IIRC it didn't some years ago, now it's a
documented feature, see  https://linux.die.net/man/7/socket) but it
doesn't in windows (where the only option is to use a non-blocking socket).
Unluckily enough, my current target is windows :-(

Test procedure:

procedure TForm1.Button1Click(Sender: TObject);
var c:TFPHTTPClient;
   inicio: QWord;
begin
   c:=TFPHTTPClient.Create(nil);
   c.IOTimeout:=500;
   inicio:=GetTickCount64;
   try
     memo1.lines.add(c.Get('http://192.168.10.221/'));

   except
     on e:exception do
       memo1.lines.add(e.Message);
   end;
   caption:=IntToStr(GetTickCount64-inicio);
   c.free;

end;


The timeout is roughly 500 ms under linux, 18-20 seconds under windows
(xp/7, didn't test 10).


Bye
--
Luca

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: fphttclient, no way to specify a connect timeout

Free Pascal - General mailing list
In reply to this post by Luca Olivetti-2
Hi,

Στις 24/2/2018 2:00 μμ, ο Luca Olivetti έγραψε:

< snip >

>> OK. I got it. Point taken.
>
> I don't think you did <snip>

No I really did ;).

>
> The only reliable way to enforce a connect timeout is to try to
> connect in non-blocking mode.

Something like this ( sorry again in ssockects.pp ) for windows only, 
very draft code proof of concept which it works ( windows 10 here ) :

Procedure TInetSocket.Connect;

Var
   A : THostAddr;
   addr: TInetSockAddr;
   Res : Integer;

   nb: DWord;
   fds: WinSock2.TFDSet;
   opt: LongInt = 1;
   len: LongInt;
   tv : timeval;
   rslt: longint;

begin
   A := StrToHostAddr(FHost);
   if A.s_bytes[1] = 0 then
     With THostResolver.Create(Nil) do
       try
         If Not NameLookup(FHost) then
           raise ESocketError.Create(seHostNotFound, [FHost]);
         A:=HostAddress;
       finally
         free;
       end;
   addr.sin_family := AF_INET;
   addr.sin_port := ShortHostToNet(FPort);
   addr.sin_addr.s_addr := HostToNet(a.s_addr);

   tv.tv_sec  := IOTIMEOUT div 1000;
   tv.tv_usec := 0;

   WinSock2.FD_ZERO(fds);
   WinSock2.FD_SET(Handle, fds);
   nb := 1;
   rslt := WinSock2.ioctlsocket(HANDLE, WinSock2.FIONBIO, @nb);
   writeln(IntToStr(rslt));
     {$ifdef unix}
   Res := ESysEINTR;
   while (Res = ESysEINTR) do
     {$endif}
     Res := fpConnect(Handle, @addr, sizeof(addr));
     if WinSock2.select(Handle, nil, @fds, nil, @tv) > 0 then
     begin
       len := SizeOf(opt);
       if WinSock2.FD_ISSET(Handle, FDS) then
         fpgetsockopt(Handle, SOL_SOCKET, SO_ERROR, @OPT, @LEN);
       Res := Opt;
     end;
   nb := 0;
   WinSock2.ioctlsocket(HANDLE, WinSock2.FIONBIO, @nb);

   If Not (Res<0) then
     if not FHandler.Connect then
       begin
       Res:=-1;
       CloseSocket(Handle);
       end;
   If (Res<0) then
     Raise ESocketError.Create(seConnectFailed, [Format('%s:%d',[FHost,
FPort])]);
end;

regards,
--
Dimitrios Chr. Ioannidis
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: fphttclient, no way to specify a connect timeout

Luca Olivetti-2
El 24/02/18 a les 15:38, Dimitrios Chr. Ioannidis via fpc-pascal ha escrit:

>>
>> The only reliable way to enforce a connect timeout is to try to
>> connect in non-blocking mode.
>
> Something like this ( sorry again in ssockects.pp ) for windows only,
> very draft code proof of concept which it works ( windows 10 here ) :

Thank you but never mind: I changed my application to use synapse.

Bye
--
Luca
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal