Call function in shared library from multiple threads

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

Call function in shared library from multiple threads

Krzysztof
Hi,

I'm wondering if shared libraries (on Linux) support call exported function from multiple threads. I made small demo. Threads send JSON PChar into shared lib function which parse it and log in syslog. No global vars, everything locally in function. But even that I'm getting random errors like:

29.03.2017 16:33    kernel    [26260.212884] traps: test_libmjrende[12177] general protection ip:7fe823eed2f1 sp:7fe8238571e0 error:0 in libmjrender.so[7fe823de0000+284000]
29.03.2017 16:36    kernel    [26443.509924] test_libmjrende[12247]: segfault at 7f5e02d179f0 ip 00007f5e02d1797c sp 00007f5e02d179f0 error 7 in libmjrender.so[7f5e02c02000+286000]
29.03.2017 16:42    kernel    [26819.543353] test_libmjrende[12350]: segfault at 7f1f1d65b0a0 ip 00007f1f1d65b0a0 sp 00007f1f0c000fd8 error 15 in libmjrender.so[7f1f1d62e000+112000]
29.03.2017 16:39    kernel    [26627.190568] test_libmjrende[12297]: segfault at 8 ip 00007fe4c6851c2c sp 00007fe4c0766c90 error 4 in libc-2.23.so[7fe4c67d0000+1bf000]

I guess that just not all RTL are thread safe and this is not supported. Even critical sections don't work here.
This is function:

procedure Test(ACounter: Integer; AJson: PChar); cdecl;
var
  js: TJSONObject=nil;
  jp: TJSONParser=nil;
  i: Integer;
  s: String;
begin
  syslog(log_info, 'Enter: %d', [ACounter]);
  try try
    jp := TJSONParser.Create(AJson);
    js := TJSONObject(jp.Parse);
    FreeAndNil(jp);
    Sleep(500);
    i := js.Get('1', -1);
    Sleep(500);
    s := js.Get('2', '');
    syslog(log_info, 'Got %d: %d %s', [ACounter, i, pchar(s)]);
  except on e: Exception do
    begin
      syslog(log_info, 'Error: %s', [PChar(e.Message)]);
      raise;
    end;
  end;
  finally
    FreeAndNil(jp);
    FreeAndNil(js);
    syslog(log_info, 'Leave: %d', [ACounter]);
  end;
end;  

Attached also full source with library and client.

Regards

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

testlib.7z (3K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Call function in shared library from multiple threads

Michael Van Canneyt


On Thu, 30 Mar 2017, Krzysztof wrote:

> Hi,
>
> I'm wondering if shared libraries (on Linux) support call exported function
> from multiple threads. I made small demo. Threads send JSON PChar into
> shared lib function which parse it and log in syslog. No global vars,
> everything locally in function. But even that I'm getting random errors
> like:

It should be supported but must have threads support compiled-in.

You didn't add cthreads to the uses list of the library, so your
library will not function when called from multiple threads.

Of course, you need to keep your code thread-safe.
And indeed, not all RTL calls are thread safe.
If any initialization must happen, it must happen the first time the library
is loaded (or it must be done in critical sections).

Michael.

>
> 29.03.2017 16:33    kernel    [26260.212884] traps: test_libmjrende[12177]
> general protection ip:7fe823eed2f1 sp:7fe8238571e0 error:0 in
> libmjrender.so[7fe823de0000+284000]
> 29.03.2017 16:36    kernel    [26443.509924] test_libmjrende[12247]:
> segfault at 7f5e02d179f0 ip 00007f5e02d1797c sp 00007f5e02d179f0 error 7 in
> libmjrender.so[7f5e02c02000+286000]
> 29.03.2017 16:42    kernel    [26819.543353] test_libmjrende[12350]:
> segfault at 7f1f1d65b0a0 ip 00007f1f1d65b0a0 sp 00007f1f0c000fd8 error 15
> in libmjrender.so[7f1f1d62e000+112000]
> 29.03.2017 16:39    kernel    [26627.190568] test_libmjrende[12297]:
> segfault at 8 ip 00007fe4c6851c2c sp 00007fe4c0766c90 error 4 in
> libc-2.23.so[7fe4c67d0000+1bf000]
>
> I guess that just not all RTL are thread safe and this is not supported.
> Even critical sections don't work here.
> This is function:
>
> procedure Test(ACounter: Integer; AJson: PChar); cdecl;
> var
>  js: TJSONObject=nil;
>  jp: TJSONParser=nil;
>  i: Integer;
>  s: String;
> begin
>  syslog(log_info, 'Enter: %d', [ACounter]);
>  try try
>    jp := TJSONParser.Create(AJson);
>    js := TJSONObject(jp.Parse);
>    FreeAndNil(jp);
>    Sleep(500);
>    i := js.Get('1', -1);
>    Sleep(500);
>    s := js.Get('2', '');
>    syslog(log_info, 'Got %d: %d %s', [ACounter, i, pchar(s)]);
>  except on e: Exception do
>    begin
>      syslog(log_info, 'Error: %s', [PChar(e.Message)]);
>      raise;
>    end;
>  end;
>  finally
>    FreeAndNil(jp);
>    FreeAndNil(js);
>    syslog(log_info, 'Leave: %d', [ACounter]);
>  end;
> end;
>
> Attached also full source with library and client.
>
> Regards
>
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Call function in shared library from multiple threads

Krzysztof
Yes, I'm aware of cthreads and tested it too (without it also critical sections don't work) but it doesn't help with attached examples. Still same errors in logs

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Call function in shared library from multiple threads

Michael Van Canneyt


On Thu, 30 Mar 2017, Krzysztof wrote:

> Yes, I'm aware of cthreads and tested it too (without it also critical
> sections don't work) but it doesn't help with attached examples. Still same
> errors in logs

Nevertheless, that should work.

I've used it in multi-threaded apache modules.

Michael.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Call function in shared library from multiple threads

Michael Van Canneyt


On Thu, 30 Mar 2017, Michael Van Canneyt wrote:

>
>
> On Thu, 30 Mar 2017, Krzysztof wrote:
>
>> Yes, I'm aware of cthreads and tested it too (without it also critical
>> sections don't work) but it doesn't help with attached examples. Still same
>> errors in logs
>
> Nevertheless, that should work.

I should add that WITHOUT using cthreads, it certainly will NOT work.

Michael.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Call function in shared library from multiple threads

Krzysztof
In reply to this post by Michael Van Canneyt
2017-03-30 15:19 GMT+02:00 Michael Van Canneyt <[hidden email]>

Nevertheless, that should work.

I've used it in multi-threaded apache modules.

Well it doesn't work or fpjson module is not thread safe in this case. You can test it by yourself with attached demo (first post). Try it few times because sometimes if you are lucky then there is no error

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Call function in shared library from multiple threads

Michael Van Canneyt


On Thu, 30 Mar 2017, Krzysztof wrote:

> 2017-03-30 15:19 GMT+02:00 Michael Van Canneyt <[hidden email]>
>>
>>
>> Nevertheless, that should work.
>>
>> I've used it in multi-threaded apache modules.
>>
>
> Well it doesn't work or fpjson module is not thread safe in this case. You
> can test it by yourself with attached demo (first post). Try it few times
> because sometimes if you are lucky then there is no error

If I find a moment, I will test.

Michael.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Call function in shared library from multiple threads

Michael Van Canneyt


On Thu, 30 Mar 2017, Michael Van Canneyt wrote:

>
>
> On Thu, 30 Mar 2017, Krzysztof wrote:
>
>> 2017-03-30 15:19 GMT+02:00 Michael Van Canneyt <[hidden email]>
>>>
>>>
>>> Nevertheless, that should work.
>>>
>>> I've used it in multi-threaded apache modules.
>>>
>>
>> Well it doesn't work or fpjson module is not thread safe in this case. You
>> can test it by yourself with attached demo (first post). Try it few times
>> because sometimes if you are lucky then there is no error
>
> If I find a moment, I will test.

I can reproduce the problem.

At least partially it is related to the threads mechanism not being initialized
correctly.

To fix that, you can do the following.
In the library startup code, create a dummy thread.
This will initialize the threads mechanism:

library testlib;

{$mode objfpc}{$H+}

uses
   cthreads, cmem, Classes, testlibu;

exports
   InitTest, FinalizeTest, Test;


begin
   with TThread.Create(False) do

end.


After this, I no longer get the problem (which doesn't mean there are no
other possible problems :) )

Michael.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Call function in shared library from multiple threads

Krzysztof
Thanks a lot! Seems to working fine now

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Call function in shared library from multiple threads

fredvs
In reply to this post by Michael Van Canneyt
Hello.

Michael Van Canneyt wrote
To fix that, you can do the following.
In the library startup code, create a dummy thread.
This will initialize the threads mechanism:

   with TThread.Create(False) do

end.
I use this for initialize my libraries:

 With TThread.Create(False) do Terminate;

It works ok but there is that warning after compiling:

Z:\home\fred\uos\examples\uos.pas(7438,29) Warning: (4046) Constructing a class "TThread" with abstract method "Execute"
Huh, is it Is it serious doctor?

Fre;D

Many thanks ;-)
Reply | Threaded
Open this post in threaded view
|

Re: Call function in shared library from multiple threads

Michael Van Canneyt


On Fri, 31 Mar 2017, fredvs wrote:

> Hello.
>
>
> Michael Van Canneyt wrote
>> To fix that, you can do the following.
>> In the library startup code, create a dummy thread.
>> This will initialize the threads mechanism:
>>
>>    with TThread.Create(False) do
>>
>> end.
>
> I use this for initialize my libraries:


Best is probably:

Type
   TDummyThread = Class(TThread)
   public
     procedure execute; override;
   end;

procedure TDummyThread.Execute;

begin
   FreeOnTerminate:=True;
   Terminate;
end;


begin
   TDummyThread.Create(True)
end.


Michael.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Call function in shared library from multiple threads

Henry Vermaak
In reply to this post by fredvs
On Fri, Mar 31, 2017 at 08:42:24AM -0700, fredvs wrote:
> > Z:\home\fred\uos\examples\uos.pas(7438,29) Warning: (4046) Constructing a
> > class "TThread" with abstract method "Execute"
>
> Huh, is it Is it serious doctor?

I use this:

function DummyThread(param: pointer): ptrint;
begin
  Result := 0;
  EndThread(Result);
end;

begin
  BeginThread(@DummyThread);
  ...
  ...
end.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Call function in shared library from multiple threads

fredvs
In reply to this post by Michael Van Canneyt
Michael Van Canneyt wrote
Best is probably:

Type
   TDummyThread = Class(TThread)
   public
     procedure execute; override;
   end;

procedure TDummyThread.Execute;

begin
   FreeOnTerminate:=True;
   Terminate;
end;


begin
   TDummyThread.Create(True)
end.

OK, perfect, no more warning, thanks doctor.

Fre;D
Many thanks ;-)
Reply | Threaded
Open this post in threaded view
|

Re: Call function in shared library from multiple threads

silvioprog
In reply to this post by Henry Vermaak
On Fri, Mar 31, 2017 at 1:15 PM, Henry Vermaak <[hidden email]> wrote:
On Fri, Mar 31, 2017 at 08:42:24AM -0700, fredvs wrote:
> > Z:\home\fred\uos\examples\uos.pas(7438,29) Warning: (4046) Constructing a
> > class "TThread" with abstract method "Execute"
>
> Huh, is it Is it serious doctor?

I use this:

function DummyThread(param: pointer): ptrint;
begin
  Result := 0;
  EndThread(Result);
end;

begin
  BeginThread(@DummyThread);
  ...
  ...
end.

What about calling TM directly? Something like this:

uses cthreads;
var
  TM: TThreadManager;
begin
  TM := Default(TThreadManager); // just hiding hint 'variable TM doesn't seem to be initialized'
  GetThreadManager(TM);
  IsMultiThread := TM.InitManager;
...
end.

I can't test it now (I would like to debug it asap), but I think FPC offer some option to start the TM without starting a new -- dummy -- thread. o.O

--
Silvio Clécio

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Call function in shared library from multiple threads

fredvs
In reply to this post by Michael Van Canneyt
>  TDummyThread.Create(True)

Huh, I have memory leak with this.

Should it not be: --> TDummyThread.Create(false) ? (because with this, no memory leak)

Fre;D

Many thanks ;-)
Reply | Threaded
Open this post in threaded view
|

Re: Call function in shared library from multiple threads

fredvs
>>   TDummyThread.Create(True)

> Huh, I have memory leak with this.

>  Should it not be: --> TDummyThread.Create(false) ? (because with this, no memory leak)

Hello.

Sorry to come back with this but without answer I am still full of doubt.

Is TDummyThread.Create(True)  wanted and so the dummy thread must exist during all the application and freed at end of application ?

Or is TDummyThread.Create(false) ok ?

Thanks.

Fre;D
Many thanks ;-)