Is System.IOResult thread-safe? (If not, I need to replace it.)

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Is System.IOResult thread-safe? (If not, I need to replace it.)

MegaBrutal
Hello,

I'm new on the list. I have a problem. I've been trying to make my
application thread-safe by using lockfiles. I used System.Rewrite to
lock the file, and System.IOResult to check the result. But I've
encountered problems, and such problems are most likely occur, when
more than one thread's trying to lock the file at the same time. I've
been thinking what might be the problem, and I wondered if
System.IOResult is thread-safe or not. I suppose not. :( Not a big
deal, though, I may use an other function to lock the file in a
thread-safe manner.

This is my function (that tries to lock the file):

function TSpoolObjectReader.Open: boolean;
begin
   {$I-} Rewrite(LockFile, 1); {$I+}
   if IOResult = 0 then begin
      try
         MailFile:= TFileStream.Create('spool\' + Name + '.eml', fmOpenRead);
         SpoolData:= TINIFile.Create('spool\' + Name + '.dat');
         Envelope.ReturnPath:= SpoolData.ReadString('SpoolObject',
'Return-Path', '');
         FOriginator:=
TIPNamePair.Create(SpoolData.ReadString('Originator', 'Name', ''),
            SpoolData.ReadString('Originator', 'IP', ''));
         FOpened:= true;
         Result:= true;
      except
         System.Close(LockFile);
         Result:= false;
      end;
   end
   else Result:= false;
end;

The variable, "LockFile" is an untyped file, previously assigned. I
decided to use Rewrite instead of another TFileStream, because I guess
TFileStream acquires much more resources to implement features I don't
even use.

I'd like to ask your opinion, how should I replace Rewrite & IOResult. My ideas:
1. Use another TFileStream to acquire the lockfile. It should work,
however as I stated before, I guess it would acquire much unused
resources.
2. Use "FileOpen", I think such function exists in the Free Pascal
library, however I never used it, and I'm not sure is it thread-safe.
It should be, after all.
3. Use direct WinAPI call, CreateFile. In this case, my unit wouldn't
be cross-platform anymore. However, now I'm writing my program for
Windows, I'm considering porting it to Linux someday, thus I'd like to
avoid platform-specific calls wherever possible.

I think option 2 is the most preferable, but I'd like to hear your
opinions as well, or maybe you have a better idea.


Best regards,
MegaBrutal
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Is System.IOResult thread-safe? (If not, I need to replace it.)

Jonas Maebe-2

On 10 Nov 2010, at 13:26, MegaBrutal wrote:

> I'm new on the list. I have a problem. I've been trying to make my
> application thread-safe by using lockfiles. I used System.Rewrite to
> lock the file, and System.IOResult to check the result. But I've
> encountered problems, and such problems are most likely occur, when
> more than one thread's trying to lock the file at the same time. I've
> been thinking what might be the problem, and I wondered if
> System.IOResult is thread-safe or not. I suppose not. :(

It is thread safe, but your function is not.

> Not a big
> deal, though, I may use an other function to lock the file in a
> thread-safe manner.
>
> This is my function (that tries to lock the file):
>
> function TSpoolObjectReader.Open: boolean;
> begin
>   {$I-} Rewrite(LockFile, 1); {$I+}

Rewrite does not cause an error if a file already exists. It simply  
overwrites it. You cannot first check whether it exists and then  
create it if it doesn't, because in between another thread could have  
created the lock file already.

I think the easiest way to get cross-platform lockfile behaviour, is to
a) create the lockfile at the start of your program (e.g., a rewrite  
followed by a close)
b) whenever you want to lock it, use

systutils.fileopen(lockfilename,fmOpenWrite or fmShareExclusive);

If that function returns -1, acquiring the lock failed.


Jonas
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Is System.IOResult thread-safe? (If not, I need to replace it.)

MegaBrutal
2010/11/10 Jonas Maebe <[hidden email]>:

>> Not a big
>> deal, though, I may use an other function to lock the file in a
>> thread-safe manner.
>>
>> This is my function (that tries to lock the file):
>>
>> function TSpoolObjectReader.Open: boolean;
>> begin
>>  {$I-} Rewrite(LockFile, 1); {$I+}
>
> Rewrite does not cause an error if a file already exists. It simply
> overwrites it. You cannot first check whether it exists and then create it
> if it doesn't, because in between another thread could have created the lock
> file already.

Well, my object should keep the lockfile opened as long as it's
working. If the file has already existed, but Rewrite is able to lock
it, that means that my application was shut down improperly. But it's
not a problem. Actually, the operating system should refuse to open
the file if a 2nd thread tries to open it. To sum it up, my code
doesn't work like you assume, that it creates and closes a file, and
it checks for the existence of the file to check whether it can work
on the object; instead it opens the file, and keeps it opened until it
finishes its work, then closes and deletes the file.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Is System.IOResult thread-safe? (If not, I need to replace it.)

Jonas Maebe-2

On 10 Nov 2010, at 13:48, MegaBrutal wrote:

> Well, my object should keep the lockfile opened as long as it's
> working. If the file has already existed, but Rewrite is able to lock
> it, that means that my application was shut down improperly. But it's
> not a problem. Actually, the operating system should refuse to open
> the file if a 2nd thread tries to open it. To sum it up, my code
> doesn't work like you assume, that it creates and closes a file, and
> it checks for the existence of the file to check whether it can work
> on the object; instead it opens the file, and keeps it opened until it
> finishes its work, then closes and deletes the file.

The problem is that calling rewrite on an already opened file works  
fine (at least on Unix platforms). This program runs fine on Mac OS X  
and Linux:

var
   f: file;
begin
{$i+}
   assign(f, 'test.txt');
   rewrite(f);
   rewrite(f);
end.

That's why I suggested the sysutils route, which enables you to  
specify the locking policy.


Jonas

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Is System.IOResult thread-safe? (If not, I need to replace it.)

MegaBrutal
2010/11/10 Jonas Maebe <[hidden email]>:

> The problem is that calling rewrite on an already opened file works fine (at
> least on Unix platforms). This program runs fine on Mac OS X and Linux:
>
> var
>  f: file;
> begin
> {$i+}
>  assign(f, 'test.txt');
>  rewrite(f);
>  rewrite(f);
> end.
>
> That's why I suggested the sysutils route, which enables you to specify the
> locking policy.

I should have noted this, but my "LockFile" is not a global variable.
It's a field of my object. So each object instance has its own
"LockFile", and threads create their own instance of the object.
However, they try to open the same file on the file system, but with
different file descriptors (under file descriptor, now I mean a Pascal
"file" typed variable).

The little program that would simulate it:

var
 f, g: file;
begin
{$i+}
 assign(f, 'test.txt');
 assign(g,'test.txt');
 rewrite(f,1);
 rewrite(g,1);
end.

I think it shouldn't work on any OS. Unless they open the file in
sharing mode, but I suppose Rewrite doesn't enable file sharing by
default.

Thanks for your suggestion anyway, I guess FileOpen would really
eliminate my problem, however not because of file sharing, like you
suspect.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Is System.IOResult thread-safe? (If not, I need to replace it.)

Jonas Maebe-2

On 10 Nov 2010, at 14:13, MegaBrutal wrote:

> I should have noted this, but my "LockFile" is not a global variable.
> It's a field of my object. So each object instance has its own
> "LockFile", and threads create their own instance of the object.
> However, they try to open the same file on the file system, but with
> different file descriptors (under file descriptor, now I mean a Pascal
> "file" typed variable).

That does not make any difference.

> The little program that would simulate it:
>
> var
>  f, g: file;
> begin
> {$i+}
>  assign(f, 'test.txt');
> assign(g,'test.txt');
>  rewrite(f,1);
>  rewrite(g,1);
> end.
>
> I think it shouldn't work on any OS.

It nevertheless does.

> Unless they open the file in
> sharing mode, but I suppose Rewrite doesn't enable file sharing by
> default.

On Unix-style OSes, Rewrite does not lock the file after opening it  
for writing (locking is a separate operation there).


Jonas
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Is System.IOResult thread-safe? (If not, I need to replace it.)

Sven Barth-2
On 10.11.2010 14:32, Jonas Maebe wrote:

>
> On 10 Nov 2010, at 14:13, MegaBrutal wrote:
>
>> I should have noted this, but my "LockFile" is not a global variable.
>> It's a field of my object. So each object instance has its own
>> "LockFile", and threads create their own instance of the object.
>> However, they try to open the same file on the file system, but with
>> different file descriptors (under file descriptor, now I mean a Pascal
>> "file" typed variable).
>
> That does not make any difference.
>
>> The little program that would simulate it:
>>
>> var
>> f, g: file;
>> begin
>> {$i+}
>> assign(f, 'test.txt');
>> assign(g,'test.txt');
>> rewrite(f,1);
>> rewrite(g,1);
>> end.
>>
>> I think it shouldn't work on any OS.
>
> It nevertheless does.
>
>> Unless they open the file in
>> sharing mode, but I suppose Rewrite doesn't enable file sharing by
>> default.
>
> On Unix-style OSes, Rewrite does not lock the file after opening it for
> writing (locking is a separate operation there).

And on Windows you have to explicitly declare that you want to deny(!)
read, write or delete operations of the file. By default a file is
shared for reading, writing and deleting.

Regards,
Sven
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

RE: Is System.IOResult thread-safe? (If not, I need to replace it.)

Cox, Stuart TRAN:EX
In reply to this post by Jonas Maebe-2
Thanks for this example, Jonas.

Stu Cox

On 10 Nov 2010, at 13:26, MegaBrutal wrote:

> I'm new on the list. I have a problem. I've been trying to make my
> application thread-safe by using lockfiles. I used System.Rewrite to
> lock the file, and System.IOResult to check the result. But I've
> encountered problems, and such problems are most likely occur, when
> more than one thread's trying to lock the file at the same time. I've
> been thinking what might be the problem, and I wondered if
> System.IOResult is thread-safe or not. I suppose not. :(

It is thread safe, but your function is not.

> Not a big
> deal, though, I may use an other function to lock the file in a
> thread-safe manner.
>
> This is my function (that tries to lock the file):
>
> function TSpoolObjectReader.Open: boolean;
> begin
>   {$I-} Rewrite(LockFile, 1); {$I+}

Rewrite does not cause an error if a file already exists. It simply  
overwrites it. You cannot first check whether it exists and then  
create it if it doesn't, because in between another thread could have  
created the lock file already.

I think the easiest way to get cross-platform lockfile behaviour, is to
a) create the lockfile at the start of your program (e.g., a rewrite  
followed by a close)
b) whenever you want to lock it, use

systutils.fileopen(lockfilename,fmOpenWrite or fmShareExclusive);

If that function returns -1, acquiring the lock failed.


Jonas
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Is System.IOResult thread-safe? (If not, I need to replace it.)

Vinzent Höfler
In reply to this post by MegaBrutal
On Fri, 19 Nov 2010 23:02:11 +0100, Cox, Stuart TRAN:EX  
<[hidden email]> wrote:

> I think the easiest way to get cross-platform lockfile behaviour, is to
> a) create the lockfile at the start of your program (e.g., a rewrite
> followed by a close)
> b) whenever you want to lock it, use
>
> systutils.fileopen(lockfilename,fmOpenWrite or fmShareExclusive);
>
> If that function returns -1, acquiring the lock failed.

This is bound to fail if someone deleted the lock file in between.

Something similar happened here with some commercial tool we use. It
then refused to open, because "another user already locked it".

Just thought, I mention it. No need to duplicate other's people mistakes.  
;)


Vinzent.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal