File handling: the TFileStream or the classical way is faster and more efficient?

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

File handling: the TFileStream or the classical way is faster and more efficient?

Géza Kovacs Géza
Hi All!

What is the faster and more efficient, using the TFileStream or the
classical way ("file of byte", or "file") type with
blockread/blockwrite and the other well-know procedures?
What is the better, faster on large files?

See the two example code below.

Program FileCopy_stream;
{$mode objfpc} {$H+}
uses classes,SysUtils;
var
        BytesRead,TotalBytesRead : Int64;
        puffer : array [1..1048576] of byte;
        InF, OutF : TFileStream;
begin
                TotalBytesRead := 0;
                BytesRead := 0;
                try
                        InF:= TFileStream.Create(ParamSTR(1),fmOpenRead);
                        OutF := TFileStream.Create(ParamSTR(2),fmCreate);
                                repeat
                                        BytesRead := InF.Read(puffer,sizeof(puffer));
                                        inc(TotalBytesRead, BytesRead);
                                        OutF.Write(puffer,BytesRead);
                                until (TotalBytesRead = InF.size);
                        finally
                        FreeAndNil(InF);
                        FreeAndNil(OutF);
                end;
end.

Program FileCopy_Classic;
{$mode objfpc} {$H+}
var
        NumRead, NumWritten : LongInt;
        puffer : array [1..1048576] of byte;
        InF,OutF : file of byte;
begin
                assign(InF,ParamStr(1));
                ReSet(InF);
                Assign(OutF,ParamStr(2));
                ReWrite(OutF);
                repeat
                        BlockRead(InF,puffer,SizeOf(puffer),NumRead);
                        BlockWrite(OutF,puffer,NumRead,NumWritten);
                until (NumRead = 0);
                close(InF);
                close(OutF);
end.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: File handling: the TFileStream or the classical way is faster and more efficient?

Michael Van Canneyt


On Mon, 25 May 2015, Géza Kovacs Géza wrote:

> Hi All!
>
> What is the faster and more efficient, using the TFileStream or the
> classical way ("file of byte", or "file") type with
> blockread/blockwrite and the other well-know procedures?
> What is the better, faster on large files?

TFileStream offers more flexibility, but as far as efficiency is concerned :
I think that will be the same for both approaches.

For example, your first example can be done easier in a single statement:

OutF.CopyFrom(Inf,0);

Michael.

>
> See the two example code below.
>
> Program FileCopy_stream;
> {$mode objfpc} {$H+}
> uses classes,SysUtils;
> var
> BytesRead,TotalBytesRead : Int64;
> puffer : array [1..1048576] of byte;
> InF, OutF : TFileStream;
> begin
> TotalBytesRead := 0;
> BytesRead := 0;
> try
> InF:= TFileStream.Create(ParamSTR(1),fmOpenRead);
> OutF := TFileStream.Create(ParamSTR(2),fmCreate);
> repeat
> BytesRead := InF.Read(puffer,sizeof(puffer));
> inc(TotalBytesRead, BytesRead);
> OutF.Write(puffer,BytesRead);
> until (TotalBytesRead = InF.size);
> finally
> FreeAndNil(InF);
> FreeAndNil(OutF);
> end;
> end.
>
> Program FileCopy_Classic;
> {$mode objfpc} {$H+}
> var
> NumRead, NumWritten : LongInt;
> puffer : array [1..1048576] of byte;
> InF,OutF : file of byte;
> begin
> assign(InF,ParamStr(1));
> ReSet(InF);
> Assign(OutF,ParamStr(2));
> ReWrite(OutF);
> repeat
> BlockRead(InF,puffer,SizeOf(puffer),NumRead);
> BlockWrite(OutF,puffer,NumRead,NumWritten);
> until (NumRead = 0);
> close(InF);
> close(OutF);
> end.
> _______________________________________________
> fpc-pascal maillist  -  [hidden email]
> http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
>
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: File handling: the TFileStream or the classical way is faster and more efficient?

Géza Kovacs Géza
Hi!

Thx your answer.


Can I use the "AssignFile" procedure from the SysUtils unit instead of
the simple assign procedure from the System unit?
What is the problem with the simple "assign"?

I read from the Free Pascal Wiki:
AssignFile (prevent the use of the older Assign procedure) - Assign a
name to a file

CloseFile (prevent the use of the older Close procedure) - Close opened file

What is the difference between simple "assign", "close", or
"AssignFile", and "CloseFile"?

Best regards, G
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: File handling: the TFileStream or the classical way is faster and more efficient?

Michael Van Canneyt


On Mon, 25 May 2015, Géza Kovacs Géza wrote:

> Hi!
>
> Thx your answer.
>
>
> Can I use the "AssignFile" procedure from the SysUtils unit instead of
> the simple assign procedure from the System unit?
> What is the problem with the simple "assign"?
>
> I read from the Free Pascal Wiki:
> AssignFile (prevent the use of the older Assign procedure) - Assign a
> name to a file
>
> CloseFile (prevent the use of the older Close procedure) - Close opened file
>
> What is the difference between simple "assign", "close", or
> "AssignFile", and "CloseFile"?
There is none, they are 100% the same.

There is no reason not to use the Assign/Close in non-gui code.

The (IMHO misguided) reason for these statements is that TPersistent and TForm (a TPersistent descendent)
have Assign and Close methods. Because of this, confusion can arise if you write Assign or Close in TForm methods.

Michael.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: File handling: the TFileStream or the classical way is faster and more efficient?

Michael Schnell
In reply to this post by Géza Kovacs Géza
On 05/25/2015 06:42 AM, Géza Kovacs Géza wrote:
> Hi All!
>
> What is the faster and more efficient, using the TFileStream or the
> classical way ("file of byte", or "file") type with
> blockread/blockwrite and the other well-know procedures?
> What is the better, faster on large files?
>
As the work that needs to be done by the OS is a lot more than what is
done by the fpc library, I suppose that there is no noticeable difference.

-Michael
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: File handling: the TFileStream or the classical way is faster and more efficient?

Jürgen Hestermann

Am 2015-05-26 um 10:27 schrieb Michael Schnell:
> As the work that needs to be done by the OS is a lot more than what is done by the fpc library, I suppose that there is no noticeable difference.
I think that depends on the size of the file(s).
If you copy thousands of files with only a few bytes each
then it may become very noticable, what "else" is done
(in additon to the pure OS copy of the files).


_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: File handling: the TFileStream or the classical way is faster and more efficient?

Michael Schnell
On 05/26/2015 06:25 PM, Jürgen Hestermann wrote:
>
> If you copy thousands of files with only a few bytes each
> then it may become very noticable, what "else" is done
> (in additon to the pure OS copy of the files).
>
I don't think so. Opening the file is a tedious task for the OS. So I
suppose the User-program/OS relation is not that different regarding the
file size.

-Michael
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: File handling: the TFileStream or the classical way is faster and more efficient?

Michael Van Canneyt


On Wed, 27 May 2015, Michael Schnell wrote:

> On 05/26/2015 06:25 PM, Jürgen Hestermann wrote:
>>
>> If you copy thousands of files with only a few bytes each
>> then it may become very noticable, what "else" is done
>> (in additon to the pure OS copy of the files).
>>
> I don't think so. Opening the file is a tedious task for the OS. So I suppose
> the User-program/OS relation is not that different regarding the file size.

Of course it is.
If you write to a file of 1Gb in blocks of 256 bytes, or you write to a 1Gb file in blocks of 1Mb,
you will surely see the difference. A kernel call always introduces overhead.

Michael.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: File handling: the TFileStream or the classical way is faster and more efficient?

Jürgen Hestermann
In reply to this post by Michael Schnell

Am 2015-05-27 um 09:38 schrieb Michael Schnell:
> On 05/26/2015 06:25 PM, Jürgen Hestermann wrote:
>>
>> If you copy thousands of files with only a few bytes each
>> then it may become very noticable, what "else" is done
>> (in additon to the pure OS copy of the files).
>>
> I don't think so. Opening the file is a tedious task for the OS. So I suppose the User-program/OS relation is not that different regarding the file size.
>
I have written a file manager that logs whole disks.
When I do this twice (so that all data is already in cache on the second run)
I can log (read the meta data like name, size, date, etc.) about 50,000 files
in one second on the second run (doing this on an SSD maybe nearly as fast
on the first run already, I have never tested it).
That means it only takes 20 nanoseconds per file to read and store the meta data
which does not sound much overhead to me.

When it now comes to reading the contents of these files this may take much longer dependent on their size.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal