buffered filestream

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

buffered filestream

Mattias Gaertner
Hi,

Is there already a buffered file stream in the fpc sources?
Or maybe a generic buffered input stream that can be used with
TFileStream?


Mattias
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: buffered filestream

zaher dirkey

On Sun, Jun 10, 2012 at 4:00 PM, Mattias Gaertner <[hidden email]> wrote:
Hi,

Is there already a buffered file stream in the fpc sources?
Or maybe a generic buffered input stream that can be used with
TFileStream?



Best Regards
Zaher Dirkey


_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: buffered filestream

Mattias Gaertner
On Sun, 10 Jun 2012 17:13:28 +0300
Zaher Dirkey <[hidden email]> wrote:

> On Sun, Jun 10, 2012 at 4:00 PM, Mattias Gaertner <[hidden email]
> > wrote:
>
> > Hi,
> >
> > Is there already a buffered file stream in the fpc sources?
> > Or maybe a generic buffered input stream that can be used with
> > TFileStream?
> >
> >
> >
> check mine,
> http://minilib.svn.sourceforge.net/viewvc/minilib/trunk/lib/mnStreams.pas?view=markup

Thanks, that seems to be a nice sequential text file reader/writer.

That's not what I search. I will try to be more specific:

I want to pass the stream to a binary reader, which skips big
parts of the stream. It can seek backwards and the files can be several
GB, so I need the int64 read/write/seek.

Maybe the word "buffer" is wrong, maybe "cache" is better.

Preferably the buffer size can be set up at creation.

Mattias
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: buffered filestream

Michael Van Canneyt
In reply to this post by Mattias Gaertner


On Sun, 10 Jun 2012, Mattias Gaertner wrote:

> Hi,
>
> Is there already a buffered file stream in the fpc sources?

Yes.

> Or maybe a generic buffered input stream that can be used with
> TFileStream?

See fcl-base/src/bufstream.pp.

It is even documented on

http://www.freepascal.org/docs-html/fcl/bufstream/index.html

Amazing, don't you think  ? ;-)

Michael.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: buffered filestream

Alexander Shishkin
In reply to this post by Mattias Gaertner
10.06.2012 18:40, Mattias Gaertner пишет:

> I want to pass the stream to a binary reader, which skips big
> parts of the stream. It can seek backwards and the files can be several
> GB, so I need the int64 read/write/seek.

Do you mean buffered random access (smth like disk access in DBMS)?

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: buffered filestream

Mattias Gaertner
On Sun, 10 Jun 2012 19:08:40 +0400
Alexander Shishkin <[hidden email]> wrote:

> 10.06.2012 18:40, Mattias Gaertner пишет:
>
> > I want to pass the stream to a binary reader, which skips big
> > parts of the stream. It can seek backwards and the files can be several
> > GB, so I need the int64 read/write/seek.
>
> Do you mean buffered random access

yes

> (smth like disk access in DBMS)?

Well, not as sophisticated as an DBMS. Just a cache, that accelerates
small sequential reads.

Mattias
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: buffered filestream

Mattias Gaertner
In reply to this post by Michael Van Canneyt
On Sun, 10 Jun 2012 16:54:30 +0200 (CEST)
Michael Van Canneyt <[hidden email]> wrote:

>
>
> On Sun, 10 Jun 2012, Mattias Gaertner wrote:
>
> > Hi,
> >
> > Is there already a buffered file stream in the fpc sources?
>
> Yes.

:)
I guessed so.

 
> > Or maybe a generic buffered input stream that can be used with
> > TFileStream?
>
> See fcl-base/src/bufstream.pp.

Ah, abbreviations. BufStream.

 
> It is even documented on
>
> http://www.freepascal.org/docs-html/fcl/bufstream/index.html
>
> Amazing, don't you think  ? ;-)

The fpc sources are always amazing.


Mattias
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: buffered filestream

Mattias Gaertner
On Sun, 10 Jun 2012 18:24:14 +0200
Mattias Gaertner <[hidden email]> wrote:

> On Sun, 10 Jun 2012 16:54:30 +0200 (CEST)
> Michael Van Canneyt <[hidden email]> wrote:
>[...]
> > It is even documented on
> >
> > http://www.freepascal.org/docs-html/fcl/bufstream/index.html
> >
> > Amazing, don't you think  ? ;-)
>
> The fpc sources are always amazing.

Maybe I miss something:

TReadBufStream.Seek calls FakeSeekForward, which calls Discard, which
reads everything up to the position.
This is the right solution for pipes and small files, but for big files
I need a real Seek.

Can TReadBufStream be extended with a property or should I create a new
class?

Mattias
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: buffered filestream

Alexander Shishkin
In reply to this post by Mattias Gaertner
10.06.2012 20:20, Mattias Gaertner пишет:

> On Sun, 10 Jun 2012 19:08:40 +0400
> Alexander Shishkin <[hidden email]> wrote:
>
>> 10.06.2012 18:40, Mattias Gaertner пишет:
>>
>>> I want to pass the stream to a binary reader, which skips big
>>> parts of the stream. It can seek backwards and the files can be several
>>> GB, so I need the int64 read/write/seek.
>>
>> Do you mean buffered random access
>
> yes
>
>> (smth like disk access in DBMS)?
>
> Well, not as sophisticated as an DBMS. Just a cache, that accelerates
> small sequential reads.

Only because yours is readonly. I dont know your real task, but I`m
afraid you will have to implement this cache from scratch.



_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: buffered filestream

Michael Van Canneyt
In reply to this post by Mattias Gaertner


On Sun, 10 Jun 2012, Mattias Gaertner wrote:

>> It is even documented on
>>
>> http://www.freepascal.org/docs-html/fcl/bufstream/index.html
>>
>> Amazing, don't you think  ? ;-)
>
> The fpc sources are always amazing.

As in: Bugs creep up at the most unexpected places...  :)

Michael.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: buffered filestream

Michael Van Canneyt
In reply to this post by Mattias Gaertner


On Sun, 10 Jun 2012, Mattias Gaertner wrote:

> On Sun, 10 Jun 2012 18:24:14 +0200
> Mattias Gaertner <[hidden email]> wrote:
>
>> On Sun, 10 Jun 2012 16:54:30 +0200 (CEST)
>> Michael Van Canneyt <[hidden email]> wrote:
>> [...]
>>> It is even documented on
>>>
>>> http://www.freepascal.org/docs-html/fcl/bufstream/index.html
>>>
>>> Amazing, don't you think  ? ;-)
>>
>> The fpc sources are always amazing.
>
> Maybe I miss something:
>
> TReadBufStream.Seek calls FakeSeekForward, which calls Discard, which
> reads everything up to the position.
> This is the right solution for pipes and small files, but for big files
> I need a real Seek.
>
> Can TReadBufStream be extended with a property or should I create a new
> class?

Feel free to extend the existing classes. The classes are by no means set in stone...

They did the job for me, but use cases vary...

Michael.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: buffered filestream

Martin Schreiber-2
In reply to this post by Mattias Gaertner
On Sunday 10 June 2012 15:00:20 Mattias Gaertner wrote:
> Hi,
>
> Is there already a buffered file stream in the fpc sources?
> Or maybe a generic buffered input stream that can be used with
> TFileStream?
>
MSEgui has a bufstream too: tcustombufstream in
lib/common/kernel/msestream.pas, I don't know if it fits your needs.

Martin
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: buffered filestream

Mattias Gaertner
On Sun, 10 Jun 2012 19:02:52 +0200
Martin Schreiber <[hidden email]> wrote:

> On Sunday 10 June 2012 15:00:20 Mattias Gaertner wrote:
> > Hi,
> >
> > Is there already a buffered file stream in the fpc sources?
> > Or maybe a generic buffered input stream that can be used with
> > TFileStream?
> >
> MSEgui has a bufstream too: tcustombufstream in
> lib/common/kernel/msestream.pas, I don't know if it fits your needs.

Looks promising. :)

Mattias
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal