Writing floating points to steams

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

Writing floating points to steams

Torsten Bonde Christiansen
Hi all.

What is the best/preferred way to write floatingpoint to a steam?

Since TStream doesn't have any native WriteFloat/Double and the likes, some conversion is needed - but what is a good strategy for this?

A simple conversion like Stream.WriteDWord( DWord(<some float variable>)) is not possible due to illegal type conversion.

Any help appriciated.

Regards,
Torsten.

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Writing floating points to steams

Graeme Geldenhuys-6
On 2016-01-12 10:45, Torsten Bonde Christiansen wrote:
> Since TStream doesn't have any native WriteFloat/Double and the likes,
> some conversion is needed - but what is a good strategy for this?

I use FloatStr() to do the conversion, then Write() to actually write it
to a stream.


Regards,
  - Graeme -

--
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://fpgui.sourceforge.net/

My public PGP key:  http://tinyurl.com/graeme-pgp
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Writing floating points to steams

Torsten Bonde Christiansen
On 2016-01-12 12:04, Graeme Geldenhuys wrote:
On 2016-01-12 10:45, Torsten Bonde Christiansen wrote:
Since TStream doesn't have any native WriteFloat/Double and the likes, 
some conversion is needed - but what is a good strategy for this?
I use FloatStr() to do the conversion, then Write() to actually write it
to a stream.


Regards,
  - Graeme -

In my case I am trying to write a datetime value, and in order not to worry about the format, separators, etc. i would much rather just write the actual floating point value...

Regards,
Torsten.

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Writing floating points to steams

Michael Van Canneyt


On Tue, 12 Jan 2016, Torsten Bonde Christiansen wrote:

> On 2016-01-12 12:04, Graeme Geldenhuys wrote:
>> On 2016-01-12 10:45, Torsten Bonde Christiansen wrote:
>>> Since TStream doesn't have any native WriteFloat/Double and the likes,
>>> some conversion is needed - but what is a good strategy for this?
>> I use FloatStr() to do the conversion, then Write() to actually write it
>> to a stream.
>>
>>
>> Regards,
>>    - Graeme -
>>
> In my case I am trying to write a datetime value, and in order not to worry
> about the format, separators, etc. i would much rather just write the actual
> floating point value...

Format the date/time as RFC3339 or ISO 8601 (they differe in details only)
and you should not have a problem. ScanDateTime from strutils will happily
perform the opposite operation.

Michael.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Writing floating points to steams

Torsten Bonde Christiansen
On 2016-01-12 12:38, Michael Van Canneyt wrote:

>
>
> On Tue, 12 Jan 2016, Torsten Bonde Christiansen wrote:
>
>> On 2016-01-12 12:04, Graeme Geldenhuys wrote:
>>> On 2016-01-12 10:45, Torsten Bonde Christiansen wrote:
>>>> Since TStream doesn't have any native WriteFloat/Double and the likes,
>>>> some conversion is needed - but what is a good strategy for this?
>>> I use FloatStr() to do the conversion, then Write() to actually
>>> write it
>>> to a stream.
>>>
>>>
>>> Regards,
>>>    - Graeme -
>>>
>> In my case I am trying to write a datetime value, and in order not to
>> worry about the format, separators, etc. i would much rather just
>> write the actual floating point value...
>
> Format the date/time as RFC3339 or ISO 8601 (they differe in details
> only) and you should not have a problem. ScanDateTime from strutils
> will happily
> perform the opposite operation.
I will give it a try.

But out of curiosity is there a reason why there are not floating point
Read/Write methods for TStream (and decendants)?

Regards,
Torsten.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Writing floating points to steams

Michael Schnell
On 01/12/2016 01:40 PM, Torsten Bonde Christiansen wrote:
>
> But out of curiosity is there a reason why there are not floating
> point Read/Write methods for TStream (and decendants)?
>

As

   procedure TStream.WriteQWord(q: QWord);
     begin
       WriteBuffer(q,8);
     end;

it's rather easy to do your own TStream.WriteDateTime procedure and friends.

-Michael

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Writing floating points to steams

Serguei TARASSOV
In reply to this post by Torsten Bonde Christiansen
On 13/01/2016 08:47, [hidden email] wrote:
> On 2016-01-12 10:45, Torsten Bonde Christiansen wrote:
>> Since TStream doesn't have any native WriteFloat/Double and the likes,
>> some conversion is needed - but what is a good strategy for this?
Depends on size constraints.
In the simple case you should convert float to locale-independent string
value (no spaces, no commas separators) then write it.
If the size does matter, you may convert float to corresponding byte
array then write it.

Regards,
Serguei
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
--
Regards,
Serguei
Reply | Threaded
Open this post in threaded view
|

Re: Writing floating points to steams

Mark Morgan Lloyd-5
Serguei TARASSOV wrote:
> On 13/01/2016 08:47, [hidden email] wrote:
>> On 2016-01-12 10:45, Torsten Bonde Christiansen wrote:
>>> Since TStream doesn't have any native WriteFloat/Double and the likes,
>>> some conversion is needed - but what is a good strategy for this?
> Depends on size constraints.
> In the simple case you should convert float to locale-independent string
> value (no spaces, no commas separators) then write it.
> If the size does matter, you may convert float to corresponding byte
> array then write it.

Can I ask a naive question here please: does a binary stream store
endianness anywhere? In other words, is there any indication that
something running on ARM is about to get into trouble by reading
something written by (big-endian) MIPS?

Should writing binary floating point to a stream note that it's IEEE
format, just in case anybody ever tries to process it on a platform that
supports alternatives?

--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Writing floating points to steams

Torsten Bonde Christiansen
On 2016-01-13 10:06, Mark Morgan Lloyd wrote:

> Serguei TARASSOV wrote:
>> On 13/01/2016 08:47, [hidden email] wrote:
>>> On 2016-01-12 10:45, Torsten Bonde Christiansen wrote:
>>>> Since TStream doesn't have any native WriteFloat/Double and the likes,
>>>> some conversion is needed - but what is a good strategy for this?
>> Depends on size constraints.
>> In the simple case you should convert float to locale-independent
>> string value (no spaces, no commas separators) then write it.
>> If the size does matter, you may convert float to corresponding byte
>> array then write it.
>
> Can I ask a naive question here please: does a binary stream store
> endianness anywhere? In other words, is there any indication that
> something running on ARM is about to get into trouble by reading
> something written by (big-endian) MIPS?
Not that I'm aware of - i would think of it as a stream of bytes.
Endianess is defined by the CPU not by the file.
>
> Should writing binary floating point to a stream note that it's IEEE
> format, just in case anybody ever tries to process it on a platform
> that supports alternatives?
>
It would be great if storing floating point could be in IEEE, to have a
standard as reference.

Regards,
Torsten.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Writing floating points to steams

Michael Schnell
In reply to this post by Mark Morgan Lloyd-5
On 01/13/2016 10:06 AM, Mark Morgan Lloyd wrote:

Can I ask a naive question here please: does a binary stream store endianness anywhere?
AFAIK, generally, such binary (file-, Network transfer, ....) formats are not supposed to be "natively" portable. To be portable they should be either propriety (defined by the single application that uses them) or Text-based (such as XML or "INITFILE").

-Michael

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Writing floating points to steams

Mark Morgan Lloyd-5
In reply to this post by Torsten Bonde Christiansen
Torsten Bonde Christiansen wrote:

> On 2016-01-13 10:06, Mark Morgan Lloyd wrote:
>> Serguei TARASSOV wrote:
>>> On 13/01/2016 08:47, [hidden email] wrote:
>>>> On 2016-01-12 10:45, Torsten Bonde Christiansen wrote:
>>>>> Since TStream doesn't have any native WriteFloat/Double and the likes,
>>>>> some conversion is needed - but what is a good strategy for this?
>>> Depends on size constraints.
>>> In the simple case you should convert float to locale-independent
>>> string value (no spaces, no commas separators) then write it.
>>> If the size does matter, you may convert float to corresponding byte
>>> array then write it.
>>
>> Can I ask a naive question here please: does a binary stream store
>> endianness anywhere? In other words, is there any indication that
>> something running on ARM is about to get into trouble by reading
>> something written by (big-endian) MIPS?

> Not that I'm aware of - i would think of it as a stream of bytes.
> Endianess is defined by the CPU not by the file.

Thanks, I assumed that was the case but thought it worth checking.

I suppose that one could stream variants, i.e. the enumeration saying
what one was followed by a value.

>> Should writing binary floating point to a stream note that it's IEEE
>> format, just in case anybody ever tries to process it on a platform
>> that supports alternatives?
>>
> It would be great if storing floating point could be in IEEE, to have a
> standard as reference.

I'm not at all sure about this, but I think I've seen something that
suggested that byte ordering in external representations was covered by
the standard.

--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Writing floating points to steams

Marco van de Voort
In reply to this post by Torsten Bonde Christiansen
In our previous episode, Torsten Bonde Christiansen said:
> What is the best/preferred way to write floatingpoint to a steam?


var d : double; // or single, avoid extended

d:=<some expression>;
stream.write(d,sizeof(d));

stream.read(d,sizeof(d));

All the stream helpers are just that.

One could add writedouble/single to the stream typehelpers in streamex.

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Writing floating points to steams

Torsten Bonde Christiansen
In reply to this post by Mark Morgan Lloyd-5
On 2016-01-13 10:40, Mark Morgan Lloyd wrote:

> Torsten Bonde Christiansen wrote:
>> On 2016-01-13 10:06, Mark Morgan Lloyd wrote:
>>> Serguei TARASSOV wrote:
>>>> On 13/01/2016 08:47, [hidden email] wrote:
>>>>> On 2016-01-12 10:45, Torsten Bonde Christiansen wrote:
>>>>>> Since TStream doesn't have any native WriteFloat/Double and the
>>>>>> likes,
>>>>>> some conversion is needed - but what is a good strategy for this?
>>>> Depends on size constraints.
>>>> In the simple case you should convert float to locale-independent
>>>> string value (no spaces, no commas separators) then write it.
>>>> If the size does matter, you may convert float to corresponding
>>>> byte array then write it.
>>>
>>> Can I ask a naive question here please: does a binary stream store
>>> endianness anywhere? In other words, is there any indication that
>>> something running on ARM is about to get into trouble by reading
>>> something written by (big-endian) MIPS?
>
>> Not that I'm aware of - i would think of it as a stream of bytes.
>> Endianess is defined by the CPU not by the file.
>
> Thanks, I assumed that was the case but thought it worth checking.
>
> I suppose that one could stream variants, i.e. the enumeration saying
> what one was followed by a value.
>
>>> Should writing binary floating point to a stream note that it's IEEE
>>> format, just in case anybody ever tries to process it on a platform
>>> that supports alternatives?
>>>
>> It would be great if storing floating point could be in IEEE, to have
>> a standard as reference.
>
> I'm not at all sure about this, but I think I've seen something that
> suggested that byte ordering in external representations was covered
> by the standard.

I'm not even sure how (or even if) TStream decendants handles endianness
for integers.... I'm guessing they are not.

Regards,
Torsten.

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Writing floating points to steams

Sven Barth-2
In reply to this post by Mark Morgan Lloyd-5

Am 13.01.2016 10:40 schrieb "Mark Morgan Lloyd" <[hidden email]>:
>
> Torsten Bonde Christiansen wrote:
>>
>> On 2016-01-13 10:06, Mark Morgan Lloyd wrote:
>>>
>>> Serguei TARASSOV wrote:
>>>>
>>>> On 13/01/2016 08:47, [hidden email] wrote:
>>>>>
>>>>> On 2016-01-12 10:45, Torsten Bonde Christiansen wrote:
>>>>>>
>>>>>> Since TStream doesn't have any native WriteFloat/Double and the likes,
>>>>>> some conversion is needed - but what is a good strategy for this?
>>>>
>>>> Depends on size constraints.
>>>> In the simple case you should convert float to locale-independent string value (no spaces, no commas separators) then write it.
>>>> If the size does matter, you may convert float to corresponding byte array then write it.
>>>
>>>
>>> Can I ask a naive question here please: does a binary stream store endianness anywhere? In other words, is there any indication that something running on ARM is about to get into trouble by reading something written by (big-endian) MIPS?
>
>
>> Not that I'm aware of - i would think of it as a stream of bytes. Endianess is defined by the CPU not by the file.
>
>
> Thanks, I assumed that was the case but thought it worth checking.
>
> I suppose that one could stream variants, i.e. the enumeration saying what one was followed by a value.

That's not the task of TStream however. For that you should use a custom class that uses TStream under the hood.

>
>>> Should writing binary floating point to a stream note that it's IEEE format, just in case anybody ever tries to process it on a platform that supports alternatives?
>>>
>> It would be great if storing floating point could be in IEEE, to have a standard as reference.
>
>
> I'm not at all sure about this, but I think I've seen something that suggested that byte ordering in external representations was covered by the standard.

I've checked recently due to big endian problems in the Castle game engine and found out that endianess isn't really covered for floating points...

Regards,
Sven


_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Writing floating points to steams

Sven Barth-2
In reply to this post by Torsten Bonde Christiansen

Am 13.01.2016 13:16 schrieb "Torsten Bonde Christiansen" <[hidden email]>:
>
> On 2016-01-13 10:40, Mark Morgan Lloyd wrote:
>>
>> Torsten Bonde Christiansen wrote:
>>>
>>> On 2016-01-13 10:06, Mark Morgan Lloyd wrote:
>>>>
>>>> Serguei TARASSOV wrote:
>>>>>
>>>>> On 13/01/2016 08:47, [hidden email] wrote:
>>>>>>
>>>>>> On 2016-01-12 10:45, Torsten Bonde Christiansen wrote:
>>>>>>>
>>>>>>> Since TStream doesn't have any native WriteFloat/Double and the likes,
>>>>>>> some conversion is needed - but what is a good strategy for this?
>>>>>
>>>>> Depends on size constraints.
>>>>> In the simple case you should convert float to locale-independent string value (no spaces, no commas separators) then write it.
>>>>> If the size does matter, you may convert float to corresponding byte array then write it.
>>>>
>>>>
>>>> Can I ask a naive question here please: does a binary stream store endianness anywhere? In other words, is there any indication that something running on ARM is about to get into trouble by reading something written by (big-endian) MIPS?
>>
>>
>>> Not that I'm aware of - i would think of it as a stream of bytes. Endianess is defined by the CPU not by the file.
>>
>>
>> Thanks, I assumed that was the case but thought it worth checking.
>>
>> I suppose that one could stream variants, i.e. the enumeration saying what one was followed by a value.
>>
>>>> Should writing binary floating point to a stream note that it's IEEE format, just in case anybody ever tries to process it on a platform that supports alternatives?
>>>>
>>> It would be great if storing floating point could be in IEEE, to have a standard as reference.
>>
>>
>> I'm not at all sure about this, but I think I've seen something that suggested that byte ordering in external representations was covered by the standard.
>
>
> I'm not even sure how (or even if) TStream decendants handles endianness for integers.... I'm guessing they are not.

TStream always read in its native order. If you know that the data is of the opposite endianess you need to convert this yourself as TStream can't know that.

Regards,
Sven


_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Writing floating points to steams

Marco van de Voort
In reply to this post by Torsten Bonde Christiansen
In our previous episode, Torsten Bonde Christiansen said:
> I'm not even sure how (or even if) TStream decendants handles endianness
> for integers.... I'm guessing they are not.

TStream is basically an abstraction for stream of bytes. The read<type>
methods are just convenience, not some protocol.

As far as endianess go, see the streamex unit's class helper for TStream,
and its use (here and there) in package chm.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Writing floating points to steams

Dennis
In reply to this post by Sven Barth-2


Sven Barth wrote:

> >>> Should writing binary floating point to a stream note that it's
> IEEE format, just in case anybody ever tries to process it on a
> platform that supports alternatives?
>
> >>>
> >> It would be great if storing floating point could be in IEEE, to
> have a standard as reference.
> >
> >
> > I'm not at all sure about this, but I think I've seen something that
> suggested that byte ordering in external representations was covered
> by the standard.
>
> I've checked recently due to big endian problems in the Castle game
> engine and found out that endianess isn't really covered for floating
> points...
>
> Regards,
> Sven
>
>
Do you imply that, unlike floating point, endianess for integers are
handled automatically by TStream?

Dennis
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Writing floating points to steams

Sven Barth-2

Am 16.01.2016 10:30 schrieb "Dennis" <[hidden email]>:
>
>
>
> Sven Barth wrote:
>>
>> >>> Should writing binary floating point to a stream note that it's IEEE format, just in case anybody ever tries to process it on a platform that supports alternatives?
>>
>> >>>
>> >> It would be great if storing floating point could be in IEEE, to have a standard as reference.
>> >
>> >
>> > I'm not at all sure about this, but I think I've seen something that suggested that byte ordering in external representations was covered by the standard.
>>
>> I've checked recently due to big endian problems in the Castle game engine and found out that endianess isn't really covered for floating points...
>>
>> Regards,
>> Sven
>>
>>
> Do you imply that, unlike floating point, endianess for integers are handled automatically by TStream?

Not at all, but it could have been that IEEE floating points are defined to be always little endian for example.

Regards,
Sven


_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal