Implementation of variant records

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Implementation of variant records

Bernd Oppolzer
This question is not directly related to FPC, but it is instead a Pascal
question in general.

Take this record definition:

type S1 = record
             X : CHAR ( 27 ) ;
             case INTEGER of
               1 :
                 ( V : INTEGER ) ;
               2 :
                 ( W : REAL ) ;
               3 :
                 ( Z : CHAR ( 8 ) ) ;
           end ;

CHAR (n) is an abbreviation for /* packed */ array [1..n] of char
which is accepted by a certain compiler.

Now the question:

X needs 27 bytes, so the variant part of the record starts at offset 27.
But on a certain platform V has alignment 4 (integer), W has alignment 8
(real).
The compiler decided to put V on offset 28, W on offset 32 and Z (no
alignment)
on offset 27, so the three variants don't overlay properly.

IMO, the variants in a variant record should always overlay correctly
(like unions in C),
so the variant part should start at offset 32 in this case, and this is
where all three
variants should start.

In other words: every record (in classical Pascal) consists of a fixed
part and a variant
part and a fixed border between the two (that's how it should be, IMO).
And this border
takes its alignment properties from the highest alignment of the fields
in the variant part.

What do you think?

Why is this important to me? Because I am working on a WITH extension
for record definitions,
which allows the integration of the components from other record and
pointer types into the
current record type namespace, and this has to be fixed, before I can do
this.

BTW: is there a forum to discuss general Pascal questions not directly
related to the FPC product?
FPC-Pascal will sure be the largest auditorium available for such
questions ...

Thank you, kind regards

Bernd

_______________________________________________
fpc-pascal maillist  -  [hidden email]
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Implementation of variant records

Tomas Hajny-2
On 2019-07-05 11:49, Bernd Oppolzer wrote:
> This question is not directly related to FPC, but it is instead a
> Pascal question in general.
  .
  .
> BTW: is there a forum to discuss general Pascal questions not directly
> related to the FPC product?
> FPC-Pascal will sure be the largest auditorium available for such
> questions ...

Obviously, fpc-other is the right list unless your question is directly
applicable to FPC.

Tomas
_______________________________________________
fpc-pascal maillist  -  [hidden email]
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Implementation of variant records

Marco van de Voort-2
In reply to this post by Bernd Oppolzer

Op 2019-07-05 om 11:49 schreef Bernd Oppolzer:
>
> IMO, the variants in a variant record should always overlay correctly
> (like unions in C),
> so the variant part should start at offset 32 in this case, and this
> is where all three
> variants should start.

This is not a guarantee case in the Pascal language, afaik many old
compilers don't even try, but make fields sequential.

As soon as you have expections of overlaying, you are dialect and
architecture specific.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Implementation of variant records

Ralf Quint
On 7/5/2019 3:27 AM, Marco van de Voort wrote:

>
> Op 2019-07-05 om 11:49 schreef Bernd Oppolzer:
>>
>> IMO, the variants in a variant record should always overlay correctly
>> (like unions in C),
>> so the variant part should start at offset 32 in this case, and this
>> is where all three
>> variants should start.
>
> This is not a guarantee case in the Pascal language, afaik many old
> compilers don't even try, but make fields sequential.
>
> As soon as you have expections of overlaying, you are dialect and
> architecture specific.

Shouldn't a PACKED Record guarantee that values are aligned at the byte
level?

Ralf


---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

_______________________________________________
fpc-pascal maillist  -  [hidden email]
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Implementation of variant records

Marco van de Voort-2
Op 2019-07-05 om 13:53 schreef Ralf Quint:

>
>>> IMO, the variants in a variant record should always overlay
>>> correctly (like unions in C),
>>> so the variant part should start at offset 32 in this case, and this
>>> is where all three
>>> variants should start.
>>
>> This is not a guarantee case in the Pascal language, afaik many old
>> compilers don't even try, but make fields sequential.
>>
>> As soon as you have expections of overlaying, you are dialect and
>> architecture specific.
>
> Shouldn't a PACKED Record guarantee that values are aligned at the
> byte level?
>
In Borland and x86 pascals in general that is somewhat the norm.

But the original Pascal architecture was not byte, but word addressable,
and "packed' there meant packing several fields into one word.  Access
to packed fields meant loading the word, and shifting the values out.

 From https://en.wikipedia.org/wiki/CDC_6600

---

The central processor had 60-bit <https://en.wikipedia.org/wiki/60-bit>
words, while the peripheral processors had 12-bit
<https://en.wikipedia.org/wiki/12-bit> words. CDC used the term "byte"
to refer to 12-bit entities used by peripheral processors; characters
were 6-bit, and central processor instructions were either 15 bits, or
30 bits with a signed 18-bit address field, the latter allowing for a
directly addressable memory space of 128K words of central memory
(converted to modern terms, with 8-bit bytes, this is 0.94 MB).

----

Strings that were packed had 10 6-bit chars in a word etc. But for some
array operations, operating on unpacked data (IOW one element per word)
was easier. There were standard procedures (builtins?) PACK and UNPACK
to convert between packed and unpacked arrays/records.

Packing was explicit and was meant as a generalized solution to counter
the memory waste of such schemes (since one 6-bit char/60bits word is
quite wastful), not to govern the memory layout precisely, that was
architecture/implementation dependent. The architectures varied to
wildly for that.

Other, newer, processors were less funky, but sometimes still had
limitations. I can still remember the initial powerpc port having a
problem on some (603) processors that couldn't load floating point
values from unaligned addresses. (lfd/stfd or so)

And of course Alpha (older multias only?) that had exceptions on
unaligned access iirc.


_______________________________________________
fpc-pascal maillist  -  [hidden email]
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Implementation of variant records

Ralf Quint
On 7/5/2019 5:07 AM, Marco van de Voort wrote:

> Op 2019-07-05 om 13:53 schreef Ralf Quint:
>>
>>>> IMO, the variants in a variant record should always overlay
>>>> correctly (like unions in C),
>>>> so the variant part should start at offset 32 in this case, and
>>>> this is where all three
>>>> variants should start.
>>>
>>> This is not a guarantee case in the Pascal language, afaik many old
>>> compilers don't even try, but make fields sequential.
>>>
>>> As soon as you have expections of overlaying, you are dialect and
>>> architecture specific.
>>
>> Shouldn't a PACKED Record guarantee that values are aligned at the
>> byte level?
>>
> In Borland and x86 pascals in general that is somewhat the norm.
>
> But the original Pascal architecture was not byte, but word
> addressable, and "packed' there meant packing several fields into one
> word.  Access to packed fields meant loading the word, and shifting
> the values out.
>
> From https://en.wikipedia.org/wiki/CDC_6600
>
> ---
>
> The central processor had 60-bit
> <https://en.wikipedia.org/wiki/60-bit> words, while the peripheral
> processors had 12-bit <https://en.wikipedia.org/wiki/12-bit> words.
> CDC used the term "byte" to refer to 12-bit entities used by
> peripheral processors; characters were 6-bit, and central processor
> instructions were either 15 bits, or 30 bits with a signed 18-bit
> address field, the latter allowing for a directly addressable memory
> space of 128K words of central memory (converted to modern terms, with
> 8-bit bytes, this is 0.94 MB).
>
> ----
>
> Strings that were packed had 10 6-bit chars in a word etc. But for
> some array operations, operating on unpacked data (IOW one element per
> word) was easier. There were standard procedures (builtins?) PACK and
> UNPACK to convert between packed and unpacked arrays/records.
>
> Packing was explicit and was meant as a generalized solution to
> counter the memory waste of such schemes (since one 6-bit char/60bits
> word is quite wastful), not to govern the memory layout precisely,
> that was architecture/implementation dependent. The architectures
> varied to wildly for that.
>
> Other, newer, processors were less funky, but sometimes still had
> limitations. I can still remember the initial powerpc port having a
> problem on some (603) processors that couldn't load floating point
> values from unaligned addresses. (lfd/stfd or so)
>
> And of course Alpha (older multias only?) that had exceptions on
> unaligned access iirc.

That's rather odd. I have done a lot of data conversion between
different architectures, always using Pascal, and can't recall where an
explicit "packed" record element was not aligned at a byte level, and
this was IIRC introduced with UCSD Pascal, which was based  on a 16bit
p-code machine, regardless of the underlying CPU, so your CDC reference
doesn't directly apply.

And the concept of a "packed record" (as coming from UCSD Pascal) should
be different from the explicit PACK/UNPACK from ISO7165, which can also
be applied arrays and sets, which I think is what you are likely
referring to...

The only time I had to deal with PACK/UNPACK was on some HP (fka Compaq
fka DEC) Pascal for (Open)VMS, and a packed record would work on a byte
level, unless sets smaller than a byte where used, with the whole record
then padded in memory (but not on storage) to the necessary
alignment/register width.

Can't find my old Metrowerks CodeWarrior manuals right now, that is the
only time I have worked with a (non-UCSD) Pascal on a PPC (and 68k)
processor to check what how that was defined in there...

Ralf


---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

_______________________________________________
fpc-pascal maillist  -  [hidden email]
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Implementation of variant records

Florian Klämpfl
In reply to this post by Ralf Quint
Am 05.07.2019 um 13:53 schrieb Ralf Quint:

> On 7/5/2019 3:27 AM, Marco van de Voort wrote:
>>
>> Op 2019-07-05 om 11:49 schreef Bernd Oppolzer:
>>>
>>> IMO, the variants in a variant record should always overlay correctly
>>> (like unions in C),
>>> so the variant part should start at offset 32 in this case, and this
>>> is where all three
>>> variants should start.
>>
>> This is not a guarantee case in the Pascal language, afaik many old
>> compilers don't even try, but make fields sequential.
>>
>> As soon as you have expections of overlaying, you are dialect and
>> architecture specific.
>
> Shouldn't a PACKED Record guarantee that values are aligned at the byte
> level?

It does in TP, but the ISO says only that it shall be economised.

_______________________________________________
fpc-pascal maillist  -  [hidden email]
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Implementation of variant records

Ralf Quint
On 7/6/2019 12:21 PM, Florian Klaempfl wrote:
> Am 05.07.2019 um 13:53 schrieb Ralf Quint:
>> Shouldn't a PACKED Record guarantee that values are aligned at the byte
>> level?
> It does in TP, but the ISO says only that it shall be economised.
>
Well, the ISO is probably what has done most of the damage to Pascal.
AFAIK, the ISO doesn't not actually specify any "packed record" per se,
but there is an explicit PACK and UNPACK procedures to more economically
store sets and arrays (and possibly records, I would have to look up the
ISO text to be certain). For example this would be something to create
bit fields, pretty much the only feature of C that I sometimes miss in
(Free)Pascal.

For me, the use of packed records is a feature that facilitates the easy
exchange of stored data between architectures, something that a lot of
people these days with their mania about 64bit this and 64bit that are
often forgetting or chose to ignore...

Ralf


---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

_______________________________________________
fpc-pascal maillist  -  [hidden email]
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Implementation of variant records

Free Pascal - General mailing list
Am 07.07.2019 um 02:55 schrieb Ralf Quint:

> On 7/6/2019 12:21 PM, Florian Klaempfl wrote:
>> Am 05.07.2019 um 13:53 schrieb Ralf Quint:
>>> Shouldn't a PACKED Record guarantee that values are aligned at the byte
>>> level?
>> It does in TP, but the ISO says only that it shall be economised.
>>
> Well, the ISO is probably what has done most of the damage to Pascal.
> AFAIK, the ISO doesn't not actually specify any "packed record" per
> se, but there is an explicit PACK and UNPACK procedures to more
> economically store sets and arrays (and possibly records, I would have
> to look up the ISO text to be certain). For example this would be
> something to create bit fields, pretty much the only feature of C that
> I sometimes miss in (Free)Pascal.
You mean like this?

=== code begin ===

struct Test
{
         uint8_t field1;
         uint16_t bfield1 : 1;
         uint16_t bfield2 : 1;
         uint16_t bfield3 : 3;
         uint16_t bfield4 : 5;
         uint16_t bfield5 : 1;
         uint16_t field2;
};

=== code end ===

=== code begin ===

type
   TTest = bitpacked record
     field1: UInt8;
     bfield1: 0..1;
     bfield2: 0..1;
     bfield3: 0..7;
     bfield4: 0..31;
     bfield5: 0..1;
     pad: UInt8; // needed due to bitpacked
     field2: UInt16;
   end;

=== code end ===

The size of both records is 6. :)

Regards,
Sven
_______________________________________________
fpc-pascal maillist  -  [hidden email]
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Implementation of variant records

Free Pascal - General mailing list
Am 07.07.2019 um 12:54 schrieb Sven Barth:

> Am 07.07.2019 um 02:55 schrieb Ralf Quint:
>> On 7/6/2019 12:21 PM, Florian Klaempfl wrote:
>>> Am 05.07.2019 um 13:53 schrieb Ralf Quint:
>>>> Shouldn't a PACKED Record guarantee that values are aligned at the
>>>> byte
>>>> level?
>>> It does in TP, but the ISO says only that it shall be economised.
>>>
>> Well, the ISO is probably what has done most of the damage to Pascal.
>> AFAIK, the ISO doesn't not actually specify any "packed record" per
>> se, but there is an explicit PACK and UNPACK procedures to more
>> economically store sets and arrays (and possibly records, I would
>> have to look up the ISO text to be certain). For example this would
>> be something to create bit fields, pretty much the only feature of C
>> that I sometimes miss in (Free)Pascal.
> You mean like this?
>
> === code begin ===
>
> struct Test
> {
>         uint8_t field1;
>         uint16_t bfield1 : 1;
>         uint16_t bfield2 : 1;
>         uint16_t bfield3 : 3;
>         uint16_t bfield4 : 5;
>         uint16_t bfield5 : 1;
>         uint16_t field2;
> };
>
> === code end ===
>
> === code begin ===
>
> type
>   TTest = bitpacked record
>     field1: UInt8;
>     bfield1: 0..1;
>     bfield2: 0..1;
>     bfield3: 0..7;
>     bfield4: 0..31;
>     bfield5: 0..1;
>     pad: UInt8; // needed due to bitpacked
>     field2: UInt16;
>   end;
>
> === code end ===

Small correction: there needs to be another "bpad: 0..31;" field between
bfield5 and pad, cause otherwise the pad field will be directly behind
the bfield5 one. This could of course be avoided by putting the bit
fields into a subrecord, though then of course there'd be the need for
an additional field name.

Regards,
Sven
_______________________________________________
fpc-pascal maillist  -  [hidden email]
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal