header translation question, 64 bit problem

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

header translation question, 64 bit problem

Bernd Kreuss
Hi,

I am translating parts of the libpurple headers. Now I seem to have a
problem with this:


typedef enum
{
        PURPLE_PLUGIN_UNKNOWN  = -1,  /**< Unknown type.    */
        PURPLE_PLUGIN_STANDARD = 0,   /**< Standard plugin. */
        PURPLE_PLUGIN_LOADER,         /**< Loader plugin.   */
        PURPLE_PLUGIN_PROTOCOL        /**< Protocol plugin. */

} PurplePluginType;


typedef int PurplePluginPriority;


struct _PurplePluginInfo
{
        unsigned int magic;
        unsigned int major_version;
        unsigned int minor_version;
        PurplePluginType type;               // <-- maybe this is the problem?
        char *ui_requirement;
        unsigned long flags;
        GList *dependencies;
        PurplePluginPriority priority;

        char *id;
        char *name;
        char *version;
        char *summary;
        char *description;
        char *author;
        char *homepage;

        /**
         * If a plugin defines a 'load' function, and it returns FALSE,
         * then the plugin will not be loaded.
         */
        gboolean (*load)(PurplePlugin *plugin);
        gboolean (*unload)(PurplePlugin *plugin);
        void (*destroy)(PurplePlugin *plugin);

        void *ui_info; /**< Used only by UI-specific plugins to build a
preference screen with a custom UI */
        void *extra_info;
        PurplePluginUiInfo *prefs_info; /**< Used by any plugin to display
preferences.  If #ui_info has been specified, this will be ignored. */

        /**
         * This callback has a different use depending on whether this
         * plugin type is PURPLE_PLUGIN_STANDARD or PURPLE_PLUGIN_PROTOCOL.
         *
         * If PURPLE_PLUGIN_STANDARD then the list of actions will show up
         * in the Tools menu, under a submenu with the name of the plugin.
         * context will be NULL.
         *
         * If PURPLE_PLUGIN_PROTOCOL then the list of actions will show up
         * in the Accounts menu, under a submenu with the name of the
         * account.  context will be set to the PurpleConnection for that
         * account.  This callback will only be called for online accounts.
         */
        GList *(*actions)(PurplePlugin *plugin, gpointer context);

        void (*_purple_reserved1)(void);
        void (*_purple_reserved2)(void);
        void (*_purple_reserved3)(void);
        void (*_purple_reserved4)(void);
};



I have translated it as follows:


{$calling cdecl}

type
  TPurplePluginType = (
    PURPLE_PLUGIN_UNKNOWN  := -1,  // Unknown type.
    PURPLE_PLUGIN_STANDARD := 0,   // Standard plugin.
    PURPLE_PLUGIN_LOADER,          // Loader plugin.
    PURPLE_PLUGIN_PROTOCOL         // Protocol plugin.
  );

  TPurplePluginPriority = Integer;

  TPurplePluginInfo = packed record
    magic: Integer;
    major_version: Integer;
    minor_version: Integer;
    plugintype: TPurplePluginType;
    ui_requirement: PChar;
    flags: LongInt;
    dependencies: PGList;
    priority: TPurplePluginPriority;
    id: PChar;
    name: PChar;
    version: PChar;
    summary: PChar;
    description: PChar;
    author: PChar;
    homepage: PChar;
    load: function(plugin: PPurplePlugin): GBoolean;
    unload: function(plugin: PPurplePlugin): GBoolean;
    destroy: procedure(plugin: PPurplePlugin);
    ui_info: Pointer;
    extra_info: Pointer;
    prefs_info: PPurplePluginUiInfo;
    actions: function(plugin: PPurplePlugin; context: Pointer): PGList;

    _purple_reserved1: Pointer;
    _purple_reserved2: Pointer;
    _purple_reserved3: Pointer;
    _purple_reserved4: Pointer;
  end;

This works perfectly well on windows 32 bit and Linux 32 bit but today
someone helped me compile it on linux x86_64 and from the debug output
of libpurple it seems that when I pass it this record initialized with
my values it *can* detect that plugintype is set to
PURPLE_PLUGIN_PROTOCOL because then it will try to read extra_info and
it thinks it is null, probably it is off a few bytes.

I cannot test this myself because I don't have a 64 bit Linux. When an
experienced programmer (should be most on this list) looks at this
header what is the obvious thing that might break it on 64 bit?
Pointers should be ok, but what is with int and Integer and what is
with enums? And are there maybe also certain compiler switches that
influence it?  What might be wrong?

Bernd
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: header translation question, 64 bit problem

ik-6
On Sun, Jun 17, 2012 at 2:02 AM, Bernd <[hidden email]> wrote:
Hi,

I am translating parts of the libpurple headers. Now I seem to have a
problem with this:


typedef enum
{
       PURPLE_PLUGIN_UNKNOWN  = -1,  /**< Unknown type.    */
       PURPLE_PLUGIN_STANDARD = 0,   /**< Standard plugin. */
       PURPLE_PLUGIN_LOADER,         /**< Loader plugin.   */
       PURPLE_PLUGIN_PROTOCOL        /**< Protocol plugin. */

} PurplePluginType;


typedef int PurplePluginPriority;


struct _PurplePluginInfo
{
       unsigned int magic;
       unsigned int major_version;
       unsigned int minor_version;
       PurplePluginType type;               // <-- maybe this is the problem?
       char *ui_requirement;
       unsigned long flags;
       GList *dependencies;
       PurplePluginPriority priority;

       char *id;
       char *name;
       char *version;
       char *summary;
       char *description;
       char *author;
       char *homepage;

       /**
        * If a plugin defines a 'load' function, and it returns FALSE,
        * then the plugin will not be loaded.
        */
       gboolean (*load)(PurplePlugin *plugin);
       gboolean (*unload)(PurplePlugin *plugin);
       void (*destroy)(PurplePlugin *plugin);

       void *ui_info; /**< Used only by UI-specific plugins to build a
preference screen with a custom UI */
       void *extra_info;
       PurplePluginUiInfo *prefs_info; /**< Used by any plugin to display
preferences.  If #ui_info has been specified, this will be ignored. */

       /**
        * This callback has a different use depending on whether this
        * plugin type is PURPLE_PLUGIN_STANDARD or PURPLE_PLUGIN_PROTOCOL.
        *
        * If PURPLE_PLUGIN_STANDARD then the list of actions will show up
        * in the Tools menu, under a submenu with the name of the plugin.
        * context will be NULL.
        *
        * If PURPLE_PLUGIN_PROTOCOL then the list of actions will show up
        * in the Accounts menu, under a submenu with the name of the
        * account.  context will be set to the PurpleConnection for that
        * account.  This callback will only be called for online accounts.
        */
       GList *(*actions)(PurplePlugin *plugin, gpointer context);

       void (*_purple_reserved1)(void);
       void (*_purple_reserved2)(void);
       void (*_purple_reserved3)(void);
       void (*_purple_reserved4)(void);
};



I have translated it as follows:

you have few problems here.
1. Please use ctypes instead, they are set to work properly with 64 bit etc.. and are properly sized to fit C's type sizes.
For example C's int, is not equal to Pascal's Integer, but to longint of Pascal
 


{$calling cdecl}

type
 TPurplePluginType = (
   PURPLE_PLUGIN_UNKNOWN  := -1,  // Unknown type.
   PURPLE_PLUGIN_STANDARD := 0,   // Standard plugin.
   PURPLE_PLUGIN_LOADER,          // Loader plugin.
   PURPLE_PLUGIN_PROTOCOL         // Protocol plugin.
 );

 TPurplePluginPriority = Integer;

 TPurplePluginInfo = packed record
   magic: Integer;
   major_version: Integer;
   minor_version: Integer;
   plugintype: TPurplePluginType;
   ui_requirement: PChar;
   flags: LongInt;
   dependencies: PGList;
   priority: TPurplePluginPriority;
   id: PChar;
   name: PChar;
   version: PChar;
   summary: PChar;
   description: PChar;
   author: PChar;
   homepage: PChar;
   load: function(plugin: PPurplePlugin): GBoolean;
   unload: function(plugin: PPurplePlugin): GBoolean;
   destroy: procedure(plugin: PPurplePlugin);
   ui_info: Pointer;
   extra_info: Pointer;
   prefs_info: PPurplePluginUiInfo;
   actions: function(plugin: PPurplePlugin; context: Pointer): PGList;

   _purple_reserved1: Pointer;
   _purple_reserved2: Pointer;
   _purple_reserved3: Pointer;
   _purple_reserved4: Pointer;

This 4 types are actually callback rather then "simple" pointers. It will work, but personally I prefer it as a callback when I can avoid general "Pointer".
 
 end;

This works perfectly well on windows 32 bit and Linux 32 bit but today
someone helped me compile it on linux x86_64 and from the debug output
of libpurple it seems that when I pass it this record initialized with
my values it *can* detect that plugintype is set to
PURPLE_PLUGIN_PROTOCOL because then it will try to read extra_info and
it thinks it is null, probably it is off a few bytes.

I cannot test this myself because I don't have a 64 bit Linux. When an
experienced programmer (should be most on this list) looks at this
header what is the obvious thing that might break it on 64 bit?
Pointers should be ok, but what is with int and Integer and what is
with enums? And are there maybe also certain compiler switches that
influence it?  What might be wrong?

Bernd
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: header translation question, 64 bit problem

Sven Barth-2
In reply to this post by Bernd Kreuss
On 17.06.2012 01:02, Bernd wrote:

> Hi,
>
> I am translating parts of the libpurple headers. Now I seem to have a
> problem with this:
>
>
> typedef enum
> {
> PURPLE_PLUGIN_UNKNOWN  = -1,  /**< Unknown type.    */
> PURPLE_PLUGIN_STANDARD = 0,   /**< Standard plugin. */
> PURPLE_PLUGIN_LOADER,         /**< Loader plugin.   */
> PURPLE_PLUGIN_PROTOCOL        /**< Protocol plugin. */
>
> } PurplePluginType;
>
>
> typedef int PurplePluginPriority;
>
>
> struct _PurplePluginInfo
> {
> unsigned int magic;
> unsigned int major_version;
> unsigned int minor_version;
> PurplePluginType type;               // <-- maybe this is the problem?
> char *ui_requirement;
> unsigned long flags;

This last line is the problem. On non-Windows 64-Bit GCC systems
sizeof(long) and sizeof(unsigned long) is 8 while it is 4 on 32-Bit
systems and on Windows 64-Bit. See also here for this problematic:
http://en.wikipedia.org/wiki/64-bit#64-bit_data_models

As ik already wrote it is preferred to use the types defined in the
ctypes.pp unit to solve this problem (because they are defined correctly
for the corresponding platform). The types are usually named similar to
their C-type with a "c" equivalent (the prefix is "cu" for unsigned
types). E.g. "unsigned int" will be "cuint" and "long" will be "clong".
The unit also compiles with Delphi.

Regards,
Sven

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: header translation question, 64 bit problem

Florian Klämpfl
In reply to this post by Bernd Kreuss
Am 17.06.2012 01:02, schrieb Bernd:
>
>   TPurplePluginInfo = packed record

You should also avoid using packed. FPC usually uses the same alignment
as C on a certain platform does if the types are correctly used.
However, on platforms (e.g. arm, sparc) which require proper alignment,
packed causes the compiler to load every data element byte by byte
because packed records do not guarantee to be aligned properly. This
slows down code a lot.

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: header translation question, 64 bit problem

Bernd Kreuss
Thank you all for the valuable info. I have now changed everything to
use only the types from ctypes, cint for int and culong for unsigned
long. Once my tester on the other side of the atlantic wakes up I can
test it.

2012/6/17 Florian Klämpfl <[hidden email]>:
> Am 17.06.2012 01:02, schrieb Bernd:
>>
>>   TPurplePluginInfo = packed record
>
> You should also avoid using packed. FPC usually uses the same alignment
> as C on a certain platform does if the types are correctly used.

I did not know this.  I have read in many different places that one
should use packed (thats the reason I did it in the first place), but
I will now remove it from all my records.

Bernd
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: header translation question, 64 bit problem

Jonas Maebe-2
In reply to this post by Florian Klämpfl

On 17 Jun 2012, at 12:08, Florian Klämpfl wrote:

> Am 17.06.2012 01:02, schrieb Bernd:
>>
>>  TPurplePluginInfo = packed record
>
> You should also avoid using packed. FPC usually uses the same alignment
> as C on a certain platform does if the types are correctly used.

And if you use {$packrecords c}, then FPC is guaranteed to use the same alignment as C compilers (aka what the platform ABI specifies).


Jonas_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: header translation question, 64 bit problem

Jonas Maebe-2
In reply to this post by Bernd Kreuss

On 17 Jun 2012, at 12:36, Bernd wrote:

> I did not know this.  I have read in many different places that one
> should use packed

Those places please probably said "you should use packed *if you want the layout of a record to be the same on all platforms and across different FPC versions*". In this particular case, you want everything but it to be the same on all platforms. Instead, you want it to change to adapt to the ABI of the different platforms.

And when using {$packrecords c}, the record layout will not change across FPC versions either (unless a bug is found in the alignment rules for that platform, but in that case the change will improve the compatibility with the C record declaration).


Jonas_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: header translation question, 64 bit problem

Bernd Kreuss
In reply to this post by Jonas Maebe-2
2012/6/17 Jonas Maebe <[hidden email]>:

> And if you use {$packrecords c}, then FPC is guaranteed to use the same alignment as C compilers (aka what the platform ABI specifies).

Done :-)
https://github.com/prof7bit/TorChat/blob/torchat2/src/purple/purple.pas
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: header translation question, 64 bit problem

Bernd Kreuss
2012/6/17 Bernd <[hidden email]>:
> 2012/6/17 Jonas Maebe <[hidden email]>:
>
>> And if you use {$packrecords c}, then FPC is guaranteed to use the same alignment as C compilers (aka what the platform ABI specifies).
>
> Done :-)
> https://github.com/prof7bit/TorChat/blob/torchat2/src/purple/purple.pas

It works! this record with the unsigned long was the only error in
6000 lines of code, everything else immediately started working on 64
bit without changing a single line of code :-)

(Almost) The only other small discontinuity was the PNG writer from
fcl-image needs to be explicitly told Indexed:=False on 64 bit while
on 32 bit this was not necessary.

Bernd
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: header translation question, 64 bit problem

Michalis Kamburelis-3
Bernd wrote:
> (Almost) The only other small discontinuity was the PNG writer from
> fcl-image needs to be explicitly told Indexed:=False on 64 bit while
> on 32 bit this was not necessary.
>

Indexed is false by default in FPC >= 2.6.1, see "BTW" and notes in
http://bugs.freepascal.org/view.php?id=21835

Michalis
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: header translation question, 64 bit problem

Bernd Kreuss
2012/6/17 Michalis Kamburelis <[hidden email]>:

> Indexed is false by default in FPC >= 2.6.1, see "BTW" and notes in
> http://bugs.freepascal.org/view.php?id=21835

Ah, then this mystery is also solved. I instructed the person who
helped me to install the .deb packages from freepascal because Ubuntu
only has 2.4.4 and these debs are 2.6.0 while I am using the 2.6 fixes
branch from svn.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal