Static linking to C library on Linux

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Static linking to C library on Linux

Free Pascal - General mailing list
I'm trying to link to a static library (built from Python sources) in Pascal and having troubles on Linux. On Mac where I have experience I successfully link and every works as expected but on Linux I get slews of linker errors in what appear to be standard C library functions.

For example:

/usr/bin/ld.bfd: /home/chris/pas/PythonBridge-master/sources/libpython3.7m.a(bytearrayobject.o): in function `memcpy':
/usr/include/x86_64-linux-gnu/bits/string_fortified.h:34: undefined reference to `memcpy'
/usr/bin/ld.bfd: /home/chris/pas/PythonBridge-master/sources/libpython3.7m.a(typeobject.o): in function `_PyType_Name':
/home/chris/Downloads/Python-3.7.4/Objects/typeobject.c:417: undefined reference to `strrchr'
/usr/bin/ld.bfd: /home/chris/Downloads/Python-3.7.4/Objects/typeobject.c:107: undefined reference to `strlen'
/usr/bin/ld.bfd: /home/chris/Downloads/Python-3.7.4/Objects/typeobject.c:108: undefined reference to `strncmp'
/usr/bin/ld.bfd: /home/chris/Downloads/Python-3.7.4/Objects/typeobject.c:103: undefined reference to `strrchr'
/usr/bin/ld.bfd: /home/chris/Downloads/Python-3.7.4/Objects/typeobject.c:107: undefined reference to `strlen'
/usr/bin/ld.bfd: /home/chris/Downloads/Python-3.7.4/Objects/typeobject.c:108: undefined reference to `strncmp'

Do I need to link to other system libraries when building on Linux? I'm also confused because I thought the purpose of the static libraries was to contain all the code they needed to run and thus these system functions like "memcpy" would be present also. I used to same make command I did on Mac to build the library but maybe I did something wrong on Linux.

Regards,
        Ryan Joseph

_______________________________________________
fpc-pascal maillist  -  [hidden email]
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Static linking to C library on Linux

Karoly Balogh (Charlie/SGR)
Hi,

On Wed, 27 Nov 2019, Ryan Joseph via fpc-pascal wrote:

> I'm trying to link to a static library (built from Python sources) in
> Pascal and having troubles on Linux. On Mac where I have experience I
> successfully link and every works as expected but on Linux I get slews
> of linker errors in what appear to be standard C library functions.
> (snipp)
> Do I need to link to other system libraries when building on Linux?

Yes. By Default, FPC doesn't link against libc on Linux by default, only
when some more advanced things, say, threading is used. To the contrary on
Darwin (macOS), we always link against libc. There comes your difference
from. You can just try to add {$LINKLIB C} to your code, and see if that
helps.

> I'm also confused because I thought the purpose of the static libraries
> was to contain all the code they needed to run and thus these system
> functions like "memcpy" would be present also.

That's not true. Static libs can depend on other static libs, just the
resulting final executable won't depend on anything if statically linked.

Hope this helps,
--
Charlie
_______________________________________________
fpc-pascal maillist  -  [hidden email]
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Static linking to C library on Linux

Michael Van Canneyt
In reply to this post by Free Pascal - General mailing list


On Wed, 27 Nov 2019, Ryan Joseph via fpc-pascal wrote:

> I'm trying to link to a static library (built from Python sources) in Pascal and having troubles on Linux. On Mac where I have experience I successfully link and every works as expected but on Linux I get slews of linker errors in what appear to be standard C library functions.
>
> For example:
>
> /usr/bin/ld.bfd: /home/chris/pas/PythonBridge-master/sources/libpython3.7m.a(bytearrayobject.o): in function `memcpy':
> /usr/include/x86_64-linux-gnu/bits/string_fortified.h:34: undefined reference to `memcpy'
> /usr/bin/ld.bfd: /home/chris/pas/PythonBridge-master/sources/libpython3.7m.a(typeobject.o): in function `_PyType_Name':
> /home/chris/Downloads/Python-3.7.4/Objects/typeobject.c:417: undefined reference to `strrchr'
> /usr/bin/ld.bfd: /home/chris/Downloads/Python-3.7.4/Objects/typeobject.c:107: undefined reference to `strlen'
> /usr/bin/ld.bfd: /home/chris/Downloads/Python-3.7.4/Objects/typeobject.c:108: undefined reference to `strncmp'
> /usr/bin/ld.bfd: /home/chris/Downloads/Python-3.7.4/Objects/typeobject.c:103: undefined reference to `strrchr'
> /usr/bin/ld.bfd: /home/chris/Downloads/Python-3.7.4/Objects/typeobject.c:107: undefined reference to `strlen'
> /usr/bin/ld.bfd: /home/chris/Downloads/Python-3.7.4/Objects/typeobject.c:108: undefined reference to `strncmp'
>
> Do I need to link to other system libraries when building on Linux?  I'm
> also confused because I thought the purpose of the static libraries was to
> contain all the code they needed to run and thus these system functions
> like "memcpy" would be present also.  I used to same make command I did on
> Mac to build the library but maybe I did something wrong on Linux.

The C library will not be in your custom-built library, but will be
referenced if it uses functions from libc.

You need to link to libc. So add a
{$linklib c}
and you should be set.

Michael.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Static linking to C library on Linux

Free Pascal - General mailing list
In reply to this post by Karoly Balogh (Charlie/SGR)


> On Nov 27, 2019, at 11:25 AM, Karoly Balogh (Charlie/SGR) <[hidden email]> wrote:
>
> Yes. By Default, FPC doesn't link against libc on Linux by default, only
> when some more advanced things, say, threading is used. To the contrary on
> Darwin (macOS), we always link against libc. There comes your difference
> from. You can just try to add {$LINKLIB C} to your code, and see if that
> helps.
>

Thanks that makes sense now. {$LINKLIB C} did help some of the errors but I'm still getting others. I'm aware of 'ldd' to show dependancies for dynamic libraries but how can I do this for static libraries on Linux? I'm getting defined references for math functions like pow, cos etc.. and some pthread ones also. There's probably more but how can I know?

Regards,
        Ryan Joseph

_______________________________________________
fpc-pascal maillist  -  [hidden email]
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Static linking to C library on Linux

Karoly Balogh (Charlie/SGR)
Hi,

On Wed, 27 Nov 2019, Ryan Joseph via fpc-pascal wrote:

> > On Nov 27, 2019, at 11:25 AM, Karoly Balogh (Charlie/SGR) <[hidden email]> wrote:
> >
> > Yes. By Default, FPC doesn't link against libc on Linux by default, only
> > when some more advanced things, say, threading is used. To the contrary on
> > Darwin (macOS), we always link against libc. There comes your difference
> > from. You can just try to add {$LINKLIB C} to your code, and see if that
> > helps.
>
> Thanks that makes sense now. {$LINKLIB C} did help some of the errors
> but I'm still getting others. I'm aware of 'ldd' to show dependancies
> for dynamic libraries but how can I do this for static libraries on
> Linux? I'm getting defined references for math functions like pow, cos
> etc.. and some pthread ones also. There's probably more but how can I
> know?

I don't know anything like that for static libs... But maybe it exist
somehow. You probably also need libm (for pow/cos), and then for the
pthreads I'd just include "uses cthreads" in my uses clause, as that will
take care of pthreads, and integrating FPC into whatever threading going
properly.

There is no silver bullet, if you Google the missing symbols you can work
your way through what library is needed. Or if you look at some C example
from the static library you're trying to use, its makefile will probably
list the additional libraries it depends on.

Charlie
_______________________________________________
fpc-pascal maillist  -  [hidden email]
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Static linking to C library on Linux

Jonas Maebe-3
In reply to this post by Free Pascal - General mailing list
On 27/11/2019 17:59, Ryan Joseph via fpc-pascal wrote:
> how can I do this for static libraries on Linux?

There is no tool that can show this, because a static library is just a
collection of random object files. It does not contain any metadata
about where these symbols are supposed to come from (and they could o,
fact come from anywhere, be it individual object files, object files
collected in another static library, or a dynamic library).


Jonas
_______________________________________________
fpc-pascal maillist  -  [hidden email]
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Static linking to C library on Linux

Free Pascal - General mailing list


> On Nov 27, 2019, at 2:22 PM, Jonas Maebe <[hidden email]> wrote:
>
> There is no tool that can show this, because a static library is just a
> collection of random object files. It does not contain any metadata
> about where these symbols are supposed to come from (and they could o,
> fact come from anywhere, be it individual object files, object files
> collected in another static library, or a dynamic library).

That's what I had originally thought. Can static libraries have dependancies on other static libraries? I noticed that when I linked a 12MB library my final binary was only 5MB so I presume there were duplicate symbols that were ignored.

Regards,
        Ryan Joseph

_______________________________________________
fpc-pascal maillist  -  [hidden email]
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Static linking to C library on Linux

Jonas Maebe-3
On 27/11/2019 21:27, Ryan Joseph via fpc-pascal wrote:
>> On Nov 27, 2019, at 2:22 PM, Jonas Maebe <[hidden email]> wrote:
>>
>> There is no tool that can show this, because a static library is just a
>> collection of random object files. It does not contain any metadata
>> about where these symbols are supposed to come from (and they could in
>> fact come from anywhere, be it individual object files, object files
>> collected in another static library, or a dynamic library).
> That's what I had originally thought. Can static libraries have dependancies on other static libraries?

Again, a static library has no dependencies because it is literally just
a collection of object files. They're packed like a zipfile into a
single file for convenience, but that's it. Those object files can
contain references to symbols that are not defined in that same
collection of object files, but I mentioned above already that those
symbols can come from anywhere. It's like saying that an object file has
a dependency on a library. It never has. It just references external
symbols, which in turn could be defined anywhere.

The only thing that can depend on a (by definition dynamic (*) )
library, is an executable or dynamic library (or their kin, like
frameworks and dylib bundles on macOS). This happens when the static
linker, while looking for those undefined symbols, at link time
discovers that they happen to be defined in a dynamic library. In that
case, it will store a reference to that dynamic library in the generated
executable/dynamic library to indicate to the dynamic linker where it
should look for external symbols.

> I noticed that when I linked a 12MB library my final binary was only 5MB so I presume there were duplicate symbols that were ignored.

Code and data that are defined in object files (in turnoptionally
contained in static libraries), but that are not referenced any other
code part of the linking proceess, are simply not linked/included into
your binary (unless you explicitly force linking everything from a
referenced static library/object file).


Jonas

(*) On some platforms, like AIX, the situation is a bit more blurry, but
the main concept holds.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal