Compiling FPC for SPARC

classic Classic list List threaded Threaded
54 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Re: Compiling for SPARC

Mark Morgan Lloyd-5
Jonas Maebe wrote:

> Given the info below, it's not needed.

I'm an engineer. Take a close look at how something behaves before opening it
up. No doubt a medical man would agree :-)

> Very strange. smartlinking can be slow sometimes, but gtkglarea only
> results in 12 object files... Can you capture some command lines of
> ar commands that are executed?

Don't trust this yet, but it might be a problem with command, environment
variable or configuration file line length. I was starting off in a fairly deep
directory off ~/pascal/src when I was hitting the problem, I've now moved the
source to /fpcsrc and appear to be progressing further.

Work continues here.

--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Compiling for SPARC

Koenraad Lelong-3
Mark Morgan Lloyd schreef:

> Jonas Maebe wrote:
>
>
>>Given the info below, it's not needed.
>
>
> I'm an engineer. Take a close look at how something behaves before opening it
> up. No doubt a medical man would agree :-)
>
>
>>Very strange. smartlinking can be slow sometimes, but gtkglarea only
>>results in 12 object files... Can you capture some command lines of
>>ar commands that are executed?
>
>
> Don't trust this yet, but it might be a problem with command, environment
> variable or configuration file line length. I was starting off in a fairly deep
> directory off ~/pascal/src when I was hitting the problem, I've now moved the
> source to /fpcsrc and appear to be progressing further.
>
> Work continues here.
>
Thanks for this info, I had exactly the same problem when compiling
2.0.4-rc3 for arm-linux. Solved now, by reducing the path-lenth.
Regards,
Koenraad Lelong.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Compiling for SPARC

Mark Morgan Lloyd-5
Koenraad Lelong wrote:

> Thanks for this info, I had exactly the same problem when compiling
> 2.0.4-rc3 for arm-linux. Solved now, by reducing the path-lenth.
> Regards,

Glad to be of some help. I'll continue working on this to see if I can find out
how long the leading path has to be before there are problems, I also want to
try make -j 2 (etc.) to see if I can speed things up a bit since the make
process is painfully slow here.

--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Compiling for SPARC

Jonas Maebe-2

On 9 aug 2006, at 23:39, Mark Morgan Lloyd wrote:

>
>> Thanks for this info, I had exactly the same problem when compiling
>> 2.0.4-rc3 for arm-linux. Solved now, by reducing the path-lenth.
>> Regards,
>
> Glad to be of some help. I'll continue working on this to see if I  
> can find out
> how long the leading path has to be before there are problems,

Internally, the compiler uses a shortstring (max 255) characters to  
build the "ar" command line. It currently passes three file names at  
a time to "ar", so the currently max is about 80 characters per dir
+filename.

> I also want to
> try make -j 2 (etc.) to see if I can speed things up a bit since  
> the make
> process is painfully slow here.

It won't work, the makefiles are not written to support this.


Jonas
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Compiling for SPARC

Florian Klämpfl
Jonas Maebe wrote:

>
> On 9 aug 2006, at 23:39, Mark Morgan Lloyd wrote:
>
>>
>>> Thanks for this info, I had exactly the same problem when compiling
>>> 2.0.4-rc3 for arm-linux. Solved now, by reducing the path-lenth.
>>> Regards,
>>
>> Glad to be of some help. I'll continue working on this to see if I can
>> find out
>> how long the leading path has to be before there are problems,
>
> Internally, the compiler uses a shortstring (max 255) characters to
> build the "ar" command line. It currently passes three file names at a
> time to "ar", so the currently max is about 80 characters per dir+filename.
>
>> I also want to
>> try make -j 2 (etc.) to see if I can speed things up a bit since the make
>> process is painfully slow here.
>
> It won't work, the makefiles are not written to support this.

At least the rtl for sparc-solaris has a makefile supporting this.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Compiling for SPARC

Jonas Maebe-2

On 10 aug 2006, at 11:27, Florian Klaempfl wrote:

>> It won't work, the makefiles are not written to support this.
>
> At least the rtl for sparc-solaris has a makefile supporting this.

The rtl for darwin too, but that doesn't help for "make all" or even  
"make cycle". Even "make clean all" doesn't work properly, because  
the "clean" and "all" are done in parallel.


Jonas
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Compiling for SPARC

Mark Morgan Lloyd-5
In reply to this post by Florian Klämpfl
Florian Klaempfl wrote:
>
> Jonas Maebe wrote:

> >> I also want to
> >> try make -j 2 (etc.) to see if I can speed things up a bit since the make
> >> process is painfully slow here.
> >
> > It won't work, the makefiles are not written to support this.

Got there already :-/

> At least the rtl for sparc-solaris has a makefile supporting this.

I find that runtime increases significantly as the paths get longer, so I think
it's safe to assume that the make operation is I/O-bound. In this case I don't
think that increasing the number of jobs (e.g. to be the same as the number of
CPUs) would help significantly, at least in part since this would reduce the
amount of memory available for file caching. In other words probably not worth
fixing (but definately worth documenting).

--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Compiling for SPARC

Mark Morgan Lloyd-5
In reply to this post by Jonas Maebe-2
Jonas Maebe wrote:

> Internally, the compiler uses a shortstring (max 255) characters to
> build the "ar" command line. It currently passes three file names at
> a time to "ar", so the currently max is about 80 characters per dir
> +filename.

Executive summary: This fails when the longest filename is around 119
characters, but is OK when it is a couple of characters shorter.

Detail follows.


Working from a starting position of the source directory being /fpcsrc (i.e. 7
chars), I can confirm that this can be "stretched" to 51 chars but not to 53
which is the same length as the original path I was using. Once it gets to 53
chars it fails in the same place as before.

The shell running the make command displays:

/usr/bin/ar: creating units/sparc-linux/libpgdkpixbuf.a
make -C gtkgl all
make[6]: Entering directory
`/fpcsrcABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890123456789/packages/extra/gtk/gtkgl'
/fpcsrcABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890123456789/compiler/ppcsparc -XX -CX
-Ur -Xs  -n
-Fu/fpcsrcABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890123456789/rtl/units/sparc-linux
-Fu/fpcsrcABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890123456789/packages/extra/x11/units/sparc-linux
-Fu/fpcsrcABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890123456789/packages/extra/opengl/units/sparc-linux
-FE.
-FU/fpcsrcABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890123456789/packages/extra/gtk/units/sparc-linux
-Fl/usr/lib/gcc-lib/sparc-linux/3.3.5 -Fl/usr/X11R6/lib -dsparc -dRELEASE
gtkglarea.pp
/usr/bin/ar: creating
/fpcsrcABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890123456789/packages/extra/gtk/units/sparc-linux/libpgtkglarea.a

ps faux displays:

\_ make -C gtkgl all

     \_ /fpcsrcABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890123456789/compiler/ppcsparc
-XX -CX -Ur -Xs -n
-Fu/fpcsrcABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890123456789/rtl/units/sparc-linux
-Fu/fpcsrcABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890123456789/packages/extra/x11/units/sparc-linux
-Fu/fpcsrcABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890123456789/packages/extra/opengl/units/sparc-linux
-FE.
-FU/fpcsrcABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890123456789/packages/extra/gtk/units/sparc-linux
-Fl/usr/lib/gcc-lib/sparc-linux/3.3.5 -Fl/usr/X11R6/lib -dsparc -dRELEASE
gtkglarea.pp

        \_ /bin/sh -c /usr/bin/ar qS
/fpcsrcABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890123456789/packages/extra/gtk/units/sparc-linux/libpgtkglarea.a

and then:

\_ make -C gtkgl all

     \_ /fpcsrcABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890123456789/compiler/ppcsparc
-XX -CX -Ur -Xs -n
-Fu/fpcsrcABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890123456789/rtl/units/sparc-linux
-Fu/fpcsrcABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890123456789/packages/extra/x11/units/sparc-linux
-Fu/fpcsrcABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890123456789/packages/extra/opengl/units/sparc-linux
-FE.
-FU/fpcsrcABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890123456789/packages/extra/gtk/units/sparc-linux
-Fl/usr/lib/gcc-lib/sparc-linux/3.3.5 -Fl/usr/X11R6/lib -dsparc -dRELEASE
gtkglarea.pp

        \_ /usr/bin/ar qS
/fpcsrcABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890123456789/packages/extra/gtk/units/sparc-linux/libpgtkglarea.a

Sorry about the wrap on that.

Looking in ....gtkglarea.sl:

-rw-r--r--  1 markMLl markMLl 441 2006-08-11 13:06 gtkglarea0s17.o
-rw-r--r--  1 markMLl markMLl 441 2006-08-11 13:06 gtkglarea0s18.o
-rw-r--r--  1 markMLl markMLl 510 2006-08-11 13:06 gtkglarea0s10.o
-rw-r--r--  1 markMLl markMLl 502 2006-08-11 13:06 gtkglarea0s11.o
-rw-r--r--  1 markMLl markMLl 496 2006-08-11 13:06 gtkglarea0s12.o
-rw-r--r--  1 markMLl markMLl 680 2006-08-11 13:06 gtkglarea0s13.o
-rw-r--r--  1 markMLl markMLl 509 2006-08-11 13:06 gtkglarea0s14.o
-rw-r--r--  1 markMLl markMLl 632 2006-08-11 13:06 gtkglarea0s15.o
-rw-r--r--  1 markMLl markMLl 441 2006-08-11 13:06 gtkglarea0s16.o
-rw-r--r--  1 markMLl markMLl 493 2006-08-11 13:06 gtkglarea0s5.o
-rw-r--r--  1 markMLl markMLl 441 2006-08-11 13:06 gtkglarea0s6.o
-rw-r--r--  1 markMLl markMLl 441 2006-08-11 13:06 gtkglarea0s7.o
-rw-r--r--  1 markMLl markMLl 511 2006-08-11 13:06 gtkglarea0s8.o
-rw-r--r--  1 markMLl markMLl 503 2006-08-11 13:06 gtkglarea0s9.o
-rw-r--r--  1 markMLl markMLl 470 2006-08-11 13:06 gtkglarea0s1.o
-rw-r--r--  1 markMLl markMLl 724 2006-08-11 13:06 gtkglarea0s2.o
-rw-r--r--  1 markMLl markMLl 704 2006-08-11 13:06 gtkglarea0s3.o
-rw-r--r--  1 markMLl markMLl 441 2006-08-11 13:06 gtkglarea0s4.o

Current time is now 14:43 so these haven't been touched for some hours despite
the fact that the make is still trying to run.

Slecting the file here with the longest name, the entire path would be something
like

/fpcsrcABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890123456789/packages/extra/gtk/units/sparc-linux/gtkglarea.sl/gtkglarea0s18.o

which is 119 characters. Going by your earlier comment 3 x 119 is 357 which is
obviously longer than 255.

What I don't see is how this worked when the path was only a couple of
characters shorter.

As an experiment, I'm going to leave the actual directory the same length but
see if setting up a short symlink to it works.

--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Compiling for SPARC

Mark Morgan Lloyd-5
Mark Morgan Lloyd wrote:

> This fails when the longest filename is around 119 characters, but is OK
> when it is a couple of characters shorter.

> As an experiment, I'm going to leave the actual directory the same length but
> see if setting up a short symlink to it works.

Didn't help, make immediately expanded it to the full path.

--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Compiling for SPARC

Jonas Maebe-2

On 14 aug 2006, at 15:19, Mark Morgan Lloyd wrote:

>> This fails when the longest filename is around 119 characters, but  
>> is OK
>> when it is a couple of characters shorter.
>
>> As an experiment, I'm going to leave the actual directory the same  
>> length but
>> see if setting up a short symlink to it works.
>
> Didn't help, make immediately expanded it to the full path.

Actually, I was wrong earlier on when I said it would always add 3  
file names. It does this:

  function GetNextFiles(const maxCmdLength : AInt; var item :  
TStringListItem) : string;
     begin
       result := '';
       while (assigned(item) and ((length(result) + length(item.str)  
+ 1) < maxCmdLength)) do begin
         result := result + ' ' + item.str;
         item := TStringListItem(item.next);
       end;
     end;

...

       repeat
         Replace(nextcmd,'$FILES',GetNextFiles(240 - length(nextcmd)  
+ 6 - length(binstr) - 1, current));
         success:=DoExec(binstr,nextcmd,false,true);
         nextcmd := cmdstr;
       until (not assigned(current)) or (not success);


I'm not sure where exactly 119 fits in here, unless length(binstr)  
(full path to your copy of ar) around 120 chars or so.

And it's clear that if not a single file name fits in this string,  
this is going to produce an endless loop of empty "ar" invocations  
(as you experienced)...


Jonas

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Compiling for SPARC

Jonas Maebe-2

On 14 aug 2006, at 15:33, Jonas Maebe wrote:

> I'm not sure where exactly 119 fits in here, unless length(binstr)  
> (full path to your copy of ar) around 120 chars or so.

I forgot to add some more relevant code:

       Replace(cmdstr,'$LIB',maybequoted
(current_module.staticlibfilename^));
       { create AR commands }
       success := true;
       nextcmd := cmdstr;
       current := TStringListItem(SmartLinkOFiles.First);
       repeat
         Replace(nextcmd,'$FILES',GetNextFiles(240 - length(nextcmd)  
+ 6 - length
(binstr) - 1, current));
         success:=DoExec(binstr,nextcmd,false,true);
         nextcmd := cmdstr;
       until (not assigned(current)) or (not success);

The first replace already puts the library name in the string. So if  
the directory + name of that library (which always is in a subdir of  
the unit output directory) is around 120 chars, things do add up.

Not sure how to fix this properly though, except by finally doing  
that whole "replace Dos and shortstrings with Sysutils and  
ansistrings for all external interfacing in the compiler" thing.


Jonas
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Compiling for SPARC

Mark Morgan Lloyd-5
In reply to this post by Jonas Maebe-2
Jonas Maebe wrote:

> I'm not sure where exactly 119 fits in here, unless length(binstr)
> (full path to your copy of ar) around 120 chars or so.

0 1>markMLl@pye-dev-04:/fpcsrc$ which ar
/usr/bin/ar

About enough room for the path to ar, some options, and a couple of filenames.

> And it's clear that if not a single file name fits in this string,
> this is going to produce an endless loop of empty "ar" invocations
> (as you experienced)...

Yes, I thought it was going to end up something like that... :-/

--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Compiling for SPARC

Mark Morgan Lloyd-5
On 14 aug 2006, at 15:33, Jonas Maebe wrote:

> I forgot to add some more relevant code:
>
>       Replace(cmdstr,'$LIB',maybequoted(current_module.staticlibfilename^));
>       { create AR commands }
>       success := true;
>       nextcmd := cmdstr;
>       current := TStringListItem(SmartLinkOFiles.First);
>       repeat
>         Replace(nextcmd,'$FILES',GetNextFiles(240 - length(nextcmd) +
> 6 - length(binstr) - 1, current));
>         success:=DoExec(binstr,nextcmd,false,true);
>         nextcmd := cmdstr;
>       until (not assigned(current)) or (not success);
>
> The first replace already puts the library name in the string. So if
> the directory + name of that library (which always is in a subdir of
> the unit output directory) is around 120 chars, things do add up.
>
> Not sure how to fix this properly though, except by finally doing
> that whole "replace Dos and shortstrings with Sysutils and
> ansistrings for all external interfacing in the compiler" thing.

I think that for the moment I'd settle for an explicit warning and/or error
message, the first triggered when a path+file gets to be longer than around 105
characters and the second fatal.

I'm hoping that at some point I'll be familiar enough with things that I might
be some use working through problems like this, but I've got a long way to go
yet and a lot of problems on my plate at this end.

--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Compiling for SPARC

Jonas Maebe-2

On 15 Aug 2006, at 12:06, Mark Morgan Lloyd wrote:

> I'm hoping that at some point I'll be familiar enough with things  
> that I might
> be some use working through problems like this, but I've got a long  
> way to go
> yet and a lot of problems on my plate at this end.

FWIW, I wouldn't spend too much time on trying to get the text mode  
IDE (fp) compiled with GDB support at this moment, unless you really  
want to use it later on. It's something which is quite separate from  
most problems you'll encounter when compiling your own programs.


Jonas
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
123