Happy tickets benchmark

classic Classic list List threaded Threaded
75 messages Options
1234
Reply | Threaded
Open this post in threaded view
|

Re: Happy tickets benchmark

Serguei TARASSOV
On 14/02/2016 15:01, [hidden email] wrote:

> Date: Sun, 14 Feb 2016 13:17:38 +0100
> From: Giuliano Colla<[hidden email]>
> To: FPC-Pascal users discussions<[hidden email]>
> Subject: Re: [fpc-pascal] Happy tickets benchmark
>
> Il 14/02/2016 12:56, Florian Klaempfl ha scritto:
>> >In theory, a compiler could decide
>> >very good if add or lea is better. But this decision applies only to a
>> >certain core and not in general. So for a all-purpose compiler this
>> >makes little sense. If your application really depends on this, rewrite
>> >the hotspots in C and use the icc to compile it. It knows about these
>> >detais.
> Good to know. Thanks a lot.
>
> Giuliano
Not so good at all.
It doesn't explain why C# with IL is significantly better than native
code generated by FPC.

Regards,
Serguei
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
--
Regards,
Serguei
Reply | Threaded
Open this post in threaded view
|

Re: Happy tickets benchmark

Jürgen Hestermann
In reply to this post by Florian Klämpfl
Am 14.02.2016 um 12:56 schrieb Florian Klaempfl:
 > No really. It is not a matter of +1 vs. inc but how it is compiled: as
 > add or lea. And the decision add vs. lea is not straight forward. It
 > depends on the surrounding code and the exact core.

After reading this (especially the comments)
http://stackoverflow.com/questions/1658294/whats-the-purpose-of-the-lea-instruction
the speed also seems to be dependend on the specific processor used
(even whithin the same family)
which makes such benchmarks somewhat arbitrary (if the compiler does not
take it into account).

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Happy tickets benchmark

leledumbo
Administrator
In reply to this post by Serguei TARASSOV
> Not so good at all.
> It doesn't explain why C# with IL is significantly better than native
> code generated by FPC.

I believe the .NET runtime has optimizations that Florian, judging from his answer a few posts behind, is not willing to commit due to low real world benefit. He seems to have Prof. Wirth spirit in that compilation must be as fast as possible while generating code as optimized as it can in that available time. I don't understand though, why it can't be made another -OoXXX that's disabled by default and perhaps activated in -O3 and above only (-O2 is used to bootstrap the compiler toolchain, if you don't override it, so it won't be affected).
Reply | Threaded
Open this post in threaded view
|

Re: Happy tickets benchmark

Florian Klämpfl
Am 14.02.2016 um 16:08 schrieb leledumbo:

>> Not so good at all.
>> It doesn't explain why C# with IL is significantly better than native
>> code generated by FPC.
>
> I believe the .NET runtime has optimizations that Florian, judging from his
> answer a few posts behind, is not willing to commit due to low real world
> benefit. He seems to have Prof. Wirth spirit in that compilation must be as
> fast as possible while generating code as optimized as it can in that
> available time. I don't understand though, why it can't be made another
> -OoXXX that's disabled by default

As the original poster was even not able to find out that -Ooloopunroll is available and even helps
(at least with 3.0.0, see Graemes post) I see no point in another switch.

> and perhaps activated in -O3 and above
> only (-O2 is used to bootstrap the compiler toolchain, if you don't override
> it, so it won't be affected).

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Happy tickets benchmark

Graeme Geldenhuys-6
In reply to this post by Marco van de Voort
On 2016-02-14 12:58, Marco van de Voort wrote:
> simply because the code is much slower otherwise.

When debugging, speed should be irrelevant really. Most of the times
I'll step through code. Can't get slower than that! ;-)


Regards,
  - Graeme -

--
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://fpgui.sourceforge.net/

My public PGP key:  http://tinyurl.com/graeme-pgp
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Happy tickets benchmark

Adrian Veith-2
In reply to this post by Michael Van Canneyt
Hm,

doing the same trick in C, it goes down from:

40ms (original) to 3ms (omit the inner loop).

This is still the same distance to fpc (v 3.0.0 with -O4 -Ooloopunroll)

185ms (original) to 12ms (omit the inner loop).

C is 4 times faster here.

Am 14.02.2016 um 12:09 schrieb Michael Van Canneyt:

>
>
> On Sun, 14 Feb 2016, Florian Klaempfl wrote:
>
>> Am 14.02.2016 um 10:45 schrieb Mattias Gaertner:
>>> On Sun, 14 Feb 2016 10:35:22 +0100
>>> Florian Klaempfl <[hidden email]> wrote:
>>>
>>>> [...]
>>>> Do you think people will bother? Nobody mentioned to the original
>>>> poster
>>>> so far:
>>>> - that the used FPC is outdated
>>>> - that only -O2 is used instead of -O3 (or -O4 with 3.0.0)
>>>> - that even FPC 2.6.4 has a -Ooloopunroll option which is never
>>>> enabled
>>>> by default and which is worth a try
>>>>
>>>> I do not know if the points above really effect the example, but it
>>>> tells me enough not to bother either :)
>>>
>>> Maybe documentation helps here.
>>
>> You mean something like the page Size Matters? See the post of Martin
>> Schreiber how much such pages help.
>>
>>>
>>> Is there already a page "pimp my fpc"?
>>
>> In this case even fpc -h would have helped :)
>>
>> But actually, before bothering randomly with command line options, I
>> would just rewrite the inner loop. Something like
>>              for n7 := 0 to 9 do
>>                if n1 + n2 + n3 + n4 - n5 - n6 - n7 in [0..9] then
>>                  Inc(TicketsCount);
>> should be much better.
>
> To back up Florian with numbers:
>
> No in:
> Found 4816030 tickets. Elapsed time, msec: 171
>
> Using in:
> Found 4816030 tickets. Elapsed time, msec: 23
>
> Michael.
> _______________________________________________
> fpc-pascal maillist  -  [hidden email]
> http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Happy tickets benchmark

Florian Klämpfl
Well, as said before: if the speed of code like this is important for you,
use C.


Am 15. Februar 2016 7:24:29 vorm. schrieb Adrian Veith <[hidden email]>:

> Hm,
>
> doing the same trick in C, it goes down from:
>
> 40ms (original) to 3ms (omit the inner loop).
>
> This is still the same distance to fpc (v 3.0.0 with -O4 -Ooloopunroll)
>
> 185ms (original) to 12ms (omit the inner loop).
>
> C is 4 times faster here.
>
> Am 14.02.2016 um 12:09 schrieb Michael Van Canneyt:
>>
>>
>> On Sun, 14 Feb 2016, Florian Klaempfl wrote:
>>
>>> Am 14.02.2016 um 10:45 schrieb Mattias Gaertner:
>>>> On Sun, 14 Feb 2016 10:35:22 +0100
>>>> Florian Klaempfl <[hidden email]> wrote:
>>>>
>>>>> [...]
>>>>> Do you think people will bother? Nobody mentioned to the original
>>>>> poster
>>>>> so far:
>>>>> - that the used FPC is outdated
>>>>> - that only -O2 is used instead of -O3 (or -O4 with 3.0.0)
>>>>> - that even FPC 2.6.4 has a -Ooloopunroll option which is never
>>>>> enabled
>>>>> by default and which is worth a try
>>>>>
>>>>> I do not know if the points above really effect the example, but it
>>>>> tells me enough not to bother either :)
>>>>
>>>> Maybe documentation helps here.
>>>
>>> You mean something like the page Size Matters? See the post of Martin
>>> Schreiber how much such pages help.
>>>
>>>>
>>>> Is there already a page "pimp my fpc"?
>>>
>>> In this case even fpc -h would have helped :)
>>>
>>> But actually, before bothering randomly with command line options, I
>>> would just rewrite the inner loop. Something like
>>>              for n7 := 0 to 9 do
>>>                if n1 + n2 + n3 + n4 - n5 - n6 - n7 in [0..9] then
>>>                  Inc(TicketsCount);
>>> should be much better.
>>
>> To back up Florian with numbers:
>>
>> No in:
>> Found 4816030 tickets. Elapsed time, msec: 171
>>
>> Using in:
>> Found 4816030 tickets. Elapsed time, msec: 23
>>
>> Michael.
>> _______________________________________________
>> fpc-pascal maillist  -  [hidden email]
>> http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
>
> _______________________________________________
> fpc-pascal maillist  -  [hidden email]
> http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
>


_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Happy tickets benchmark

Serguei TARASSOV
In reply to this post by Serguei TARASSOV
On 15/02/2016 12:00, [hidden email] wrote:

> Date: Mon, 15 Feb 2016 07:55:55 +0100
> From: Florian Kl?mpfl<[hidden email]>
> To:<[hidden email]>, "FPC-Pascal users discussions"
> <[hidden email]>, Adrian Veith<[hidden email]>
> Subject: Re: [fpc-pascal] Happy tickets benchmark
> Message-ID:
> <[hidden email]>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Well, as said before: if the speed of code like this is important for you,
> use C.
It's a wrong choice.
As we can see and reproduce, at least C# or other Pascal-like
environments (Oxygene) are significantly faster.
http://www.arbinada.com/main/en/node/1532

Regards,
Serguei
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
--
Regards,
Serguei
Reply | Threaded
Open this post in threaded view
|

Re: Happy tickets benchmark

Michael Van Canneyt


On Mon, 15 Feb 2016, Serguei TARASSOV wrote:

> On 15/02/2016 12:00, [hidden email] wrote:
>> Date: Mon, 15 Feb 2016 07:55:55 +0100
>> From: Florian Kl?mpfl<[hidden email]>
>> To:<[hidden email]>, "FPC-Pascal users discussions"
>> <[hidden email]>, Adrian Veith<[hidden email]>
>> Subject: Re: [fpc-pascal] Happy tickets benchmark
>> Message-ID:
>> <[hidden email]>
>> Content-Type: text/plain; charset="iso-8859-1"
>>
>> Well, as said before: if the speed of code like this is important for you,
>> use C.
> It's a wrong choice.
> As we can see and reproduce, at least C# or other Pascal-like environments
> (Oxygene) are significantly faster.
> http://www.arbinada.com/main/en/node/1532
>

What Florian means is that this is very artificial code, and that - although
he has been able to apply the necessary patches to make FPC faster - the
necessary optimizations are not likely to help in real-life programs.

Michael.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Happy tickets benchmark

dmitry boyarintsev
In reply to this post by Mattias Gaertner
On Sun, Feb 14, 2016 at 4:45 AM, Mattias Gaertner <[hidden email]> wrote:
Maybe documentation helps here.

Is there already a page "pimp my fpc"?

In fact there's but.

But the information is a bit outdated.

thanks,
Dmitry 

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Happy tickets benchmark

Serguei TARASSOV
In reply to this post by Serguei TARASSOV
On 16/02/2016 12:00, [hidden email] wrote:

> Date: Mon, 15 Feb 2016 13:02:48 +0100 (CET)
> From: Michael Van Canneyt<[hidden email]>
> To: FPC-Pascal users discussions<[hidden email]>
> Subject: Re: [fpc-pascal] Happy tickets benchmark
>
> On Mon, 15 Feb 2016, Serguei TARASSOV wrote:
>
>> >On 15/02/2016 12:00,[hidden email]  wrote:
>>> >>Date: Mon, 15 Feb 2016 07:55:55 +0100
>>> >>From: Florian Kl?mpfl<[hidden email]>
>>> >>To:<[hidden email]>, "FPC-Pascal users discussions"
>>> >> <[hidden email]>, Adrian Veith<[hidden email]>
>>> >>Subject: Re: [fpc-pascal] Happy tickets benchmark
>>> >>Message-ID:
>>> >> <[hidden email]>
>>> >>Content-Type: text/plain; charset="iso-8859-1"
>>> >>
>>> >>Well, as said before: if the speed of code like this is important for you,
>>> >>use C.
>> >It's a wrong choice.
>> >As we can see and reproduce, at least C# or other Pascal-like environments
>> >(Oxygene) are significantly faster.
>> >http://www.arbinada.com/main/en/node/1532
>> >
> What Florian means is that this is very artificial code, and that - although
> he has been able to apply the necessary patches to make FPC faster - the
> necessary optimizations are not likely to help in real-life programs.
>
> Michael.

Sounds like the real-life programs don't use inner loops or don't solve
NP-complete problems  :)
For info, my real-life examples are the application server and the DSL
script engine.
So any improvement of quality of FPC's generated code are welcome.

Regards,
Serguei
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
--
Regards,
Serguei
Reply | Threaded
Open this post in threaded view
|

Re: Happy tickets benchmark

Michael Van Canneyt


On Tue, 16 Feb 2016, Serguei TARASSOV wrote:

>>> environments
>>> >(Oxygene) are significantly faster.
>>> >http://www.arbinada.com/main/en/node/1532
>>> >
>> What Florian means is that this is very artificial code, and that -
>> although
>> he has been able to apply the necessary patches to make FPC faster - the
>> necessary optimizations are not likely to help in real-life programs.
>>
>> Michael.
>
> Sounds like the real-life programs don't use inner loops or don't solve
> NP-complete problems  :)
> For info, my real-life examples are the application server and the DSL script
> engine.
> So any improvement of quality of FPC's generated code are welcome.

I have asked Florian to integrate his patch anyway, he has agreed,
so I imagine it will result in a new -OoNNN switch.

Michael.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Happy tickets benchmark

Adrian Veith
In reply to this post by Serguei TARASSOV
small remark for your testing series:
AVG makes no sense, you should test against MIN  - why ? the measured
results are contaminated by other activities on your system, so the
fastest result is the most accurate, because there is no way to make a
program to run faster, but many ways to make it run slower.

Am 16.02.2016 um 12:19 schrieb Serguei TARASSOV:

> On 16/02/2016 12:00, [hidden email] wrote:
>> Date: Mon, 15 Feb 2016 13:02:48 +0100 (CET)
>> From: Michael Van Canneyt<[hidden email]>
>> To: FPC-Pascal users discussions<[hidden email]>
>> Subject: Re: [fpc-pascal] Happy tickets benchmark
>>
>> On Mon, 15 Feb 2016, Serguei TARASSOV wrote:
>>
>>> >On 15/02/2016 12:00,[hidden email]  wrote:
>>>> >>Date: Mon, 15 Feb 2016 07:55:55 +0100
>>>> >>From: Florian Kl?mpfl<[hidden email]>
>>>> >>To:<[hidden email]>, "FPC-Pascal users discussions"
>>>> >>    <[hidden email]>, Adrian Veith<[hidden email]>
>>>> >>Subject: Re: [fpc-pascal] Happy tickets benchmark
>>>> >>Message-ID:
>>>> >>  
>>>> <[hidden email]>
>>>> >>Content-Type: text/plain; charset="iso-8859-1"
>>>> >>
>>>> >>Well, as said before: if the speed of code like this is important
>>>> for you,
>>>> >>use C.
>>> >It's a wrong choice.
>>> >As we can see and reproduce, at least C# or other Pascal-like
>>> environments
>>> >(Oxygene) are significantly faster.
>>> >http://www.arbinada.com/main/en/node/1532
>>> >
>> What Florian means is that this is very artificial code, and that -
>> although
>> he has been able to apply the necessary patches to make FPC faster - the
>> necessary optimizations are not likely to help in real-life programs.
>>
>> Michael.
>
> Sounds like the real-life programs don't use inner loops or don't
> solve NP-complete problems  :)
> For info, my real-life examples are the application server and the DSL
> script engine.
> So any improvement of quality of FPC's generated code are welcome.
>
> Regards,
> Serguei
> _______________________________________________
> fpc-pascal maillist  -  [hidden email]
> http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Happy tickets benchmark

Mark Morgan Lloyd-5
In reply to this post by Serguei TARASSOV
Serguei TARASSOV wrote:

>>>> >>Well, as said before: if the speed of code like this is important
>>>> for you,
>>>> >>use C.
>>> >It's a wrong choice.
>>> >As we can see and reproduce, at least C# or other Pascal-like
>>> environments
>>> >(Oxygene) are significantly faster.
>>> >http://www.arbinada.com/main/en/node/1532
>>> >
>> What Florian means is that this is very artificial code, and that -
>> although
>> he has been able to apply the necessary patches to make FPC faster - the
>> necessary optimizations are not likely to help in real-life programs.
>>
>> Michael.
>
> Sounds like the real-life programs don't use inner loops or don't solve
> NP-complete problems  :)
> For info, my real-life examples are the application server and the DSL
> script engine.
> So any improvement of quality of FPC's generated code are welcome.

Anybody using a high-level language for real work would be advised to
understand the problem a bit better so that they didn't have to use your
sort of brute-force approach. And at that point, having something that
expresses higher-level algorithms and coding practices (threads, dynamic
arrays) becomes at least as important as brute force efficiency.

In the past I've come across people trying to solve recreational
problems asking (stupid) questions like "would Python be faster than
FORTRAN", and then having somebody respond that their approach is so
naive that language choice makes no real difference.

So in the case of your chosen problem: you know whether the left-hand
sum is even or odd, so a trivial optimisation would be only looking at
half of the possible values of the final (right-hand) digit. At which
point, pointing out that FPC doesn't have a STEP or BY clause in the FOR
statement would be far more useful criticism.

--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Happy tickets benchmark

Paulo Costa-2
On 16-Feb-16 18:53, Mark Morgan Lloyd wrote:
> Serguei TARASSOV wrote:
...

>> Sounds like the real-life programs don't use inner loops or don't
>> solve NP-complete problems  :)
>> For info, my real-life examples are the application server and the DSL
>> script engine.
>> So any improvement of quality of FPC's generated code are welcome.
>
> Anybody using a high-level language for real work would be advised to
> understand the problem a bit better so that they didn't have to use your
> sort of brute-force approach. And at that point, having something that
> expresses higher-level algorithms and coding practices (threads, dynamic
> arrays) becomes at least as important as brute force efficiency.

I think that the original example should be treated as the code that is
used to trigger a bug. It must be the simplest possible so the bug can
be identified and corrected.

>... At which
> point, pointing out that FPC doesn't have a STEP or BY clause in the FOR
> statement would be far more useful criticism.

Depending on the problems that we want to solve, different language
capabilities become more or less important...

Paulo Costa
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Happy tickets benchmark

Serguei TARASSOV
In reply to this post by Serguei TARASSOV
On 17/02/2016 12:00, [hidden email] wrote:

> Date: Tue, 16 Feb 2016 14:44:42 +0100
> From: Adrian Veith<[hidden email]>
> To: FPC-Pascal users discussions<[hidden email]>
> Subject: Re: [fpc-pascal] Happy tickets benchmark
>
> small remark for your testing series:
> AVG makes no sense, you should test against MIN  - why ? the measured
> results are contaminated by other activities on your system, so the
> fastest result is the most accurate, because there is no way to make a
> program to run faster, but many ways to make it run slower.
No, the test against MIN shows only the case when the result was
_minimally contaminated_ in the series. But we don't know whether the
unused time was bigger or smaller than for other program.
Also, it is very probably that the minimal time in series of 1000 will
be better that in series of 10 and so on.

The average approach smooth the contaminated time in series for all
programs.
But you could use some better approaches like the direct measure of the
time used by CPU or simply remove extreme values.

I don't suppose that it changes anything in the relative comparison that
is the goal of test.

Regards,
Serguei
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
--
Regards,
Serguei
Reply | Threaded
Open this post in threaded view
|

Re: Happy tickets benchmark

Adrian Veith
I don't want to insist on this, but: if you measure the runtime of your
program you result = runtime + error. If you measure a series against
MIN you measure MIN(result) = runtime + MIN(error) which delivers the
best value for runtime.

Am 17.02.2016 um 12:28 schrieb Serguei TARASSOV:

> On 17/02/2016 12:00, [hidden email] wrote:
>> Date: Tue, 16 Feb 2016 14:44:42 +0100
>> From: Adrian Veith<[hidden email]>
>> To: FPC-Pascal users discussions<[hidden email]>
>> Subject: Re: [fpc-pascal] Happy tickets benchmark
>>
>> small remark for your testing series:
>> AVG makes no sense, you should test against MIN  - why ? the measured
>> results are contaminated by other activities on your system, so the
>> fastest result is the most accurate, because there is no way to make a
>> program to run faster, but many ways to make it run slower.
> No, the test against MIN shows only the case when the result was
> _minimally contaminated_ in the series. But we don't know whether the
> unused time was bigger or smaller than for other program.
> Also, it is very probably that the minimal time in series of 1000 will
> be better that in series of 10 and so on.
>
> The average approach smooth the contaminated time in series for all
> programs.
> But you could use some better approaches like the direct measure of
> the time used by CPU or simply remove extreme values.
>
> I don't suppose that it changes anything in the relative comparison
> that is the goal of test.
>
> Regards,
> Serguei
> _______________________________________________
> fpc-pascal maillist  -  [hidden email]
> http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Happy tickets benchmark

wkitty42
In reply to this post by Serguei TARASSOV
On 02/17/2016 06:28 AM, Serguei TARASSOV wrote:
> On 17/02/2016 12:00, [hidden email] wrote:
>> small remark for your testing series: AVG makes no sense, you should test
>> against MIN  - why ? the measured results are contaminated by other
>> activities on your system, so the fastest result is the most accurate,
>> because there is no way to make a program to run faster, but many ways to
>> make it run slower.
> No, the test against MIN shows only the case when the result was _minimally
> contaminated_ in the series. But we don't know whether the unused time was
> bigger or smaller than for other program.

i thought about this, too... especially considering starting the test's binary
from the command line each time and how disk caching may affect loading and
unloading...

> Also, it is very probably that the minimal time in series of 1000 will be
> better that in series of 10 and so on.
>
> The average approach smooth the contaminated time in series for all
> programs. But you could use some better approaches like the direct measure of
> the time used by CPU or simply remove extreme values.

one might also have the program go a little further than a simple one-time
execution and have it loop internally while keeping up with the minimum, maximum
and average times and output them at the end in addition to the other output in
the test itself... [code below]

> I don't suppose that it changes anything in the relative comparison that is
> the goal of test.

possibly not...


===== snip =====
program HappyTickets;

uses
   SysUtils, DateUtils;

var
   loopcnt : integer;
   elapsedtime, mintime, maxtime, avgtime: int64;


   procedure runtest;

   var
     n1, n2, n3, n4, n5, n6, n7, n8: 0..9;
     TicketsCount: int64;
     d1, d2: TDateTime;
   begin
     TicketsCount := 0;
     elapsedtime := 0;
     d1 := Now;
     for n1 := 0 to 9 do
       for n2 := 0 to 9 do
         for n3 := 0 to 9 do
           for n4 := 0 to 9 do
             for n5 := 0 to 9 do
               for n6 := 0 to 9 do
                 for n7 := 0 to 9 do
                   for n8 := 0 to 9 do
                     if n1 + n2 + n3 + n4 = n5 + n6 + n7 + n8 then
//                    if n1 + n2 + n3 + n4 - n5 - n6 - n7 in [0..9] then
                       Inc(TicketsCount);
//                      TicketsCount += 1;
//                      TicketsCount := TicketsCount + 1;
     d2 := Now;
     elapsedtime := DateUtils.MilliSecondsBetween(d1, d2);
     writeln('Round ',loopcnt:4,' Found ', TicketsCount, ' tickets. Elapsed
time, msec: ',elapsedtime );
   end;

begin
   elapsedtime := 0;
   mintime := 0;
   maxtime := 0;
   avgtime := 0;
   for loopcnt := 1 to 1000 do
     begin
       runtest;
       if mintime = 0 then mintime := elapsedtime;
       if maxtime = 0 then maxtime := elapsedtime;
       if avgtime = 0 then avgtime := elapsedtime
       else avgtime := (avgtime + elapsedtime) div 2;
       if elapsedtime < mintime then mintime := elapsedtime;
       if elapsedtime > maxtime then maxtime := elapsedtime;
       writeln('Min: ',mintime:5,'   Max: ',maxtime:5,'   Avg: ',avgtime);
     end;
end.

===== snip =====

--
  NOTE: No off-list assistance is given without prior approval.
        *Please keep mailing list traffic on the list* unless
        private contact is specifically requested and granted.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Happy tickets benchmark

Serguei TARASSOV
In reply to this post by Serguei TARASSOV
On 18/02/2016 12:00, [hidden email] wrote:
> Date: Wed, 17 Feb 2016 18:55:29 +0100
> From: Adrian Veith<[hidden email]>
> To: FPC-Pascal users discussions<[hidden email]>
> Subject: Re: [fpc-pascal] Happy tickets benchmark
>
> I don't want to insist on this, but: if you measure the runtime of your
> program you result = runtime + error. If you measure a series against
> MIN you measure MIN(result) = runtime + MIN(error) which delivers the
> best value for runtime.
Not at all, any series against MIN delivers the best value for
MIN(result) not for runtime!

Ex.
Prog 1
Run 1: 130 = 100 + 30
Run 2: 160 = 100 + 60
Run 3: 170 = 100 + 70
Min = 130, Avg = 153

Prog 2
Run 1: 120 = 110 + 10
Run 2: 150 = 110 + 40
Run 3: 200 = 110 + 90
Min = 120, Avg = 157

MIN shows that Prog 2 is faster that is wrong.
AVG shows that Prog 1 is faster.

Regards,
Serguei

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
--
Regards,
Serguei
Reply | Threaded
Open this post in threaded view
|

Re: [OT] Happy tickets benchmark

Adrian Veith
I mark this OT because, this group is surly not interested in this and I
am not going to answer on this anymore. We have a saying in Germany:
"Wer misst, misst Mist".

cheers, Adrian.

Am 18.02.2016 um 12:15 schrieb Serguei TARASSOV:

> On 18/02/2016 12:00, [hidden email] wrote:
>> Date: Wed, 17 Feb 2016 18:55:29 +0100
>> From: Adrian Veith<[hidden email]>
>> To: FPC-Pascal users discussions<[hidden email]>
>> Subject: Re: [fpc-pascal] Happy tickets benchmark
>>
>> I don't want to insist on this, but: if you measure the runtime of your
>> program you result = runtime + error. If you measure a series against
>> MIN you measure MIN(result) = runtime + MIN(error) which delivers the
>> best value for runtime.
> Not at all, any series against MIN delivers the best value for
> MIN(result) not for runtime!
>
> Ex.
> Prog 1
> Run 1: 130 = 100 + 30
> Run 2: 160 = 100 + 60
> Run 3: 170 = 100 + 70
> Min = 130, Avg = 153
>
> Prog 2
> Run 1: 120 = 110 + 10
> Run 2: 150 = 110 + 40
> Run 3: 200 = 110 + 90
> Min = 120, Avg = 157
>
> MIN shows that Prog 2 is faster that is wrong.
> AVG shows that Prog 1 is faster.
>
> Regards,
> Serguei
>
> _______________________________________________
> fpc-pascal maillist  -  [hidden email]
> http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
1234