Optimal number of threads for SMP

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Optimal number of threads for SMP

Xiangrong Fang
Hi All,

If my application needs SPEED, i.e. take full advantage of CPU capabilities, and the application has no I/O operation at all (neither disk nor network), it seems no need to create threads more than the number of CPU cores (hyper-threads)?

Am I right?

Xiangrong

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Optimal number of threads for SMP

Marco van de Voort
In our previous episode, Xiangrong Fang said:
> If my application needs SPEED, i.e. take full advantage of CPU
> capabilities, and the application has no I/O operation at all (neither disk
> nor network), it seems no need to create threads more than the number of
> CPU cores ?hyper-threads)?
>
> Am I right?

My rule of thumb is physical cores + a percentage. (like 10-20%).

The idea is to saturate the CPU, but only just so, since saturating by way
too many threads increasing switching time and thus decreases performance.

The percentage is a rule of thumb approach to account for some minor I/O (if
only OS background), scheduler/locks switch time,  and, possibly
hyperthreading.

But in my apps I always make it configurable, and use this sum only to seed
the default of the configuration setting.

This because you don't always want to fully utilize the computer (e.g.
to give some air to background services, or because you are working on it),
and because automatic detection regularly goes wrong with new CPUs etc.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Optimal number of threads for SMP

Bernd K.
In reply to this post by Xiangrong Fang
2014-09-15 11:10 GMT+02:00 Xiangrong Fang <[hidden email]>:
> (hyper-threads)?


If the CPU has hyper threading then these should show up as individual
cores (CPUs). They call a "core" with 2 processors on it a "core" but
each of these "cores" is actually two complete processors (cores in
the conventional meaning of the word) that just share some common
circuitry to speed up some things (and slow down other things).

If you have 2 hyper cores then your operating system should actually
report 4 CPUs (cores) because that is what it actually is if you
remove all the intel marketing terminology, no need to double any
numbers. If it reports 4 CPUs then you have exactly 4 CPUs.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Optimal number of threads for SMP

Marco van de Voort
In our previous episode, Bernd said:

> If the CPU has hyper threading then these should show up as individual
> cores (CPUs). They call a "core" with 2 processors on it a "core" but
> each of these "cores" is actually two complete processors (cores in
> the conventional meaning of the word) that just share some common
> circuitry to speed up some things (and slow down other things).
>
> If you have 2 hyper cores then your operating system should actually
> report 4 CPUs (cores) because that is what it actually is if you
> remove all the intel marketing terminology, no need to double any
> numbers. If it reports 4 CPUs then you have exactly 4 CPUs.

Hyperthreading doubles the execution stream logic, so that one core can
take two incoming streams of instructions. The idea is that when one thread
stalls, or executes unoptimal code, the other thread can run, increasing
utilization. But that is more something like 10%, not doubling it.

Since the rest of the core (the execution units) can't actually handle
processing two streams at full speed.

You are probably confused with AMD's module system, which does duplicate
cores, and only shares some logic (like AVX)
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Optimal number of threads for SMP

Mark Morgan Lloyd-5
Marco van de Voort wrote:

> Hyperthreading doubles the execution stream logic, so that one core can
> take two incoming streams of instructions. The idea is that when one thread
> stalls, or executes unoptimal code, the other thread can run, increasing
> utilization. But that is more something like 10%, not doubling it.

I'd expect somewhat better than 10%, but another complicating factor is
that it will depend on the cache architecture. Also in the general case
some architectures can manage a lot of integer threads but have a
massive bottleneck if multiple threads try to do floating point operations.

--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Optimal number of threads for SMP

Marco van de Voort
In our previous episode, Mark Morgan Lloyd said:
> > Hyperthreading doubles the execution stream logic, so that one core can
> > take two incoming streams of instructions. The idea is that when one thread
> > stalls, or executes unoptimal code, the other thread can run, increasing
> > utilization. But that is more something like 10%, not doubling it.
>
> I'd expect somewhat better than 10%,

It's a bit low. It is more like 15%. Based on kernel compile benchmarks
on phoronix.com done with and without HT.

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Optimal number of threads for SMP

Michael Schnell
In reply to this post by Marco van de Voort
On 09/15/2014 11:19 AM, Marco van de Voort wrote:
> too many threads increasing switching time and thus decreases performance

Not only this.

Additionally Linux tries to keep a thread/Task at the same CPU when
re-scheduling the CPUs. By this the count of misses on the (primary)
caches is decreased.

AFAIK, cache misses can result in a huge degradation.

Hence - depending on the task to be done - using more threads than CPUs
might not be a good idea.

-Michael
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Optimal number of threads for SMP

Michael Schnell
In reply to this post by Bernd K.
On 09/18/2014 06:53 PM, Bernd wrote:
> If it reports 4 CPUs then you have exactly 4 CPUs.
Correct from a algorithm POV, but not from a performance POV.

-Michael
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Optimal number of threads for SMP

Graeme Geldenhuys-6
In reply to this post by Marco van de Voort
On 2014-09-15 10:19, Marco van de Voort wrote:
> My rule of thumb is physical cores + a percentage. (like 10-20%).

Tell that to most developers out there, they clearly don't know that
"rule of thumb". :-)  Mozilla Thunderbird under Win7 shows 47 threads.
EditPad Pro 7 shows 18 threads. etc etc... I only have a mere dual-core
system.


Regards,
  - Graeme -

--
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://fpgui.sourceforge.net/
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Optimal number of threads for SMP

Sven Barth-2
On 20.09.2014 01:48, Graeme Geldenhuys wrote:
> On 2014-09-15 10:19, Marco van de Voort wrote:
>> My rule of thumb is physical cores + a percentage. (like 10-20%).
>
> Tell that to most developers out there, they clearly don't know that
> "rule of thumb". :-)  Mozilla Thunderbird under Win7 shows 47 threads.
> EditPad Pro 7 shows 18 threads. etc etc... I only have a mere dual-core
> system.

And look at that, I'm sure that you have more than one process running.

Only because there are 47 or 18 threads does not mean that every thread
is doing something right now. Most threads in those applications are
just waiting (blocking) for the right event to trigger them to do
something. And in that case the "rule of thumb" doesn't help much. But
when we're talking about really active threads that do calculate
something or so then the "rule of thumb" is quite good, because
otherwise there will be more thread switching between the threads your
program has (plus the other threads of other processes) than when you
only have a bit more threads active than cores.

Regards,
Sven

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Optimal number of threads for SMP

Marco van de Voort
In reply to this post by Graeme Geldenhuys-6
In our previous episode, Graeme Geldenhuys said:
> > My rule of thumb is physical cores + a percentage. (like 10-20%).
>
> Tell that to most developers out there, they clearly don't know that
> "rule of thumb". :-)  Mozilla Thunderbird under Win7 shows 47 threads.
> EditPad Pro 7 shows 18 threads. etc etc... I only have a mere dual-core
> system.

Not all thraeds are about increasing performance. Usually threads are just
to do (or monitor) things asynchroniously.

IOW, this is the about guestimating optimal threadcount if you divide one
task over multiple threads.  The point is that this is often not equal to
the number of hardware threads.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Optimal number of threads for SMP

Michael Schnell
In reply to this post by Graeme Geldenhuys-6
On 09/20/2014 01:48 AM, Graeme Geldenhuys wrote:
> On 2014-09-15 10:19, Marco van de Voort wrote:
>> My rule of thumb is physical cores + a percentage. (like 10-20%).
> Tell that to most developers out there, they clearly don't know that
> "rule of thumb". :-)  Mozilla Thunderbird under Win7 shows 47 threads.
>
Increasing Performance is only  one of the purposes you might have in
mind when using threads. "Traditionally" threads are just that:
unrelated threads of execution in a project, i.e. multitasking-tasks
that share some memory.

-Michael
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal