Multicore support needs a big overhaul

[steko]

Thx

First the only official statement on multiprocessor/core load distribution in an now closed beta thread. Now actually nothing on hyperthreading, besides the "no" I linked to…

WTF?

[pencilrocket]

So am I getting this right?:

On OS X hyperthreading works in Live anyway, options.txt line not needed.

On Windows hyperthreading only works with the options.text line activated?

<confused>

Then why was your stress test for Live awful comparing to that of Reaper? It doesn't make sense...

[steko]

http://forum.ableton.com/viewtopic.php? ... 4&start=30

On my 2008 8-core (no virtual cores/hyperthreads!)

Mine wasn't awful, compared. You're confusing things...

http://forum.ableton.com/viewtopic.php?f=1&t=172654

As for Vance (OP) - still unclear... Would be interesting, if someone made a comparison test with non U-he plugins!

[siliconarc]

So am I getting this right?:

On OS X hyperthreading works in Live anyway, options.txt line not needed.

On Windows hyperthreading only works with the options.text line activated?

<confused>

i think what happens on OSX is the OS takes care of the hyperthreading to a certain extent.
but pushing the cpu with Live to breaking point = 50% usage on each virtual core (8 on a quad i7 MBP here), so Live still needs hyperthreading support on OSX. the options entry has no effect.

[steko]

i think what happens on OSX is the OS takes care of the hyperthreading to a certain extent.

+1

[pencilrocket]

http://forum.ableton.com/viewtopic.php? ... 4&start=30

On my 2008 8-core (no virtual cores/hyperthreads!)

Mine wasn't awful, compared. You're confusing things...

http://forum.ableton.com/viewtopic.php?f=1&t=172654

As for Vance (OP) - still unclear... Would be interesting, if someone made a comparison test with non U-he plugins!

Ah sry. I mistook the test. Thanks for clarifying.

[cotdagoo]

Would be interesting, if someone made a comparison test with non U-he plugins!

There's definitely something not entirely right with this hyper threading stuff.

Just tried a few similar tests to the ones I did with DIVA but used Kairatune (http://www.futucraft.com/kairatune/) instead.

Tried both with, and without the line in options.txt and not getting the same results at all.

Without the line in options.txt I do manage to get all cores going, but it took me 32 tracks of Kairatune to do so (a single track used 11% CPU according to Live's meter)

Or maybe something not right with the understanding of how Live chooses what cores to use for what..

I was under the impression that each track was dedicated to a core unless it was linked somehow to another track via sends or routed to another track.

http://forum.ableton.com/viewtopic.php?p=459019#p459019

Live distributes threads on both processors/cores only from separated track: it is not possible to separate one track's processing, not to separate the calculation of tracks routed in one another.

So assuming you've got nothing routed or sends and just track after track after track using up all your cores, one would expect the load to be balanced between them the same way each time.

It seems to tie directly to how much CPU is being utilized when deciding to split the load or start using another core.. So if a low CPU VST (anything compared to DIVA, ACE) is used, the load isn't balanced until it reaches a certain point of CPU utilization.

So if say 8 tracks are used, you'd expect on a quad core that each core would get it's own track.. but that doesn't seem to be the case until the CPU is getting pushed beyond a certain point.

So confused by all this.. think I'll just step back and see how it all plays out.

[steko]

For thread distribution also: http://forum.ableton.com/viewtopic.php? ... 9#p1345049

& yeah, strange shit...

[Todd OMG]

Having no problem with Live at all here. Not able to repeat OP's results. This is how I tested:

Live 8.3, DCAM Cypher on 4x realtime with 4 voices, 4 unison playing straight 16th notes on C3.

Was able to load 31 instances before I heard the first dropout. Then I turned on HyperThreading using the Options.txt and got the exact same results.

I then tested using Pro Tools 9 using exact same parameters. CPU usage showed a little different than Live's in both PT's meter and Task Manager's meters, but got the first dropout at 31 tracks all the same. Totally identical. In fact, Live was more accurate in its metering than PT was. The percentage was exactly the same as Task Managers using Live, but was different when using PT. PT displayed the CPU usage as about 10-20% less than Task Manager showed, for whatever reason.

Rig:
Windows 7 64bit
i5 2500k OC 3.5ghz

[friend_kami]

....i think something else i the reason for the peaks and dropouts! Have you set up the MacPro for audio? Turning off "Spotlight" for instance? I got the 8 Core MacPro and have newer been happier and I dont have any dropout even when using alot of synths and plugins....

uh.
if reaper can handle 64 tracks and live can only handle 24 tracks, do you think spotlight will solve that problem?

[h4nc0]

guess I wasn't the only one experiencing the big gap of performance difference between Live and other sequencers.

[mattadms]

Most certainly not...
I wonder what I have the i7 for if ableton only seems to use like 25% of its total potential...
its ridiculous and probably the only DAW which makes a modern computer feel like its 2005 all over again...

[Valiumdupeuple]

Common Ableton!
Can't you solve this problem? Our CPUs almost sleep when Live's geting crazy.

[Vance]

Wow, cool to see this thread is still alive

Having no problem with Live at all here. Not able to repeat OP's results. This is how I tested:

Live 8.3, DCAM Cypher on 4x realtime with 4 voices, 4 unison playing straight 16th notes on C3.

Was able to load 31 instances before I heard the first dropout. Then I turned on HyperThreading using the Options.txt and got the exact same results.

I then tested using Pro Tools 9 using exact same parameters. CPU usage showed a little different than Live's in both PT's meter and Task Manager's meters, but got the first dropout at 31 tracks all the same. Totally identical. In fact, Live was more accurate in its metering than PT was. The percentage was exactly the same as Task Managers using Live, but was different when using PT. PT displayed the CPU usage as about 10-20% less than Task Manager showed, for whatever reason.

Rig:
Windows 7 64bit
i5 2500k OC 3.5ghz

Todd OMG: the synth setting you're using is the issue - it's just not comparable in terms of CPU usage to the ACE preset I used in my initial posts because it uses less CPU. You could run a million different tests with a million different plugins but you can't compare against my results unless you use the same preset.

If I use less-intensive presets or plugins in this test, then yes I can push beyond 24 tracks as well... but that doesn't prove anything at all IMO. The interesting thing about my test is that there is a strong correlation between the point where Live starts having show-stopping issues and the number of "logical CPUs" the machine has, and this reveals something key about how multicore support actually works in Live. The CPU meter screenshots I provided show only that Live starts to have big issues well before my machine's CPUs are actually anywhere close to being fully used - so again, it implies the issue is with Live, not my machine itself running out of power.

You might be able to get more instances out of other plugins and/or other presets, but that doesn't alter the underlying behaviour of Live. My particular test just happens to be good at making Live's behaviour apparent.

Just for the sake of replicating my results properly, here's what I've been using:

Ableton Project: https://dl.dropbox.com/u/7988988/multicore_ace.alp
ACE preset (in the native ACE preset format): https://dl.dropbox.com/u/7988988/multicore_test.h2p

All this talk about hyperthreading is also missing the mark IMO. CPUs have it for a reason, and modern OSes are designed to use it. My 12-core machine is viewed by the OS as having 24 "logical CPUs" (due to Hyperthreading), and in my tests Ableton works perfectly up 24 tracks... so how could disabling Hyperthreading (and therefore halving the number of "logical CPUs" visible to the OS) possibly help? I just don't think it has anything to do with Hyperthreading. Seems to me like Hyperthreading is what allows my machine to get 24 usable tracks in this test instead of 12.

[fishmonkey]

If I use less-intensive presets or plugins in this test, then yes I can push beyond 24 tracks as well... but that doesn't prove anything at all IMO. The interesting thing about my test is that there is a strong correlation between the point where Live starts having show-stopping issues and the number of "logical CPUs" the machine has, and this reveals something key about how multicore support actually works in Live. The CPU meter screenshots I provided show only that Live starts to have big issues well before my machine's CPUs are actually anywhere close to being fully used - so again, it implies the issue is with Live, not my machine itself running out of power.

yes, you've found a scenario that stresses your system in a particular way. it's not a magic number though. if you are really interested in this behaviour, why not learn about how computational parallelisation actually works?

You might be able to get more instances out of other plugins and/or other presets, but that doesn't alter the underlying behaviour of Live. My particular test just happens to be good at making Live's behaviour apparent.

again, the behaviour is no great mystery.

All this talk about hyperthreading is also missing the mark IMO. CPUs have it for a reason, and modern OSes are designed to use it. My 12-core machine is viewed by the OS as having 24 "logical CPUs" (due to Hyperthreading), and in my tests Ableton works perfectly up 24 tracks... so how could disabling Hyperthreading (and therefore halving the number of "logical CPUs" visible to the OS) possibly help? I just don't think it has anything to do with Hyperthreading. Seems to me like Hyperthreading is what allows my machine to get 24 usable tracks in this test instead of 12.

actually, it's quite possible that with hyperthreading turned off, you might actually end up with similar performance for this particular test.

Live will never ever be able to utilise CPU power in the way that other linear DAWs can. it's an unavoidable tradeoff that comes from the dynamic nature of Live, and there will always be a tradeoff between performance latency and CPU utilisation.