Multicore support needs a big overhaul

Discussion of music production, audio, equipment and any related topics, either with or without Ableton Live
steko
Posts: 3354
Joined: Thu Jan 22, 2009 8:29 pm

Re: Multicore support needs a big overhaul

Post by steko » Tue Apr 10, 2012 7:53 pm

Thx

First the only official statement on multiprocessor/core load distribution in an now closed beta thread. Now actually nothing on hyperthreading, besides the "no" I linked to…

WTF?
Rahad Jackson wrote:My Awesome Mix Tape #6

pencilrocket
Posts: 1718
Joined: Tue Jun 15, 2010 10:46 am

Re: Multicore support needs a big overhaul

Post by pencilrocket » Wed Apr 11, 2012 5:04 am

steko wrote:So am I getting this right?:

On OS X hyperthreading works in Live anyway, options.txt line not needed.

On Windows hyperthreading only works with the options.text line activated?

<confused>
Then why was your stress test for Live awful comparing to that of Reaper? It doesn't make sense...
Image

steko
Posts: 3354
Joined: Thu Jan 22, 2009 8:29 pm

Re: Multicore support needs a big overhaul

Post by steko » Wed Apr 11, 2012 4:03 pm

http://forum.ableton.com/viewtopic.php? ... 4&start=30
steko wrote:On my 2008 8-core (no virtual cores/hyperthreads!)
Mine wasn't awful, compared. You're confusing things...

http://forum.ableton.com/viewtopic.php?f=1&t=172654

As for Vance (OP) - still unclear... Would be interesting, if someone made a comparison test with non U-he plugins!
Rahad Jackson wrote:My Awesome Mix Tape #6

garyboozy
Posts: 2612
Joined: Mon Jul 16, 2007 12:27 pm
Location: UK
Contact:

Re: Multicore support needs a big overhaul

Post by garyboozy » Wed Apr 11, 2012 4:19 pm

steko wrote:So am I getting this right?:

On OS X hyperthreading works in Live anyway, options.txt line not needed.

On Windows hyperthreading only works with the options.text line activated?

<confused>
i think what happens on OSX is the OS takes care of the hyperthreading to a certain extent.
but pushing the cpu with Live to breaking point = 50% usage on each virtual core (8 on a quad i7 MBP here), so Live still needs hyperthreading support on OSX. the options entry has no effect.

steko
Posts: 3354
Joined: Thu Jan 22, 2009 8:29 pm

Re: Multicore support needs a big overhaul

Post by steko » Wed Apr 11, 2012 4:25 pm

i think what happens on OSX is the OS takes care of the hyperthreading to a certain extent.
+1
Rahad Jackson wrote:My Awesome Mix Tape #6

pencilrocket
Posts: 1718
Joined: Tue Jun 15, 2010 10:46 am

Re: Multicore support needs a big overhaul

Post by pencilrocket » Wed Apr 11, 2012 4:51 pm

steko wrote:http://forum.ableton.com/viewtopic.php? ... 4&start=30
steko wrote:On my 2008 8-core (no virtual cores/hyperthreads!)
Mine wasn't awful, compared. You're confusing things...

http://forum.ableton.com/viewtopic.php?f=1&t=172654

As for Vance (OP) - still unclear... Would be interesting, if someone made a comparison test with non U-he plugins!
Ah sry. I mistook the test. Thanks for clarifying.
Image

cotdagoo
Posts: 1041
Joined: Fri Oct 26, 2007 5:34 pm
Location: canuhduh

Re: Multicore support needs a big overhaul

Post by cotdagoo » Wed Apr 11, 2012 5:04 pm

steko wrote:Would be interesting, if someone made a comparison test with non U-he plugins!
There's definitely something not entirely right with this hyper threading stuff.

Just tried a few similar tests to the ones I did with DIVA but used Kairatune (http://www.futucraft.com/kairatune/) instead.

Tried both with, and without the line in options.txt and not getting the same results at all.

Without the line in options.txt I do manage to get all cores going, but it took me 32 tracks of Kairatune to do so (a single track used 11% CPU according to Live's meter)

Or maybe something not right with the understanding of how Live chooses what cores to use for what..

I was under the impression that each track was dedicated to a core unless it was linked somehow to another track via sends or routed to another track.

http://forum.ableton.com/viewtopic.php?p=459019#p459019
Amaury wrote: Live distributes threads on both processors/cores only from separated track: it is not possible to separate one track's processing, not to separate the calculation of tracks routed in one another.


So assuming you've got nothing routed or sends and just track after track after track using up all your cores, one would expect the load to be balanced between them the same way each time.

It seems to tie directly to how much CPU is being utilized when deciding to split the load or start using another core.. So if a low CPU VST (anything compared to DIVA, ACE) is used, the load isn't balanced until it reaches a certain point of CPU utilization.

So if say 8 tracks are used, you'd expect on a quad core that each core would get it's own track.. but that doesn't seem to be the case until the CPU is getting pushed beyond a certain point.

So confused by all this.. think I'll just step back and see how it all plays out.

steko
Posts: 3354
Joined: Thu Jan 22, 2009 8:29 pm

Re: Multicore support needs a big overhaul

Post by steko » Wed Apr 11, 2012 5:11 pm

For thread distribution also: http://forum.ableton.com/viewtopic.php? ... 9#p1345049

& yeah, strange shit...
Rahad Jackson wrote:My Awesome Mix Tape #6

Todd OMG
Posts: 23
Joined: Wed Nov 25, 2009 1:31 pm
Location: CA
Contact:

Re: Multicore support needs a big overhaul

Post by Todd OMG » Wed Apr 11, 2012 9:19 pm

Having no problem with Live at all here. Not able to repeat OP's results. This is how I tested:

Live 8.3, DCAM Cypher on 4x realtime with 4 voices, 4 unison playing straight 16th notes on C3.

Was able to load 31 instances before I heard the first dropout. Then I turned on HyperThreading using the Options.txt and got the exact same results.

I then tested using Pro Tools 9 using exact same parameters. CPU usage showed a little different than Live's in both PT's meter and Task Manager's meters, but got the first dropout at 31 tracks all the same. Totally identical. In fact, Live was more accurate in its metering than PT was. The percentage was exactly the same as Task Managers using Live, but was different when using PT. PT displayed the CPU usage as about 10-20% less than Task Manager showed, for whatever reason.

Rig:
Windows 7 64bit
i5 2500k OC 3.5ghz

friend_kami
Posts: 2255
Joined: Mon May 29, 2006 10:10 pm

Re: Multicore support needs a big overhaul

Post by friend_kami » Sun Jun 24, 2012 3:53 pm

xzusa8ky wrote:....i think something else i the reason for the peaks and dropouts! Have you set up the MacPro for audio? Turning off "Spotlight" for instance? I got the 8 Core MacPro and have newer been happier and I dont have any dropout even when using alot of synths and plugins.... :D

uh.
if reaper can handle 64 tracks and live can only handle 24 tracks, do you think spotlight will solve that problem?

h4nc0
Posts: 141
Joined: Fri Dec 05, 2008 10:58 pm

Re: Multicore support needs a big overhaul

Post by h4nc0 » Sun Jun 24, 2012 3:58 pm

guess I wasn't the only one experiencing the big gap of performance difference between Live and other sequencers.

mattadms
Posts: 108
Joined: Thu Nov 04, 2010 11:43 am

Re: Multicore support needs a big overhaul

Post by mattadms » Wed Jun 27, 2012 1:03 pm

Most certainly not...
I wonder what I have the i7 for if ableton only seems to use like 25% of its total potential...
its ridiculous and probably the only DAW which makes a modern computer feel like its 2005 all over again...

Valiumdupeuple
Posts: 1048
Joined: Mon May 31, 2010 12:36 pm
Contact:

Re: Multicore support needs a big overhaul

Post by Valiumdupeuple » Tue Sep 18, 2012 12:01 am

Common Ableton!
Can't you solve this problem? Our CPUs almost sleep when Live's geting crazy.
Image

Vance
Posts: 348
Joined: Mon Aug 30, 2004 10:35 am

Re: Multicore support needs a big overhaul

Post by Vance » Tue Sep 18, 2012 1:23 am

Wow, cool to see this thread is still alive :)
Todd OMG wrote:Having no problem with Live at all here. Not able to repeat OP's results. This is how I tested:

Live 8.3, DCAM Cypher on 4x realtime with 4 voices, 4 unison playing straight 16th notes on C3.

Was able to load 31 instances before I heard the first dropout. Then I turned on HyperThreading using the Options.txt and got the exact same results.

I then tested using Pro Tools 9 using exact same parameters. CPU usage showed a little different than Live's in both PT's meter and Task Manager's meters, but got the first dropout at 31 tracks all the same. Totally identical. In fact, Live was more accurate in its metering than PT was. The percentage was exactly the same as Task Managers using Live, but was different when using PT. PT displayed the CPU usage as about 10-20% less than Task Manager showed, for whatever reason.

Rig:
Windows 7 64bit
i5 2500k OC 3.5ghz

Todd OMG: the synth setting you're using is the issue - it's just not comparable in terms of CPU usage to the ACE preset I used in my initial posts because it uses less CPU. You could run a million different tests with a million different plugins but you can't compare against my results unless you use the same preset.

If I use less-intensive presets or plugins in this test, then yes I can push beyond 24 tracks as well... but that doesn't prove anything at all IMO. The interesting thing about my test is that there is a strong correlation between the point where Live starts having show-stopping issues and the number of "logical CPUs" the machine has, and this reveals something key about how multicore support actually works in Live. The CPU meter screenshots I provided show only that Live starts to have big issues well before my machine's CPUs are actually anywhere close to being fully used - so again, it implies the issue is with Live, not my machine itself running out of power.

You might be able to get more instances out of other plugins and/or other presets, but that doesn't alter the underlying behaviour of Live. My particular test just happens to be good at making Live's behaviour apparent.

Just for the sake of replicating my results properly, here's what I've been using:

Ableton Project: https://dl.dropbox.com/u/7988988/multicore_ace.alp
ACE preset (in the native ACE preset format): https://dl.dropbox.com/u/7988988/multicore_test.h2p

All this talk about hyperthreading is also missing the mark IMO. CPUs have it for a reason, and modern OSes are designed to use it. My 12-core machine is viewed by the OS as having 24 "logical CPUs" (due to Hyperthreading), and in my tests Ableton works perfectly up 24 tracks... so how could disabling Hyperthreading (and therefore halving the number of "logical CPUs" visible to the OS) possibly help? I just don't think it has anything to do with Hyperthreading. Seems to me like Hyperthreading is what allows my machine to get 24 usable tracks in this test instead of 12.

fishmonkey
Posts: 4133
Joined: Wed Oct 24, 2007 4:50 am

Re: Multicore support needs a big overhaul

Post by fishmonkey » Tue Sep 18, 2012 2:43 am

Vance wrote: If I use less-intensive presets or plugins in this test, then yes I can push beyond 24 tracks as well... but that doesn't prove anything at all IMO. The interesting thing about my test is that there is a strong correlation between the point where Live starts having show-stopping issues and the number of "logical CPUs" the machine has, and this reveals something key about how multicore support actually works in Live. The CPU meter screenshots I provided show only that Live starts to have big issues well before my machine's CPUs are actually anywhere close to being fully used - so again, it implies the issue is with Live, not my machine itself running out of power.
yes, you've found a scenario that stresses your system in a particular way. it's not a magic number though. if you are really interested in this behaviour, why not learn about how computational parallelisation actually works?
Vance wrote:You might be able to get more instances out of other plugins and/or other presets, but that doesn't alter the underlying behaviour of Live. My particular test just happens to be good at making Live's behaviour apparent.
again, the behaviour is no great mystery.
Vance wrote:All this talk about hyperthreading is also missing the mark IMO. CPUs have it for a reason, and modern OSes are designed to use it. My 12-core machine is viewed by the OS as having 24 "logical CPUs" (due to Hyperthreading), and in my tests Ableton works perfectly up 24 tracks... so how could disabling Hyperthreading (and therefore halving the number of "logical CPUs" visible to the OS) possibly help? I just don't think it has anything to do with Hyperthreading. Seems to me like Hyperthreading is what allows my machine to get 24 usable tracks in this test instead of 12.
actually, it's quite possible that with hyperthreading turned off, you might actually end up with similar performance for this particular test.

Live will never ever be able to utilise CPU power in the way that other linear DAWs can. it's an unavoidable tradeoff that comes from the dynamic nature of Live, and there will always be a tradeoff between performance latency and CPU utilisation.

Post Reply