Multicore support needs a big overhaul

[steko]

I do know that Live's CPU-meter & the OS CPU-meter are not comparable: http://forum.ableton.com/viewtopic.php?f=42&t=159990


On my 2008 8-core (no virtual cores/hyperthreads!) MacPro I get (quite proportional):

8 tracks u-he ACE = 370% on OS X's CPU-meter & 47% on Live's CPU-meter

16 tracks u-he ACE = 715% on OS X's CPU-meter & 91% on Live's CPU-meter

adding a 17th track = 330% on OS X's CPU-meter & 130% on Live's CPU-meter = crackle hell

I guess the lower OS X CPU is a result of effectively less processing taking place because Live gets a hiccup…


Oddly with Logic (which actually is way less of a CPU hog than Live!) I get about the same results (with u-he ACE Au-version) up to 14th track - then get system overload message…


I even downloaded Reaper & at first opening managed to create tracks, insert u-he ACE & 1-bar midi-clips - but then the fucker crashed & all was gone...


What plug-ins buffer size have you set in Live's CPU preferences, Vance? I have "As Audio Buffer".

What version of Live?

Edit:

Got it workin':

Reaper 8 tracks 370% on OS X's CPU-meter

Reaper 16 tracks 715% on OS X's CPU-meter

Reaper 18 tracks around 800% on OS X's CPU-meter

Reaper 19 tracks around 800% on OS X's CPU-meter = crackle!

[pencilrocket]

Please read my post again, and you'll learn that I was criticizing people who were questioning the results of the test, not the original tester. You should probably brush up on your English before getting into arguments in the language.

Okay sorry about that. My bad.

[ze2be]

Very nice steko, thanks for all the info!
So, it seems on 8 cores it aint that much of a difference.
Maybe there is something going on with the 12 core systems?

[masterblasterofdisaster]

Vance, try disabling hyperthreading.

From a shell: sudo nvram SMT=0

to enable hyperthreading: sudo nvram -d SMT

Reboot for changes to take effect.

[Vance]

steko - thanks for trying out the same test. If it's OK can I give a few pointers to try and get a more valid comparison between our results?

- Yep, plugin buffers were set to "As Audio Buffer" on my setup (512 samples).

- CPU meter readouts are largely irrelevant here... the essence of this test is: how many tracks can you get before it all grinds to a catastrophic halt? On my system, 24 tracks hums along with no issues, but the 25th track has so many dropouts it's unusable. Is this also the case with you?

- The significance of the CPU meter pictures is that the computer clearly has lots of processing headroom at the point where Live starts to produce massive dropouts. As a secondary point, it also shows that Reaper uses more of the overall power of the machine before it starts to produce dropouts. Anything else is irrelevant - comparing Ableton's meter readout to OS X's meter readout is not of any significance at all.

- Did you set "Stack" to 8 and Quality to "Accurate" on your system as well?

- Were things perfectly smooth on the 16th track before you added the 17th? The behaviour I seem to be experiencing is that everything goes well until you hit a hard limit, when suddenly things get horrible. Conversely with Reaper, not only can you use way more tracks before problems start happening, but the occurrence of dropouts ramps up gradually rather than slamming you all of a sudden as it does in Live.

- Can you please also try and see if the results are the same using the internal output of the Mac Pro? The purpose of this is to eliminate all the variables associated with your audio interface (i.e. its particular brand, drivers, the usual issues related to the firewire bus, etc.)... if I do this test using the internal output on my own Mac Pro, I get the exact same results as when I use my normal Firewire audio interface, although the dropouts aren't quite as catastrophically bad when I play that 25th track (but I'd still call the dropouts so bad as to be unusable).

[Vance]

Vance, try disabling hyperthreading.

From a shell: sudo nvram SMT=0

to enable hyperthreading: sudo nvram -d SMT

Reboot for changes to take effect.

I am reluctant to do this for a couple of reasons:

a) Nowhere have I read of any software company that recommends doing this as a way of increasing performance, nor have I read anywhere that Ableton recommends it for best performance.

b) My gut feeling/prediction is that, considering Live's multicore implementation as it currently stands, it would actual decrease performance. $20 says if I disabled it, my 24 core limit would suddenly become 12.

c) It seems that this is an imperfect thing at best - it seems some things benefit from it and others don't. There's no indication that Live would benefit from it at all (at least no indication that I'm aware of - has anyone tried this before?)

d) We really shouldn't have to resort to hackery like this. Hyperthreading exists for a reason, it seems like the general consensus is that it benefits performance.

e) There's also no indication that this issue I'm describing is even caused by Hyperthreading in the first place.

[masterblasterofdisaster]

Vance, try disabling hyperthreading.

From a shell: sudo nvram SMT=0

to enable hyperthreading: sudo nvram -d SMT

Reboot for changes to take effect.

I am reluctant to do this for a couple of reasons:

a) Nowhere have I read of any software company that recommends doing this as a way of increasing performance, nor have I read anywhere that Ableton recommends it for best performance.

b) My gut feeling/prediction is that, considering Live's multicore implementation as it currently stands, it would actual decrease performance.

c) It seems that this is an imperfect thing at best - it seems some things benefit from it and others don't.

A Google search along the lines of "hyperthreading audio glitches" might convince you otherwise.

Anyway - maybe, maybe not. I'll keep poking around. Interesting problem.

[Vance]

A Google search along the lines of "hyperthreading audio glitches" might convince you otherwise.

10 seconds of Googling has given me:

- A thread on the Cubase forum which says that it was recommended to turn HT off but it made no difference
- A Sound On Sound article from 2006
- An overview/puff piece on multicore processing as an overal concept from Sound On Sound dated 2008

I'm yet to be convinced.

[masterblasterofdisaster]

A Google search along the lines of "hyperthreading audio glitches" might convince you otherwise.

10 seconds of Googling has given me:

- A thread on the Cubase forum which says that it was recommended to turn HT off but it made no difference
- A Sound On Sound article from 2006
- An overview/puff piece on multicore processing as an overal concept from Sound On Sound dated 2008

I'm yet to be convinced.

10 entire seconds, eh?

[Vance]

Yes, 10 of your Earth "seconds"

In all seriousness though, I believe the whole "disable Hyperthreading" thing is a relic of a time when DAWs weren't multicore-enabled whatsoever, and operating systems weren't much better... I don't honestly think it has much relevance to modern machines with modern OSes.

I could be wrong but I can't see anything convincing to the contrary right now.

[masterblasterofdisaster]

Yes, 10 of your Earth "seconds"

In all seriousness though, I believe the whole "disable Hyperthreading" thing is a relic of a time when DAWs weren't multicore-enabled whatsoever, and operating systems weren't much better... I don't honestly think it has much relevance to modern machines with modern OSes.

I could be wrong but I can't see anything convincing to the contrary right now.

lol. Yeah, I hear you.

Nonetheless, I just ran a test to satisfy my curiosity.
I'm on a 2.3Ghz Intel Core i7 macbook pro, 8 gigs of ram, MOTU Ultralight audio interface, OS X 10.6.8, Abelton Suite 8.2.6

Setup 16 instances of u-he ACE, playing a proper 4 note chord held sustained.

I then deleted instances until it was glitch free at 8 instances.

Next, I turned off hyper-threading and repeated the test and found that I could run 14 instances glitch free.

In the interests of getting this sorted, I shall remain skeptical and give it another shot in the morning.

Edit: ran one more test making sure I had a "freshie" for the hyper-threaded test. Did a reboot and gave it another shot. Performance was slightly less catastrophically crappy, but still glitched all the way down to 9 instances of u-he ACE. Again, 8 instances worked fine.

[Earwax69]

Paying 6500$ for a 12-cores+24gb of ram to use Live is, like you so nicely said, ri-god-damn-diculous.

For the OP, yeah, I kinda fast read through it and you are right. Reaper obviously win. Pretty good for a 60$ soft.

[steko]

First some more general info when we're already at it: http://forum.ableton.com/viewtopic.php? ... 9#p1345049


From my results, for you Vance, one should expect a track count of around 55 in Live.

Would be nice, if others with machines comparable to Vance's would do the test too! So we can see clearer.

Would be nice too, if you Vance could contact Ableton & ask about hyperthreading.

"Maybe there is something going on with the 12 core systems?

Possible, but not neccesarily...

the essence of this test is: how many tracks can you get before it all grinds to a catastrophic halt? On my system, 24 tracks hums along with no issues, but the 25th track has so many dropouts it's unusable. Is this also the case with you?

Yes

- Did you set "Stack" to 8 and Quality to "Accurate" on your system as well?

Yes. Had to set stack to 8, quality was already set to "accurate" in the default preset.

- Were things perfectly smooth on the 16th track before you added the 17th?

Yes (as much as I can tell from that default sound...)

- Can you please also try and see if the results are the same using the internal output of the Mac Pro?

Same results



Well, the thing could hint to:

A: Live has a hyperthreading problem. (in Logic 9.1.2 hyperthreading got disabled completely & then after a massive howling started in the forums they made a "Processing Threads setting" in 9.1.3:

http://support.apple.com/kb/HT4426 - http://support.apple.com/kb/TS3561

There are numerous factors which influence how the OS distributes the workload to the cores on your Mac. The optimal value for the Processing Threads may vary not only for different configurations but for different types of Projects. Selecting the highest value may not always be the best choice. You will need to experiment to determine which setting provides the best balance of performance and reliability with your setup and particular Logic Project.

http://www.gearslutz.com/board/music-co ... ktest.html

https://discussions.apple.com/thread/26 ... 7&tstart=0

http://www.macosxaudio.com/forums/viewt ... 11#p372411


B: (Some of) your Live preferences, Application Support etc. files could be corrupt.

Ever checked system.log in Console (Applications/Utilities) when doing the u-he ACE test in Live?)

You could try to reset Live completely:

Note: this will erase your template set (if you have one), audio and MIDI hardware needs to be setup again in Live's preferences. Also plug-in support needs to be switched on again.

Do the following:

- close Live

- delete the "Preferences.cfg", "Template.als" and "Undo.cfg" files (if available) from: /Users/username/Library/Preferences/Ableton/Live 8.x.x

- delete the content of these two folders:

/Users/username/Library/Caches/Ableton/Cache/Web

and

/Users/username/Library/Application Support/Ableton/Live 8.x.x/Database

- restart Live

[Vance]

- Thanks for the update steko

- I just the test on my Macbook Pro. It's a dual-core Mid-2009 machine, with hyperthreading. Basically, I could predict from my original results that I would be able to run 4 tracks seamlessly, with massive issues on the 5th track.

- This is exactly what happened - 4 tracks was seamless, the 5th track brought it to a screeching halt.

- I tried disabling hyperthreading by using the "sudo nvram SMT=0" command in Terminal. After rebooting, it made no difference to the test - 4 tracks was fine, 5 tracks was horrible. But I believe that's because the command actually didn't disable hyperthreading properly. As a dual-core CPU, with HT disabled, OS X should only be able to see the 2 physical cores. The OS X CPU meter showed the usual 4 cores, and during the test, OS X showed activity on all 4 of those cores.

- So it seems that that command isn't properly disabling hyperthreading, at least on my MBP. I'm reluctant to go too deep on testing that solution on my Mac Pro because I have to stay productive & can't afford downtime on that machine. I'll do my tinkering on the MBP (now that I've established that my test scales appropriately) and once I see something change/improve on the MBP I'll give it a go on the Mac Pro.

- I'll check Console but I can't see it making any difference - Live isn't crashing, it's just producing big dropouts... doesn't seem to be the kind of thing that would get system logs involved.

- Perhaps there's a Live-specific setting to disable hyperthreading, like there is now in Logic? A flag to "disable virtual cores" or something along those lines where you can set a limit to the number of threads Live can see? Perhaps an options.txt flag might be created for us to test? http://www.ableton.com/pages/faq/options_text

[Vance]

And for comparison with Reaper, my MBP can run 16 tracks without issue. The 17th track starts to create some dropouts but they're not catastrophic. Beyond that, more tracks makes for more dropouts, but as with the Mac Pro test it ramps up gradually rather than becoming horrible all of a sudden like it does with Live.

So in summary for my MBP:

- Live: 4 tracks = smooth... 5 tracks = unusable
- Reaper: 16 tracks = smooth... 17 tracks = dropouts start to kick in but not really unusable... 18+ tracks = dropouts ramp up.