So wait a second. What do you mean by "restart Live"? Do you mean when you stop and restart playback, or you completely close Live and relaunch it? If it's the latter, I wouldn't think it would be hardware related, but rather a bug in either Live or OS X.Pretty inconsistently, ... the CPU all the sudden spikes up to insane values (22.000% and such) the audio doesn't really crackle but basically slows to a crawl.
That problem only goes away again when I restart Live.
Have you looked at the system log messages in Console (Applications/Utilities/Console.app), especially while it's happening? There will be a ton of irrelevant stuff in there, but you might be able to see if OS X thinks something is fubar'd.
You might check the Disk tab on Activity Monitor while the problem is happening to see if Live (or something) is reading or writing crazy amounts of data. Compare the readings when it's working normally with those when it's messing up. (Look at the summary and graph at the bottom of the window.)
Also check the Memory tab on Activity Monitor. Look at the Memory Pressure graph at the bottom of the window to see if you're close to running out of memory. See if there's any amount reported in the "Swap Used:" summary at the bottom.
Also check the Network tab in Activity Monitor when it's messing up. See if one of the vast number of "helpful" online services that Apple just knows you'll love has suddenly decided to download a multi-gigabyte update at that time. (You DO have automatic updates turned OFF everywhere, don't you?)
Just for grins, have you tried turning off "Multicore/Multiprocessor Support" in Live? (Preferences -> CPU) Not sure what that would tell us if it makes a difference, but it would be interesting.