live 3 hopes

Alex Reynolds · Post by **Alex Reynolds** » Sat Jul 05, 2003 9:54 pm

Anonymous wrote:Nope. A spinning disk in a laptop will consume far less current than a cpu clocking away like billy-o crunching numbers.

I think you're wrong, because, otherwise, why would -- to name a common example you may be familiar with -- an MP3 player like an iPod buffer data from the hard drive?

Simply put, constant hard drive access costs in time and power, compared with chip memory.

I'll show you another real-world example below.

Anonymous wrote:If part of the file(s) is/are pre-buffered then that gets around potential latency but that buffer(s) needs to be filled as fast as it/they are potentialy streamed and then what has already been decoded has now gone and will need to be decoded next time around, etc.

????

Sound samples in Live do not change (in general).

So if the decoded bits are in memory, if you're clever you shouldn't need to decode them again. This technique is called caching.

Disk arrays use fast RAM to cache frequently accessed data, to name another example:

-- http://www.sun.com/storage/highend/9980/

Though there the benefit is less for power consumption and more for speed and reliability.

It is a common technique and certainly high-end desktop computers that would be used with Live often have hundreds or thousands of megabytes of memory. Why not take advantage?

Anonymous wrote:I'm not arguing that decoding into ram wouldn't be more battery efficient than streaming from hard disk. However your point as stated was nothing to do with that...

Alex Reynolds wrote:MP3s also take up much less disk space, which would be nice from a storage library point of view and also in that Live reads a sample file over and over and again from the hard drive. A smaller file would mean less battery usage, less time required for Live to analyze the file, etc.

Erm, what about this is wrong, exactly?

1. MP3s take less disk space than AIFF files.
2. Latency for memory access is lower than latency for disk access.
3. Smaller files require less data transfer.
4. Sound files in Live do not generally change during a usage session of Live.

Decode-and-write-once-into-memory costs less than decode-repeatedly-and-read-repeatedly-from-disk.

If I'm wrong about the benefits from that approach, please show me where.

Cheers,
Alex

Guest · Post by **Guest** » Sat Jul 05, 2003 10:28 pm

Okay. Try following this....

Unless ALL, repeat ALL, of the clips/tracks to be played are in memory they are being only PARTLY buffered. With me so far? Okay...

So, we have just decoded and pre-buffered the beginning of 36 MP3 clips (we couldn't do ALL of the files because they would need too much memory uncompressed not to mention the time it would take to umcompress them ALL in the first place) and then we hit play... Okay, we are now playing and all of those buffers are rapidly emptying and needs to be filled with new data from those clips, and decoded, and now a little later the end of the buffer is reached, wraps round and overwrites data that is now NOT being played as we are only "buffering" here. Fine, on with the show except now the end of the long clip has been reached and needs to start again. Guess what?? It's not in memory as we are "buffering" and that buffer (one of many for each clip) ran out long ago on this occasion. So... back to decoding it again into that buffer.

If everyone has shedloads of ram and fast cpu's and all the samples they need will fit in that ram then all will be fine. But we don't, no this year. That's why Ableton went for te flexibility of streaming from disk which is far more pocket efficient than RAM.

Oh, as to your question as to why iPod's etc buffer. They buffer "part" of the encoded file as it goes so that there "shouldn't" be any glitches during streaming through such things as hard disk retries (due to a nudge, etc), access times, fragmentation, etc, etc. It's not all in memory after it's been decoded, trust me it isn't.

I could trot out the "I do this for a living" line here but I guess you'd have an answer for that as well being a Panther tester and "developer".#

Alex Reynolds · Post by **Alex Reynolds** » Sat Jul 05, 2003 10:45 pm

Anonymous wrote:Unless ALL, repeat ALL, of the clips/tracks to be played are in memory they are being only PARTLY buffered. With me so far? Okay...

So, we have just decoded and pre-buffered the beginning of 36 MP3 clips
(we couldn't do ALL of the files because they would need too much memory uncompressed not to mention the time it would take to umcompress them ALL in the first place)...

Good thing we're not using circa-1990 computers!

I doubt people use that many at once to begin with, but playing devil's advocate, we're not talking 36 4+ minute MP3 songs but in the context of Live, really 2-3 second clips, some mono percussion loops, etc.

Anonymous wrote:I could trot out the "I do this for a living" line here but I guess you'd have an answer for that as well being a Panther tester and "developer".

You've made it pretty clear on what level you're on in this discussion, don't distinguish yourself with more quotation marks, Mr "Anonymous"...

-Alex

Guest · Post by **Guest** » Mon Jul 07, 2003 8:14 pm

Okay. So we now have a lovely huge library of MP3 clips we want to use in a live situation, nice and compact (compressed even). We only want to use say a dozen few second clips as you say. Fine.

We loaded up some clips, MP3'd of course, and pressed play and are now performing. We'd now like to call up the odd clip from that huge library now and again, a few here a few there. There are 2 methods, and Live is now playing remember, 1) The cpu has to stop what it is doing and dart off to uncompress the clip into memory. This is obviously a non-starter, things will break pretty rapidly as cycles are stolen from plugins, engine, etc. Okay how about 2) The task of uncompressing clips into memory is interleaved with whatever else is going on at the time but not at such a high priority that any of that glitches. Fine, but memory is filling here with these new clips or we have to release some of the old non-used ones based on some scheme. It will also take more cycles, read more power, to uncompress those clips than via straight streaming not to mention those cycles get stolen from whatever else is going on.

At this moment it would be less of a potential problem on PC hardware but of course with the G5 coming along that will be suited as well. But that's a heck of a lot of bucks to play MP3 clips when other things that could be optimised to give better performance, PC & Mac, could be looked into because many people will still be having to use G4's etc for while yet. We don't all sit on pots of gold to buy new hardware to solve a relatively simple problem or two.

It's do-able but way down the list of what Live needs imo. In fact, buy a shitload of ram and setup a ramdisk and you can do what you want with a few second clips just in WAV/AIFF format now if it's that important they stay in memory!! RamDisk XP Pro does the trick on PC anyway if that's all that's needed.

If only those few clips are to be used and kept in memory, great no problem (MP3 licensing excluded) however it was on the premise of having a huge library of clips available on hard drive and they aren't going to fit in memory anytime in the near future. Also, if you can only use a set number ofclips at a time without upsetting the apple cart it makes no sense to have that huge library in the first place!!??

I'm sure you have it all worked out Alex. I look forward to you firing up your favourite compiler and showing us all how it's done.

Mr Anonymous

Alex Reynolds · Post by **Alex Reynolds** » Mon Jul 07, 2003 8:22 pm

Ever hear of multithreading? Ever play a DVD movie in the background or burn a CD while reading your email?

Save your insults. I've made my suggestion and I'm done with this thread.

Cheers,
Alex

Guest · Post by **Guest** » Tue Jul 08, 2003 1:21 am

Ever hear of multithreading?

I believe that option 2 covered that one.

stew · Post by **stew** » Tue Jul 08, 2003 10:11 am

This is going nowhere. The discussion wether or not playing samples from RAM will be less load to the battery is futile as all the modern operating systems use hard drive space for virtual memory - at the same time, caching files from the hard drive in RAM. In effect, your application has not too much control over hard drive activity. Note how iTunes on OS 9 has a "battery saving" option where the OS X version doesn't.

Guest · Post by **Guest** » Tue Jul 08, 2003 2:54 pm

Thank you.

Alex Reynolds · Post by **Alex Reynolds** » Tue Jul 08, 2003 3:16 pm

In general you're correct that applications grab pages from the memory manager and have no control over where those pages are coming from.

However, there are mechanisms for setting up a disk image stub that points to a RAM disk. How do I know this? Because I whipped out my compiler and wrote a utility to do this last year.

-Alex

Guest · Post by **Guest** » Tue Jul 08, 2003 5:16 pm

Resist! Resist! You must resist!!

Guest · Post by **Guest** » Tue Jul 08, 2003 6:18 pm

In an attempt to bring this back to the original topic, my only desire at the moment revolves around Live's Rewire slave ability:

Rewire2 usage to allow midi integration between the master app and live.

This way I can 'record' and edit what I've done through my controller to Live. Reason 2.5 uses the latest rewire, and it's an absolute dream to be able to use it in this fashion.

stevont · Post by **stevont** » Tue Jul 08, 2003 10:11 pm

THE SOLUTION:

I think Ableton should build a Rewire "Host" sequencer application, which would contain more standard midi & audio features. That way the programmers could stay more focused and the code could remain autonomous. It would also open up many more options for potential customers. Rewire technology has blown a huge hole in the "all in one box" mentality that bloats so many DAWs.

If you look at the feature set of the "classic" Cubase VST program for inspiration which Steinberg has abandoned for the sake of copying Digidesigns PT model (see Nuendo & Cubase SX) You will take over the universe.

s

stew · Post by **stew** » Thu Jul 10, 2003 6:25 pm

Alex Reynolds wrote:However, there are mechanisms for setting up a disk image stub that points to a RAM disk.

Yes, but who tells you that this RAM disk doesn't get paged out to disk in a low-memory condition?

Alex Reynolds · Post by **Alex Reynolds** » Thu Jul 10, 2003 7:06 pm

Disks are not paged out to other disks. I guess what you're asking is what if you assign a larger RAM disk than free memory. You can measure free system memory (kernel parameter) and make sure your RAM disk utility doesn't try to do this...

-Alex

Guest · Post by **Guest** » Thu Jul 10, 2003 7:45 pm

The OS will still be cacheing the disk reads/writes unless you specifically hook into the OS calls specifically for the ramdisk (best not do that really). So it has the potential at any time to be going through whatever disk(s) is/are being used for virtual memory. Mind you, the virtual memory/cacheing could always be set to RAM i guess although there may be a tiny flaw in the logic of that somewhere

live 3 hopes

The solution