I have a 100Gig library of hand-picked EXS format multisamples on my desktop Mac's external HD from my Logic Pro era. I was going on a vacation, I had a Mac laptop to take with Live, I thought I'll use the opportunity to convert all 5,030 EXS files to Sampler during my idle time there. So I copied them to the laptop's HD preserving the file structure and successfully converted them there in about 4 days' work.
Back from my vacation, I copied the 5.030 converted Sampler files to the correct folder on the desktop Mac, called the first import up in Live, and ... "Media files are missing". Oh, @#$%. With the correct top sample folder pointed out, Live 8 has been auto-searching for the right files since 45 minutes (I know, that is a 100 Gigs folder with hundreds of organized subfolders, but still...) for just the first multisample with 5,029 more to go...
Where things derailed? I duplicated all the folder / HD names to the letter between the computer and the laptop. True, the HD was internal on the laptop and external on the computer but both had the same name. Yes, the usernames differ between the two Macs too, but the EXS files were kept in a public folder, not in any user level folder. What went wrong, and how do I (preferably batch-) fix it without spending a kajillion hours? I could use some insights...
Relinking samples for 5,030 imported Sampler multisamples?
Re: Relinking samples for 5,030 imported Sampler multisamples?
You went wrong in assuming that Live remembers file paths outside of its own library.
Live uses a checksum system so it can tell files apart even when they have the same name.
So if you create presets outside of the library with sound files from outside the library, it will only work on the computer you made them on.
Once you fixed your files with the library manager, its best to either keep them in your Live library or throw them all inside a project and do a collect all and save, so that it works on other computers too.
Live Auto-Search is a bit of a pain. You'll achieve faster results by manually locating a missing file and then telling live to look near the replaced samples.
If you make it scan 100GB, that is exactly what it will do... Scan 100GB.
Live uses a checksum system so it can tell files apart even when they have the same name.
So if you create presets outside of the library with sound files from outside the library, it will only work on the computer you made them on.
Once you fixed your files with the library manager, its best to either keep them in your Live library or throw them all inside a project and do a collect all and save, so that it works on other computers too.
Live Auto-Search is a bit of a pain. You'll achieve faster results by manually locating a missing file and then telling live to look near the replaced samples.
If you make it scan 100GB, that is exactly what it will do... Scan 100GB.
Re: Relinking samples for 5,030 imported Sampler multisamples?
I went by the EXS school of logic; in ESX the the location of individual samples is kept as an ASCII string inside the multisample file. So a relinking problem of this sort would be simply remedied by a batch text-string-rename within the 5,030 files.
Live's Auto-search just finished relinking multisample #1 in 70 minutes. So Live has no "now that you relinked multisample #1, relink the other 5029 following the same file path offset" routine or plugin or something?
Hmm. I already gave the laptop away, it's a photo editing machine in Asia now; not much of musical anything remains on it. Does a program exist that could batch convert the 5,030 EXS files to Sampler on autopilot without me having to do each single one manually again? It wasn't that much of fun. Or can these converted import files somehow be helped without spending 5,030 hours on auto-search? (That's 30 weeks... to relink 4 days worth of work?)
Live's Auto-search just finished relinking multisample #1 in 70 minutes. So Live has no "now that you relinked multisample #1, relink the other 5029 following the same file path offset" routine or plugin or something?
Hmm. I already gave the laptop away, it's a photo editing machine in Asia now; not much of musical anything remains on it. Does a program exist that could batch convert the 5,030 EXS files to Sampler on autopilot without me having to do each single one manually again? It wasn't that much of fun. Or can these converted import files somehow be helped without spending 5,030 hours on auto-search? (That's 30 weeks... to relink 4 days worth of work?)
Re: Relinking samples for 5,030 imported Sampler multisamples?
- Locate one of the missing files using Live's browser
- Drag it to the corresponding slot in the Missing Samples list
Live will now ask if you want to search near the sample you just added. Choose Yes.
You'll find that the files are located significantly faster.
- Drag it to the corresponding slot in the Missing Samples list
Live will now ask if you want to search near the sample you just added. Choose Yes.
You'll find that the files are located significantly faster.
Re: Relinking samples for 5,030 imported Sampler multisamples?
Hmm... while I typed this. the Library window came up on its own. It says, 84319 of the referenced files (such as samples) are missing. Maybe this program does have higher smarts than it lets us on at first...
I'll click the Go button next to "Search" and leave the Mac on for a few days; let's see what happens. I have 2 new CPUs coming in anyway.
(2 more reasons to make converted multisamples easy to sling around between CPUs...
I'll click the Go button next to "Search" and leave the Mac on for a few days; let's see what happens. I have 2 new CPUs coming in anyway.
(2 more reasons to make converted multisamples easy to sling around between CPUs...
Re: Relinking samples for 5,030 imported Sampler multisamples?
An hour later...
"Ableton Live
A serious program error has occurred.
Live will shut down after this message box is closed.
Out of memory
A crash log was dumped into: /Users/mrdelurk/Library/Preferences/Ableton/Live 8..."
Kinda familiar. When I drop a whole folder of EXS files onto Sampler to convert, Live usually dies after the first few hundred. That's why I had to do the 5,030 by hand.
"Ableton Live
A serious program error has occurred.
Live will shut down after this message box is closed.
Out of memory
A crash log was dumped into: /Users/mrdelurk/Library/Preferences/Ableton/Live 8..."
Kinda familiar. When I drop a whole folder of EXS files onto Sampler to convert, Live usually dies after the first few hundred. That's why I had to do the 5,030 by hand.
Re: Relinking samples for 5,030 imported Sampler multisamples?
After sleeping on the problem, I found a solution for the future. (Though this still doesn't solve the problem I already have)
On all new computers, I'll install only one (1) program on the boot OS. A virtualization app like VMware. Then, using the same OS I'll create a virtual machine (VM). I'll install all the music apps like Ableton and its plugins, and presets and all other files to this virtual machine instead of the boot OS, since virtualization apps store the VM to a giant disk file. Which can be copied, moved to a different computer, and upon launch the whole environment comes up as it was.
A couple of major benefits spring to my mind:
- no more CPU migration hassles, with new computers. Throw the VM file on the new CPU, and you're in business with all the work environment intact without having to re-install, re-register, etc. Heck, with virtualization apps like VMWare available cross platform, you can even migrate from Windows to Mac or vice versa.
- want to continue working on a laptop during a vacation? Throw the VM file on it, and everything comes up there, ready to go.
- no more Net malware worries. Create a separate virtual machine for just Net browsing tasks, store an extra backup copy at default state in a folder... if some malware infects the VM, it stays inside that VM; just trash that VM and replace it with a fresh backup copy.
I ran Live in VM already, although it was always a Windows VM on a Mac CPU. As soon as my new Windows 7 CPUs arrive, I'll try Tarekith's Live performance test how many CPU % points difference it makes to run Live on boot Windows 7 vs VM Windows 7 on the same machine. My hunch is, probably not many.
On all new computers, I'll install only one (1) program on the boot OS. A virtualization app like VMware. Then, using the same OS I'll create a virtual machine (VM). I'll install all the music apps like Ableton and its plugins, and presets and all other files to this virtual machine instead of the boot OS, since virtualization apps store the VM to a giant disk file. Which can be copied, moved to a different computer, and upon launch the whole environment comes up as it was.
A couple of major benefits spring to my mind:
- no more CPU migration hassles, with new computers. Throw the VM file on the new CPU, and you're in business with all the work environment intact without having to re-install, re-register, etc. Heck, with virtualization apps like VMWare available cross platform, you can even migrate from Windows to Mac or vice versa.
- want to continue working on a laptop during a vacation? Throw the VM file on it, and everything comes up there, ready to go.
- no more Net malware worries. Create a separate virtual machine for just Net browsing tasks, store an extra backup copy at default state in a folder... if some malware infects the VM, it stays inside that VM; just trash that VM and replace it with a fresh backup copy.
I ran Live in VM already, although it was always a Windows VM on a Mac CPU. As soon as my new Windows 7 CPUs arrive, I'll try Tarekith's Live performance test how many CPU % points difference it makes to run Live on boot Windows 7 vs VM Windows 7 on the same machine. My hunch is, probably not many.
Re: Relinking samples for 5,030 imported Sampler multisamples?
It's done... the CPU% difference between running Ableton Live on a host OS vs. in a virtual machine is 10% vs. 13%.
To put this into perspective, I shaved more off with overclocking. (4%)
Story with screen shots at http://www.southseascave.com/?p=41
To put this into perspective, I shaved more off with overclocking. (4%)
Story with screen shots at http://www.southseascave.com/?p=41
Re: Relinking samples for 5,030 imported Sampler multisamples?
Very interesting approach. I would not have thought a VM would be so fast.
I lost many months of my life to that abysmal librarian. I doubt if I will ever make a big project using Sampler ever again. But I only had 2772 samples in that project, so you got me beat. However, I had to do it several times due to name changes, etc.
I lost many months of my life to that abysmal librarian. I doubt if I will ever make a big project using Sampler ever again. But I only had 2772 samples in that project, so you got me beat. However, I had to do it several times due to name changes, etc.
Re: Relinking samples for 5,030 imported Sampler multisamples?
If there is one bit of live that needs to utterly destroyed and rewritten - its that horrendous library search system and the way live struggles so badly with relinking samples.
NI samplers (battery, kontakt) usually manage to find and load moved sample in second when live can take several hours to scan the same file structure.
From what I remember from trying to fix a broken project file in a binary editor once, live does store file paths (and therefore file names) as well as crcs, so I dont understand why it has been implemented so badly.
Actually - someone start a petition to have this sorted out ASAP - I can't imagine it's that much effort to rewrite what should be a fairly small and self contained bit of code.
NI samplers (battery, kontakt) usually manage to find and load moved sample in second when live can take several hours to scan the same file structure.
From what I remember from trying to fix a broken project file in a binary editor once, live does store file paths (and therefore file names) as well as crcs, so I dont understand why it has been implemented so badly.
Actually - someone start a petition to have this sorted out ASAP - I can't imagine it's that much effort to rewrite what should be a fairly small and self contained bit of code.
Nothing to see here - move along!
Re: Relinking samples for 5,030 imported Sampler multisamples?
Relinking them again by hand, now... just finished the Bass folder.
I took a shortcut, I pre-converted 5017 to .sf2 format with CDxtract first. There were 13 (the "dirty 13"... mostly Garageband ones) that would cause CDxtract to crash, I attached those holdouts to a CDxtract bug report.
I also filed a bug report with Chickensys about Translator Pro (which I also bought). The app would just quit instead of a batch conversion.
Amazing, huh. 3 whizbang tools (total cost $1200) for one EXS to Ableton task, neither works. (Still finishing the relinking by hand...) I feel like a government procurement office.
I took a shortcut, I pre-converted 5017 to .sf2 format with CDxtract first. There were 13 (the "dirty 13"... mostly Garageband ones) that would cause CDxtract to crash, I attached those holdouts to a CDxtract bug report.
I also filed a bug report with Chickensys about Translator Pro (which I also bought). The app would just quit instead of a batch conversion.
Amazing, huh. 3 whizbang tools (total cost $1200) for one EXS to Ableton task, neither works. (Still finishing the relinking by hand...) I feel like a government procurement office.
Re: Relinking samples for 5,030 imported Sampler multisamples?
Guess what.
Having hand-converted the first 1000 .sf samples to .adv inside the virtual machine, (and checked in Ableton if they work there - yup) I decided to check if those .adv files can be moved indeed. (Future-proofing is the name of the game.) So I went into the Windows 7 virtual machine, selected the entire "Imports / Bass" folder, copied it and pasted it in the corresponding place in my host Windows 7. Fired up Ableton 8 on the host Windows, double clicked on of the files, and... "Media files are missing. Click here for more information."
This is so totally, hopelessly, absolutely FUBAR... turns out, after converting a self-contained multisample to .adv (a ZIPped text file, really) Ableton jettisons the actual samples out of it, into Library / Samples / Imported. This is a system conceived for a small-timer user, who might convert maybe 6 multis in his life; he won't care if he must copy 2 folders with a handful of files in each instead of one. But if you have 5000 .adv's in an elaborate system of subdirectories, there is no way to manage them. (Unless keeping 5000 files in a single folder is your idea of management.)
I'm having serious thoughts if I should switch to composing on a Yamaha XS8. After thousands of hours of trying everything on earth, even virtualizing, I still cannot make Sampler work well for me. I used Kontakt before, NI's copy protection niggles got on my bad side, so that's out... looks like no matter how much I dislike the prospect, I got a major decision to make.
Having hand-converted the first 1000 .sf samples to .adv inside the virtual machine, (and checked in Ableton if they work there - yup) I decided to check if those .adv files can be moved indeed. (Future-proofing is the name of the game.) So I went into the Windows 7 virtual machine, selected the entire "Imports / Bass" folder, copied it and pasted it in the corresponding place in my host Windows 7. Fired up Ableton 8 on the host Windows, double clicked on of the files, and... "Media files are missing. Click here for more information."
This is so totally, hopelessly, absolutely FUBAR... turns out, after converting a self-contained multisample to .adv (a ZIPped text file, really) Ableton jettisons the actual samples out of it, into Library / Samples / Imported. This is a system conceived for a small-timer user, who might convert maybe 6 multis in his life; he won't care if he must copy 2 folders with a handful of files in each instead of one. But if you have 5000 .adv's in an elaborate system of subdirectories, there is no way to manage them. (Unless keeping 5000 files in a single folder is your idea of management.)
I'm having serious thoughts if I should switch to composing on a Yamaha XS8. After thousands of hours of trying everything on earth, even virtualizing, I still cannot make Sampler work well for me. I used Kontakt before, NI's copy protection niggles got on my bad side, so that's out... looks like no matter how much I dislike the prospect, I got a major decision to make.
Re: Relinking samples for 5,030 imported Sampler multisamples?
Just to make sure, have you tried my suggestion?
To reiterate:
- relink a sample manually by dragging it on top of a missing sample
- Live should ask to look for samples near the one you just replaced, select yes
In my case this usually finds a whole bunch of candidates. With hundreds of samples missing, I usually don't have to relink more than 10.
Once everything is linked, there are only 2 ways to future proof these presets.
Collect them in a project and save all the samples with it or keep them in your library along with the samples.
If you want to load the presets on a different computer, you either need the project or your ENTIRE library.
Ever wondered why a self contained Live project looks identically structured to the Live library? Its because a project is a mini-library. And both these libraries have database files that keep samples and presets linked. Without these files, you get the mess you're in.
An easy way to archive your sample sets is to make Live Packs.
To reiterate:
- relink a sample manually by dragging it on top of a missing sample
- Live should ask to look for samples near the one you just replaced, select yes
In my case this usually finds a whole bunch of candidates. With hundreds of samples missing, I usually don't have to relink more than 10.
Once everything is linked, there are only 2 ways to future proof these presets.
Collect them in a project and save all the samples with it or keep them in your library along with the samples.
If you want to load the presets on a different computer, you either need the project or your ENTIRE library.
Ever wondered why a self contained Live project looks identically structured to the Live library? Its because a project is a mini-library. And both these libraries have database files that keep samples and presets linked. Without these files, you get the mess you're in.
An easy way to archive your sample sets is to make Live Packs.
Re: Relinking samples for 5,030 imported Sampler multisamples?
Hello Hoffman2k,
When the computer is left on to relink 10,000's of samples, after an hour, it says:
"Ableton Live
A serious program error has occurred.
Live will shut down after this message box is closed.
Out of memory"
The relinking routine may work fine at a 100's file count, but at 10,000's it's a tad floor-attracting.
I tried your Live Pack creation suggestion. The total size of my samples is 100 Gigs, so I picked just the 1GB Bass folder to try. I clicked it in Live's Browser - Selected File/Manage Files - Clicked Manage Library button - waited 1.5 Hours for scanning just a 6GB Library - Got a Contents of the Library window, with no simple way to select the Bass Folder. OK, Plan B. Clicked the Bass folder again - Selected Manage Files - Clicked Manage Project - Got a Project Temp-1 files window, with no simple way to select the Bass Folder. Under Packing, the help file said, use External Samples section of the File Manager to collect external samples into the project. Except there is no such thing. There is an External Files, which says all the files are contained. So I clicked, Create Live Pack, gave it a name - and got a 4KB file. I know size doesn't matter, but still...
Perhaps I need to add the Bass folder to some sort of dummy Ableton project to be able to package it. But dragging the folder around a blank Live project Session window only produces a "nope" symbol. The nice thing about the PDF manual is, a search on "create a Live Pack" returns a negative result in 1 second. Here is true Artificial Intelligence: a computer that tells me to use something else.
When the computer is left on to relink 10,000's of samples, after an hour, it says:
"Ableton Live
A serious program error has occurred.
Live will shut down after this message box is closed.
Out of memory"
The relinking routine may work fine at a 100's file count, but at 10,000's it's a tad floor-attracting.
I tried your Live Pack creation suggestion. The total size of my samples is 100 Gigs, so I picked just the 1GB Bass folder to try. I clicked it in Live's Browser - Selected File/Manage Files - Clicked Manage Library button - waited 1.5 Hours for scanning just a 6GB Library - Got a Contents of the Library window, with no simple way to select the Bass Folder. OK, Plan B. Clicked the Bass folder again - Selected Manage Files - Clicked Manage Project - Got a Project Temp-1 files window, with no simple way to select the Bass Folder. Under Packing, the help file said, use External Samples section of the File Manager to collect external samples into the project. Except there is no such thing. There is an External Files, which says all the files are contained. So I clicked, Create Live Pack, gave it a name - and got a 4KB file. I know size doesn't matter, but still...
Perhaps I need to add the Bass folder to some sort of dummy Ableton project to be able to package it. But dragging the folder around a blank Live project Session window only produces a "nope" symbol. The nice thing about the PDF manual is, a search on "create a Live Pack" returns a negative result in 1 second. Here is true Artificial Intelligence: a computer that tells me to use something else.
Re: Relinking samples for 5,030 imported Sampler multisamples?
Fair enough, the biggest packs I've ever made were around 3GB.
I'm not surprised it takes ages to deal with massive libraries like that, just look how long Live needs to install a 1GB library.
I've never experienced those errors before, but that might be a RAM or a platform related thing.
Live just isn't made to deal with huge collections of samples linked to presets being processed all at once.
You'll probably have better results with putting each imported preset in its own Live Pack.
The browser is one of Live's weakest links and doing file management with it on the scale you are attempting is going to be a rough challenge.
I'm not surprised it takes ages to deal with massive libraries like that, just look how long Live needs to install a 1GB library.
I've never experienced those errors before, but that might be a RAM or a platform related thing.
Live just isn't made to deal with huge collections of samples linked to presets being processed all at once.
You'll probably have better results with putting each imported preset in its own Live Pack.
The browser is one of Live's weakest links and doing file management with it on the scale you are attempting is going to be a rough challenge.