Rearranging sampled speech to form new sentences

[Krisser]

Hello

I wonder if anyone can help? I'm working on a project that requires me to take a three minute recorded speech, chop it into individual words and rearrange to create new sentences - if that makes sense.

I realise of course that Live can quite easily chop samples to midi. But there are a lot more words in my sample than there are midi channels. So I think this is going to be a task for a different piece of software.

Ideally, I'm on the hunt for something that can:

1) Automatically chop sentences into individual words with a fair degree of accuracy;
2) Allow me to label each audio snippet by the word it represents;
3) Allow me to drag and drop each snippit into a new order;
4) Mess around with time/pitch so that the new sentences sound reasonably natural;
5) And constantly audition the results

If this can all be achieved with Live then that would be great. If not, any suggestions on how I can get close to the above would be fantastic.

Thanks!

[andydes]

Trying to frame someone are we?

[simmerdown]

done it for phrases before...

put the full audio in a clip, put a warp marker at every word, then Slice to New Midi track, being sure to choose Warp Markers when it asks for the slice parameters (forget the actual term)..if you want it 'automatic' you can try your luck with 'transients', but it depends the rec, how close the words are...either way, you will have a drumrack with a shitload of hits, each one a word...might want to go thru and rename each pad accordingly, then draw your midi sentences

[pepezabala]

easy as pie:

1 put your recording into arrange. Make sure it's not warped.
2 split (ctrl-e) between words.
3 rename (ctrl-r) each piece.
4 then consolidate (ctrl-j) each piece.

You will find all the new sound-files neatly named in your project-folder (samples/processed/consolidated/...).

Now you can drop them all into sampler, or a drum-rack, or into an audio channel in session view or into arrange, whatever you want to.

[infernal.machine]

easy as pie:

1 put your recording into arrange. Make sure it's not warped.
2 split (ctrl-e) between words.
3 rename (ctrl-r) each piece.
4 then consolidate (ctrl-j) each piece.

You will find all the new sound-files neatly named in your project-folder (samples/processed/consolidated/...).

Now you can drop them all into sampler, or a drum-rack, or into an audio channel in session view or into arrange, whatever you want to.

Yeah this is the way to do it. Timing can be manipulated by changing the clips BPM and pitch can be altered with either transpose or detune (although you'll probably get better pitching results with a 3rd party plugin like melodyne).

[merges]

I think what's missing here is that OP asked about automatic slicing at words, and naming those slices automatically.

I know there is software out there that can do this (or at least guess reasonably well, as my phone does it when I dictate text messages to it), but I'm not sure I've ever encountered it in an accessible (purchasable, downloadable, usable by mere mortals) form… I've only seen it wrapped up in voice-recognition software.

[Krisser]

Thanks loads for the replies. But 'merges' is correct - what I'm really hoping is that someone will point me towards software that extends what happens when you ask google a question on an Android handset. Google understands what you ask (mostly) and it displays your words as a waveform.

If dictation software can recognise what you say by analysing a waveform and type it out, then it doesn't seem like too much of a leap to also be able to auto chop that waveform into individual, automatically named words.

Manually slicing several minutes of spoken audio into many hundreds of separate words before naming them, as kindly described by pepezabala, would be a royal PIA. I just find it hard to believe there isn't a way to automate this with at least a reasonable degree of accuracy - it's an area that would conceivably have many useful applications.

Anyone?

[infernal.machine]

If you were doing more than a few minutes of audio, it would make sense to spend time to find a program to do it, or even write your own. However what you are trying to accomplish will take a lot less time than you think if you use the methods described above by pepezabala, and your result will be a lot better too.

[regretfullySaid]

Yes, finding software like that would be great (could be out there), but by the time the question was posed to now, it could've been finished by hand.
PITA, sure, a little. But focusing and getting into a rhythm/zone, time will fly.
I've had to do it numerous times on soundforge back in the day for radio broadcasts.

[fishmonkey]

Yes, finding software like that would be great (could be out there), but by the time the question was posed to now, it could've been finished by hand.
PITA, sure, a little. But focusing and getting into a rhythm/zone, time will fly.
I've had to do it numerous times on soundforge back in the day for radio broadcasts.

whether or not doing by hand is feasible depends on how many times you need to do it. and chopping into phrases is a lot less work than chopping every single word.

[regretfullySaid]

Right, but if it's a one time thing....
And I didn't say anything about phrases, I was talking about words.

Google wavknife and see if it's still available. It was a free program where you could make some settings fir slicing audio up auomatically and also giving an option for auto-renaming.
It can't all be done in Live as listed in the OP.

Maybe try searching "vocal" or "speech" at sourceforge.

http://www.spacetaxi.de/sf/waveknife.html

[Krisser]

It's interesting I think that no mainstream solution exists. It isn't a one-off - I'm probably going to be doing a lot of this. It's for a sound installation project that I'm developing. So the idea of painstakingly chopping up hours of audio word by word without any automation makes me feel like having a lie down.

I checked out waveknife - thanks for the tip shadx312.. Unfortunately, it's windows and I'm mac. Also, it appears to work by simply detecting silences or audio below a certain volume. I just can't imagine it would be much good at dividing words in a normally spoken sentence, so I don't think I'll follow this one down the bootcamp road.

I've googled this in as many different ways as I can phrase the question, spoke to some software guys at music tech store, but to no avail. However, I wanted to ask if anyone has experience of Edison? I think its windows only again, but if anyone thinks it could be helpful it might be worth jumping the hurdles. Autonaming isn't so important as accurate word detection and splitting - and easy drag and dropping for changing word order.

Thanks again.

[fishmonkey]

there is actually quite a lot of research work into automatic speech recognition, but as you say, simply chopping based on transients or amplitude is very clumsy. the best algorithms are much more sophisticated than that.

computationally, it's a hard problem...

http://en.wikipedia.org/wiki/Speech_segmentation