MIDI delay recording

[Amaury]

I'm just requoting this post from earlier in the thread, because it phrases my thoughts better than I can express.

Amaury

What Ableton have concluded is that a human playing a synth with 30ms latency will automatically compensate for this latency by playing consistently 30ms early. They assume that the human is capable of latency compensation. Anyone who has ever tried this will know how impossible this is.

Actually, this isn't true at all -- piano players do it all the time. I agree that many musicians who are not piano players may not have this training (though my main training is in piano so I don't really know how hard it is to learn for e.g. a snare player). I ran across this interesting article (google cache of a pdf) while trying to get the figures for mechanical latency of a piano. It seems that there are even experimental studies on it: "The most important aspect of this is the fact that we can subconsciously adjust our performance to compensate for such different feedback conditions. During experiments with delayed feedback, subjects clearly altered their behavior according to the characteristics of each trial, forcing the researchers to introduce control trials between each pair of trials (Aschersleben and Prinz 1997; Mates and Aschersleben 2000). In piano performance, the time elapsed between pressing a key and the corresponding note onset is around 100ms for piano notes and around 30ms for staccato, forte notes (Askenfelt and Jansson 1990). Even if we assume that the pianist expects the note onset to happen somewhere in the middle of the course of the key, it is very likely that latencies will be different for different dynamic levels. Still, pianists have no problem dealing with such different latencies; since voices in pieces for the piano usually have dynamics that change continuously, the performer has the opportunity to adjust himself to the corresponding changes in latency."

[popslut]

My Roland MKS 70 module has an internal latency of between 10 and 15 ms and that is perfectly playable. I also have a Kurzweil K2000 and I've measured that at 10ms - again, not a problem.

because you play it between 10 and 15 ms ahead of time, and are able to play well in rhythm (the sound is in sync) with you other pre-recorded instruments

Absolutely not. I am first and foremost a drummer. I cannot help but hit the keys in time with the rhythm. I couldn't play "15ms ahead of time" if I wanted to.

As I said;

If I record me playing my MKS 70 as midi into Cubase with no quantisation and I look at the midi notes I see that I actually played "in the pocket" and the notes more or less line up with the neat grid.

Which means, yes, my MKS 70 sounds late. To correct this I advance the midi track, post recording, to compensate.

If we are talking of external synth, the problem is different.

No it isn't. Live treats external synths and VSTi's exactly the same with regard to monitoring/latency compensation.

I could happily substitute the words "MKS 70" for "Pro 53" in the above example and the observation would still be true.

The point is, for those of us whose hands play in time, the current situation regarding latency compensation in Ableton Live is unworkable.

For those whose hands play early [!!!] it works perfectly.

Again, it is nearly impossible to deal with too high latencies for any musician. As I said earlier on the thread, it may be ok up to 15 ms, but above, it isn’t.

We know this. We do it every day. I do it for my living. I have been doing it since I was 13 years old.

And it wouldn’t be if you would have to play the notes ‘on time’, means of the grid, and hear the sound 25 or 50 ms later. This is as impossible if not worst as playing 50 ms early.

Again, this is obvious.

High latency = impossible to play. But this is the case whichever way you try to cope; whether you play early or hear late, it's still unsatisfactory. I'm not sure which point you are making here.

I know pianists, drummers and the like are very much able to adapt to a small latency in order to hear the sound on time.

I've been doing this professionally, day in, day out, for twenty years and I have never met a drummer who would comfortably play ahead of the other musicians.

The example I would give is that professional tape machines have two playback heads - sync and repro. The sync head is positioned adjacent to the record head - the repro head an inch away. Whilst recording, one would monitor the tracks being recorded and those being reproduced from the sync head because monitoring from the repro head would introduce latency [the time taken for the tape to carry the recorded signal from the record head to the repro head]. This delay is measurable in milliseconds.

Occasionally I would forget to place the tape machine in "sync" mode and would enter record whilst monitoring from the "repro" head - thus introducing a latency similar to recording through a DAW with the buffers set to 512 samples.

Whenever this happened, the musician[s] being recorded would notice instantly and stop playing. They wouldn't "adapt" and start trying to play ahead to compensate. They just stopped. It felt wrong.

If your theory were correct, they would simply adapt to the latency and play ahead, maintaining a silky groove and turning in fabulous performances.

That never happened. Not once.


Now the tape machines have all been wheeled away to the dumpster [much to my regret] and we all know the situation regarding DAW latency. It is the payback clause for all this random access, auto-tuning, audio quantising, non-destructive editing magic we can now do.

But - musicians are still musicians.

Drummers, bass players guitarists and vocalists are not naturally equipped to deal with latency. It is a totally new and alien concept to them. For that reason I avoid or minimise monitoring latency whilst recording as much as possible. Expecting them to "cope" or "adapt" is simply not an option.

On the odd occasion I cannot use zero latency Direct Monitoring I make sure the system is capable of running with extremely small buffers - usually by bouncing anything live to a slave track and minimising CPU overhead.

Synth players, on the other hand, have always had latency to deal with - it is an inherent trait of their chosen instrument whether they use computers or not. When I play VSTi synths, I accept that there will be latency and accept that the sound will be late. I don't ever "play early" to compensate because I don't PLAY with my ears - I PLAY with my fingers.

I HEAR with my ears and PLAY with my fingers.

I'm not saying everyone does it like me, but I'm willing to bet a week's wages that at least half of the DAW users out there do think like me and bry2k and the others who have agreed with us.

I know no one who is able to give a good performance by hearing the sound not in time, mentally knowing that it will be later on time.

Well you do now. 10 - 15ms is no problem. Having worked on some slow systems I can even cope with 30ms. More than that and I'm beaten I'm afraid.

I would be happy to be proven wrong, but that’s my experience and the one of people I’m talking to. It is simply impossible to play a good funky clavinet riff without hearing yourself on time.

I could play a superbly funky clav part with the speakers switched OFF! I play with my fingers, not my ears!!!

Maybe neither of us is right...

Maybe...

Meanwhile, somebody should be assigned the task of working out why my external midi gear should be subject to the same latency as my VSTi's...

That is taken in account. I can’t tell if and when it will be succesfull, but it is.

Thanks.

[popslut]

Why not have both worlds?

1. For those who adapt by using hands leave it as it is now.
2. For those who adapt by using ears have the MIDI note time adjustment disabled and "Auto-compensate for delay after record pass" enabled.

It is the opposite. If you do it by hear, the current behaviour is the right one.

No it isn't Amaury. If that were the case then neither myself of bry2k would be complaining and typing until our fingers bleed.


This incredibly simple concept is being made far more complicated than it needs to be in my opinion....

[Tarekith]

I'm surprised it's 12 pages long.

[Amaury]

I'm surprised it's 12 pages long.

Do you have anything else to bring in the conversation? I mean, yes, it is incredible.. Maybe I am to 'talky' here - just out of personal interest.

best,
Amaury

[hoffman2k]

I'm surprised it's 12 pages long.

Do you have anything else to bring in the conversation? I mean, yes, it is incredible.. Maybe I am to 'talky' here - just out of personal interest.

best,
Amaury

Just keep it going. Its fun to read.
I'm a geek and to me this is part of "days of our lives".
The suspence of how this episode is going to end is killing me.

Will they fix it? Is it broken? Who is the father of the baby? Why did Christian bitchslap Dom? Who will die?

Tune in next time to find out.

*Cheesy organ riff*

[Amaury]

Hi,

I'm ready to believe what you say as it is the way you seem to like things, but since I love this thread and feel like pushing the discussion further:

Which means, yes, my MKS 70 sounds late. To correct this I advance the midi track, post recording, to compensate.

What happens if if you record the MKS 70 on tape? You certainly can't advance the playing? Don't musicians record MKS 70 on tape, on top of other takes? How does that work? See recording in ive as recording on a tape. For instance, imagine you want to record audio from your MKS 70 directly into Live, through a mixer. You monitor through the mixer and record audio in Live without monitoring. How would you manage to be in time?

And it wouldn’t be if you would have to play the notes ‘on time’, means of the grid, and hear the sound 25 or 50 ms later. This is as impossible if not worst as playing 50 ms early. High latency = impossible to play. But this is the case whichever way you try to cope; whether you play early or hear late, it's still unsatisfactory. I'm not sure which point you are making here.

Again, this is obvious.

Good, we agree on something I'm not making any point, I'm just aknowledging the impossibility to play comfortably with too high latencies, whether in the manner I described or the one you prefer.

I know pianists, drummers and the like are very much able to adapt to a small latency in order to hear the sound on time.

I've been doing this professionally, day in, day out, for twenty years and I have never met a drummer who would comfortably play ahead of the other musicians.

I know the others ones . It is obviously very uncomfortable to experience any latency at all, but when it is there, the human brain is smart enough to adapt. See the earlier post from tylenol. I mean, how does a pianist manages to play his piano? Have you considered the problem?

Synth players, on the other hand, have always had latency to deal with - it is an inherent trait of their chosen instrument whether they use computers or not. When I play VSTi synths, I accept that there will be latency and accept that the sound will be late. I don't ever "play early" to compensate because I don't PLAY with my ears - I PLAY with my fingers. Well you do now. 10 - 15ms is no problem. Having worked on some slow systems I can even cope with 30ms. More than that and I'm beaten I'm afraid.

But you do listen to what you are playing while you are playing right? How does that go if you don't hear anything good, groove-wise? Even 5 ms off the rhythm is huge, musically. Can you provide a good and groovy performance for a full track while hearing something wrong?
Are you really able to hit the keys on time, hear the sound, say, 5 or 20 ms late, and still perform in a satisfactory way? It is not a sneaky question at all, I am curious. Can you really? For how long? Is it pleasant, and why don't you record notes 'deafly'?
I know I can't, I'm just instantly lost. Just talking of the small test I've proposed before, if I'm wanting to play a drum sound on the metronome, it's much more easy for me to adapt my finder gesture to what I hear to have it on time than the opposite. I'm not able of the opposite - being precise it is. Or I do turn of the drum sound I'm wanting to record, turn monitoring off, and then I hit the keys on the metronome, silently. But not with a late sound that disturbs me. It could be a lack of skills..

I could play a superbly funky clav part with the speakers switched OFF! I play with my fingers, not my ears!!!

Sure, I understand that, but that is not saying anything about being able to hit the keys in time while hearing the sound late.

Maybe neither of us is right...

Maybe...

I bet! Again, don't take me wrong, I'm only trying to understand others' opinion, and am quite very surprised, so I'm asking many times the same thing. BTW, I would love to hear about other people opinion on the subject. No need of a poll or anything, no need to be 'partisan', I hear you guys and will discuss it with my colleagues, but again I'm very surprised.

Meanwhile, somebody should be assigned the task of working out why my external midi gear should be subject to the same latency as my VSTi's...

That is taken in account. I can’t tell if and when it will be succesfull, but it is.

Thanks.

You're welcome!

Best regards,
Amaury

[elevation1972]

Bottom line to all this that Logic, Reason, etc. compensate for this latency problem. Play ahead, give me break. The question is can & will Ableton fix this problem. This topic can be talked about all day. Bottom line, please Ableton fix this problem.

[popslut]

I'm really wary of indulging this line of conversation because I fear we are in danger of obfuscating a very simple issue...

Which means, yes, my MKS 70 sounds late. To correct this I advance the midi track, post recording, to compensate.

What happens if you record the MKS 70 on tape? You certainly can't advance the playing? Don't musicians record MKS 70 on tape, on top of other takes? How does that work? See recording in ive as recording on a tape. For instance, imagine you want to record audio from your MKS 70 directly into Live, through a mixer. You monitor through the mixer and record audio in Live without monitoring. How would you manage to be in time?

With an unquantised recording, 10ms is an insignificant figure. If I were to be playing my MKS 70 direct to tape the notes would would fall 10ms late, 12ms early, 21ms late, 18ms early - because I'm not a sequencer. The latency of my synth would be a minor issue in this case, although experience tells me that for a biting, percussive bass sound, the MKS 70 would not be my first choice of synth - maybe because it is fairly slow to react to a key press. Anyone know the internal latency of a Minimoog? Pretty snappy I'd bet...

I've recorded drummers who play consistently 15ms late on the snare drum and it is just part of their sound. They didn't know they were doing it - they were just "grooving". It doesn't sound "wrong" because thats what musicians do - they slip and slide around the tempo.

However, in the case of sample accurate, quantised sequencer music 10 ms is a significant amount. In fact, before I was able to record my MKS 70 to a computer and analyse the result, I had no idea it had any latency at all.

It is obviously very uncomfortable to experience any latency at all...

I disagree. Up to a certain point, latency is very easy to ignore. I barely notice 5ms. 5ms feels tight to me, probably because I grew up with lazy old antiques like the MKS 70..

10 ms feels ok, 15 ms is a bit unfunky and 20ms makes me want to bounce a few fx down and reduce the ASIO buffers.

If I have to, I can cope with 30ms but it'll definitely need quantising afterwards.

But that's just me - I'm sure everyone has their own threshold of comfort when it comes to this subject.

how does a pianist manages to play his piano? Have you considered the problem?

A pianist is no more than 5 feet from his or her sound source. That = 5ms latency. Perfectly feasible.

So - let me turn the question around and ask you - how do solo church organists manage to play? They have to contend with HUGE delays.

A solo musician can't play "ahead" to compensate because there is nothing to play ahead of - and so they must just get used to the concept of hearing the note 50ms or 100ms after they hit the key. It is just an unavoidable fact of life.

Imagine having to play Bach's Toccata In Fugue on the Eisenbarth Organ at Hohen Dom in Passau. It is the largest church organ in the world and the organist has to deal with latencies of up to 100ms!! How does he or she manage to play at all? By your reckoning it is impossible. The evidence suggests otherwise.

But you do listen to what you are playing while you are playing right?

I hear it - after I've played it. Not simultaneously - that is temporally impossible. Even a Stratocaster through a Marshall amp has a latency - however small. Therefore the player will always be hearing the sound after he or she has played it - not while he or she is playing it. This may seem like splitting hairs but it isn't. This idea is fundamental to what we are discussing.

Even 5 ms off the rhythm is huge, musically.

No it isn't. With human musicians playing natural human time - ie no quantise - 5ms is insignificant. Next time you get the chance, analyse a live drum track by a brilliantly tight drummer and note just how many times they stray more than 5ms off the grid. You'll be amazed...

Are you really able to hit the keys on time, hear the sound, say, 5 or 20 ms late, and still perform in a satisfactory way? It is not a sneaky question at all, I am curious.

Yes. See above.

Can you really?

Yes. See above.

For how long?

Until dinner time.

Is it pleasant, and why don't you record notes 'deafly'?

These are rather silly questions to which you already know the answers. As I explained above, 5ms makes no difference [it being the equivalent latency of a piano] and 20ms is etc etc...

I know I can't, I'm just instantly lost. Just talking of the small test I've proposed before, if I'm wanting to play a drum sound on the metronome, it's much more easy for me to adapt my finder gesture to what I hear to have it on time than the opposite.
I'm not able of the opposite - being precise it is. Or I do turn of the drum sound I'm wanting to record, turn monitoring off, and then I hit the keys on the metronome, silently. But not with a late sound that disturbs me. It could be a lack of skills..

To which the only possible answer is "Oh. Interesting." It is becoming more and more obvious that there are two approaches to the problem of latency, one of which is addressed by Ableton and the other not. You are fortunate in that your approach is assumed by the Ableton developers to be the "correct" one. The rest of us are just unlucky and will have to hope that our working methods are acknowledged in future.

I could play a superbly funky clav part with the speakers switched OFF! I play with my fingers, not my ears!!!

Sure, I understand that, but that is not saying anything about being able to hit the keys in time while hearing the sound late.

See above - the example of the church organist.


Try this as an experiment.

Open up Live and create a midi track. Don't bother routing it to anything but route your master keyboard to its input.

Switch the metronome on. The metronome is all you should be able to hear.

Set Monitoring on the midi track to "On".

Start recording and tap any key in time with the metronome, trying to be as accurate as possible.

After a minute or so, stop.

Open the midi clip you just created and see how your midi notes correspond to the beat markers/grid.

Aside from the small natural variations in timing, you will notice that your notes are consistently late by the amount of host latency present in your system.

Now, regardless of why this should be the case, regardless of any theories about the way humans react to latency, regardless of any anecdotes about church organs, funky clavinets, sync/repro heads and the like, bear this important point in mind.

No other DAW exhibits this behaviour. Not one.

Logic, Pro Tools, Sonar, Cubase, Nuendo, Digital Performer, Sequoia, Reason, Fruityloops.

Every single one will have those midi notes more or less on the beat markers/grid.

Furthermore, if you go to their user forums, you will not find ONE post about the way they deal with latency compensation because they deal with it silently, invisibly, logically and satisfactorily.

They just work.

Nobody writes 13 page threads about how they prefer to compensate manually by playing ahead.

With every other DAW it simply isn't an issue.

I leave you to draw your own conclusions.

[Synthbuilder]

I'm not sure if I'm really following this, but surely Live should record the midi input as it is played and not bother with any compensation until its time for playback.

Right now, if I record onto three tracks with different settings of In/Off/Mon I get three different sets of results. This does seem rather disturbing.

Is there not another issue worthy of consideration... what if you change the VSTi device to an external one [or even change your latency settings] after you have recorded the midi clip? Is not the original compensation applied by Live now unaffective with the new source [or setting]?

BTW a nice idea is how Tracktion handles this. You get to see what pre-delay is applied to each track by the automatic compensation system. You can also change it if you want to. But it only compensates on playback and not on record.

I think it would also help if the manual was a little more detailed in the explanation of this. We could do with a write up on how Live handles midi in general.

The Rewire problem is a major one and affects some and not others. It would be nice to have something written down about this.

The fact that midi through data is delayed by the soundcard latency is also not mentioned. This is a major problem for those of us that use Live as a midi router, passing midi data from one controller to external midi modules.

[Amaury]

...

Hi,

I won't 'argue' and take note of your points, they will be considered.

About your last point: I repeat what I've said before: you are right, you sould be able to play an external synth with no additional latency, and that is 'worked on'.

In all of my posts I was not saying that this is the right approach.

Regards,
Amaury

[Amaury]

Hi,

I'm not sure if I'm really following this, but surely Live should record the midi input as it is played and not bother with any compensation until its time for playback.

Right now, if I record onto three tracks with different settings of In/Off/Mon I get three different sets of results. This does seem rather disturbing.

You should get the same results with auto and IN right?

Is there not another issue worthy of consideration... what if you change the VSTi device to an external one [or even change your latency settings] after you have recorded the midi clip? Is not the original compensation applied by Live now unaffective with the new source [or setting]?

That shouldn't be a problem. The way it works now (and is the point of discussion) is 'you get what you hear'. It means the MIDI notes get recorded where you hear the sound when you're performing. So, later on, when they are recorded, you can change device or whatever, they will still sound where they are.

BTW a nice idea is how Tracktion handles this. You get to see what pre-delay is applied to each track by the automatic compensation system. You can also change it if you want to. But it only compensates on playback and not on record.

As far as I understand, that's how Live wroks: appart from 'seein what pre-delay is appies', you can adjust each track's delay, and Delay Compensation is turned OFF on the track that is monitored and/or that contains the instrument, otherwise you'd have huge latencies while monitoring, always.

I think it would also help if the manual was a little more detailed in the explanation of this. We could do with a write up on how Live handles midi in general.

That is so right. Thanks for mentionning this.

The Rewire problem is a major one and affects some and not others. It would be nice to have something written down about this.

The fact that midi through data is delayed by the soundcard latency is also not mentioned. This is a major problem for those of us that use Live as a midi router, passing midi data from one controller to external midi modules.

I think the 'MIDI through' point is what popslut is talking about in the last part of his late message. We know that is a problem and off course we should allow users to record MIDI through Live to play external machines without added latency. Same goes for Rewire, though the issue is a bit differnene. We are looking for a good solution. Basically, for rewire, it can be a problem as soon as you send MIDI from Live to the host, and get back the audio on a different track. It is no problem if you use the host's sequencer.

If you have time, read the message about the other cases (playing a soft synth and recording MIDI), and also if you feel like I proposed a little test, to help finding out what we are more comfortable with.

Regards,
Amaury

[Tarekith]

I'm surprised it's 12 pages long.

Do you have anything else to bring in the conversation? I mean, yes, it is incredible.. Maybe I am to 'talky' here - just out of personal interest.

best,
Amaury

Not really, everyone else has summed up my thoughts exactly, no need to keep rehashing this. Just seems to be a fundamental difference of opinion (and understanding) concerning what the problem is and how it should be fixed. Indeed, sounds like Ableton do not even think it is a problem.

[Amaury]

I'm surprised it's 12 pages long.

Do you have anything else to bring in the conversation? I mean, yes, it is incredible.. Maybe I am to 'talky' here - just out of personal interest.

best,
Amaury

Not really, everyone else has summed up my thoughts exactly, no need to keep rehashing this. Just seems to be a fundamental difference of opinion (and understanding) concerning what the problem is and how it should be fixed. Indeed, sounds like Ableton do not even think it is a problem.

Well, now it seems Ableton sees it is a problem. But I still think it is worth a 'real-world' approach, and not a pure intellectual one. Have you tried the little test I proposed? I would really love to know what people are more comfortable with in a real-world. I would love to know if you are able to hit the keys on time while you hear the sound coming late, or if your brain is able to adjust to hit the keys in order to hear the sound at the right place.

regards,
Amaury

[Synthbuilder]

You should get the same results with auto and IN right?

Just did some more testing. Yes this is correct. No matter what the midi track is doing the auto and IN settings give the same result.

Is there not another issue worthy of consideration... what if you change the VSTi device to an external one [or even change your latency settings] after you have recorded the midi clip? Is not the original compensation applied by Live now unaffective with the new source [or setting]?

That shouldn't be a problem. The way it works now (and is the point of discussion) is 'you get what you hear'. It means the MIDI notes get recorded where you hear the sound when you're performing. So, later on, when they are recorded, you can change device or whatever, they will still sound where they are.

Ah yes I see that changes in latency settings shouldn't affect this part since the compensation is fixing only for the latency while its recording.

However, I don't agree that changing the destination of the midi track will not affect the sound.

Right now, Live records the midi data 'late' because our ears hear the sound after we actually play the note. In playback PDC will compensate for this and thus the note is heard the moment the midi data is played back.

But, if the midi note that is recorded is late in respect to when the keyboard was hit, then changing the channel output to an external instrument with its own latency produces a note that is even later. What should happen is that when you remove the VSTi for an external instrument any lateness is now removed and the note is set back to as if the channel was set originally set to off.

To illustrate this further - use a VSTi with a compressor immediately after it and play a few notes. The compressor's look ahead will be compensated by the recording process and the midi data will be recorded sufficiently 'late'. In playback the notes will sound as they were when you recorded it because PDC will compensate for the inherent delay in the VST instrument and effect.

But now remove the compressor. Does the midi note data move forward in the track to compensate for the lack of compressor. In other words, does the track now sound as if you recorded it without the compressor. Currently in Live it does not. The notes will sound late since it is assuming the compressor is still there.