Improved overall MIDI latency

[en5ca84]

Hi!

I've read the https://www.ableton.com/en/manual/midi-fact-sheet/ and understand(?) that Ableton Live introduces some 'extra' midi latency/buffer to eliminate jitter and to maintain sync better.

Quote:
'Ableton‘s approach to MIDI timing is based on two key assumptions:
1. In all cases, latency is preferable to jitter. Because latency is consistent and predictable, it can be dealt with much more easily by both computers and people.
[...]'

I also understand the inevitable latency when monitoring audio through software/DAW. And how some audio processing plugins introduce additional latency, and how the Delay Compension and Reduced Latency When Monitoring functions work in these situations.

But I would love to see some kind of 'near realtime' MIDI Performance mode in Live. I think this would greatly improve the experience when performing, well, live with Live. Even at the nowdays greatly reduced risk of jitter.

An atomic example with overly large buffer size of 2048 at 44k sample rate. With my audio interface, this results in a total roundtrip latency of around 100 ms. About 50 ms for audio input and output.

I setup a very simple project. One MIDI track taking input from a USB MIDI keyboard and routing the MIDI output to an external hardware desktop synthesizer, which is also connected with USB and monitored directly. I press a MIDI note, and instead of hearing the sound in near realtime, I hear the sound after 100 ms, even though the Ableton Live audio engine plays no part in this from my perspective. Allright, if were using buffer size of 256, the latency would be about 15 ms, which would be tolerable in most cases.

Even if I disable all audio inputs and outputs from the global preferences, and am routing MIDI from one USB device to another, the audio interface total roundtrip latency seems to be the reference for passing through the MIDI data. But I guess that's how Ableton meant it to be.

But if I setup a more complex project, issues arise even with low buffer sizes. But let's assume the 100 ms roundtrip for an example.

I setup one MIDI track taking input from a MIDI keyboard as in the previous example. I route it to an external hardware MIDI sequencer, to control it's note transposition/key changes or something similar. The external sequencer takes MIDI sync from Live, and the resulting MIDI output from external sequencer is routed back to Live and is used to play a software instrument on another track. Now, for the external MIDI sequencer to be more accurately in sync with the Live internal sequencer, I need to set a MIDI sync offset of around -100 ms for the external sequencer, which is roughly the negative of the total roundtrip latency. This makes sense.

But if I where to perform with this setup, instead of just adjusting myself for the 100 ms latency, I would actually have to double that in the anticipation of playing the MIDI keyboard. Because when I hear the sequencer controlling the software instrument in sync with Live click, the sequencer is actually running 100 ms 'before', because of the offset. Again, makes sense. But when I press a note on the keyboard, it's also susceptible for another 100 ms of MIDI latency when routed via Live. So, for the note from MIDI keyboard to land on musical time in the external sequencer it needs to be anticipated 200 ms instead of the 100 ms roundtrip latency. With a buffer size of 256, this means around 30 ms of 'performance offset', which starts to make me late in my timing when perfoming; in practice meaning in a 1/16th based sequence always missing the first note etc.

Now, if I were to take the previous setup, and instead of using software instrument, I were to use the same chain to control that external hardware synthesizer using the recommended External Instrument device method, it would add yet another 'pass' of both audio and MIDI latency of 100 ms, resulting in a 300 ms of monitoring latency. 100 ms for MIDI keyboard > Live > MIDI sequencer, then 100 ms for MIDI sequencer > Live > External Instrument MIDI, then 100 ms for the audio from the hardware synth to playback through Live and so on.

And this is my real life situation at the moment, and with 256 buffer size the resulting latency is closer to 50 ms, which makes performing a bit of struggle. From my perspective the buffer size/sample rate latency is 'just numbers', the latency still accumulates every time MIDI is routed between external hardware devices via Live no matter what the buffer size is, and this could probably be significantly reduced if MIDI was routed through Live 'as soon as possible'.

I'm aware that this can be worked around to some extent by setting up a single channelized MIDI cable chain for all external hardware, but readjusting/changing the setup would always be quite a load of work. In this type of setup I would also lose the ability to use my different MIDI controllers/keyboards/sequencers as discrete supported control surfaces for Live, as Live supports only one control surface per MIDI port (for a good reason).

I've also used rtpMIDI to route MIDI via the computer with USB connections 'as soon as possible', basically bypassing Live for this, and have had very little if any issues with jitter or MIDI latency. In 2018 from a 400 euro Windows laptop and a 100 euro audio interface to 3000 euro Macbook and a 1500 euro interface made no noticeable difference for this from live performing perspective.

I believe in many cases Live is used as a 'closed' system in both studio and live applications even when external hardware is used, and the resulting audio is monitored/processed through Live, because it's convinient, so online audio and MIDI don't have necessarily to be in absolute sync. There are of course scenarios, where it's critical to maintain absolute sync between audio and MIDI through the entire signal path, such as both feeding audio and MIDI control data from Live for an external audio processor or VFX equipment, but from an average performing musician perspective this seems quite high 'penalty' to pay for such specific situations. Especially when these kind of scenarios usually involve separate tedious tweaking of the timing alingments anyways, and no matter how accurate the show control source material sync is, it's always just a starting point.

So, it would be nice to have this dealt natively inside Live. Perhaps some sort of global preference for MIDI latency, something like Stable vs. Fast, where Stable (the default) would be the current method and Fast would be 'as soon as possible' at the risk of jitter. But for now the 'Live' in Ableton Live for me feels more like 'Accurate and stable sync' rather than 'Live'.

Again, the quote from the article behind the link in the beginning:
'Ableton‘s approach to MIDI timing is based on two key assumptions:
1. In all cases, latency is preferable to jitter. Because latency is consistent and predictable, it can be dealt with much more easily by both computers and people.
[...]'

I don't know when this was written, but it would probably be safe to argue that latency over 15 ms becomes an issue in any case of people performing music. I've had practically no real life issues with MIDI jitter even with low-end consumer level equipment. Only exception being human performance related MIDI transmitted over WLAN, although it has nothing to do with the DAW/related audio or musical equipment.

Or someone please tell me 'as soon as possible' MIDI is already available through some single slightly obscure toggle somewhere, or some other method of fast MIDI routing through Live.

[en5ca84]

To sum it up:

MIDI latency starts to become an issue when working with 2 or more external MIDI devices and using Live as the MIDI router/hub to route the MIDI traffic between them.

So, it would be nice if Ableton updated their philosophy on MIDI latency/jitter and would implement 'as fast as possible' type of MIDI traffic as an option.

[pottering]

http://expressiveness.org/2012/12/04/midi-jitter

[en5ca84]

http://expressiveness.org/2012/12/04/midi-jitter

Thank you for response and the related article!

After reading it I would assume Live uses the audio input latency as an extra time window for MIDI jitter buffering, maybe based on MIDI driver event timestamps, and delays the MIDI output to match audio output latency. But with decent computer and hardware this feels like an overkill.

I've done similar tests with MLA v2, setting up software MIDI routing between multiple USB MIDI devices using MainStage 3, and measured around 0.15 ms of MIDI jitter RMS per port, with jitter peaks at around 0.7 ms. Total roundtrip latency of 13 ms with 256 buffer + jitter for playback. At least for my brain the MIDI jitter and latency is practically non-existent, and the total audio roundtrip latency is acceptable.

The point is that although Ableton encourages the usage of best possible hardware there is no benefit from this, when it comes to the DAW computer related MIDI jitter and latency. Live can achieve steady and somewhat 'jitter free' MIDI on low end computers at the cost of rather large extra latency. But there's no option for higher performance computers and hardware to get 'as fast as possible' MIDI traffic when the DAW computer and hardware would be capable of running at low MIDI jitter and latency, which I think would improve live performing experience.

Of course this would require a 'clean' operating system, decent computer and not-the-cheapest hardware, but it would nice to have it as an option.

[hoffman2k]

Which audio interface are you using? Because those are some bad numbers you're listing.
Also note that the higher the sample rate, the lower the latency. The difference between 44.1 and 96 is spectacular.
And yes, higher sample rates demand more resources. But you can only play live what you plan to play live, all the tracks you're not playing live can be frozen or prerecorded. Thus using less resources and allowing for smaller buffers.

[en5ca84]

Which audio interface are you using? Because those are some bad numbers you're listing.
Also note that the higher the sample rate, the lower the latency. The difference between 44.1 and 96 is spectacular.
And yes, higher sample rates demand more resources. But you can only play live what you plan to play live, all the tracks you're not playing live can be frozen or prerecorded. Thus using less resources and allowing for smaller buffers.

Thank you for your concern and advice!

Please read the previous posts more carefully. I'm talking about MIDI latency/jitter. I used an overly large audio buffer as an example to keep the numbers more easily comprehensible.

The current way Ableton Live handles MIDI traffic is very good for simple setups when maintaining absolute sync between audio and MIDI, and minimizing MIDI jitter, is cricital.

When using Live as a MIDI router between different external MIDI devices, the MIDI latency caused by jitter buffer and audio sync delay accumulates every time the MIDI data passes through Live, no matter what the initial buffer size or total roundtrip latency is. With 256 buffer and 44k sample rate. MIDI from keyboard to external synth 14 ms through Live (could be ~1 ms with 'as soon as possible' kind of unbufferd MIDI traffic), and then audio monitoring through Live 14 ms. Total latency 28 ms with Live (could be 14 ms + ~1 ms). Let's add another external MIDI device in the chain between the keyboard and synth and route all traffic and handle monitoring through Live. Now the latency becomes 14 + 14 + 14 ms resulting 42 ms, when it probably could be ~1 + ~1 + 14 ms (14 + ~2 ms). And from what I've measured the MIDI jitter seems to be way below 1 ms even on moderate hardware. In fact so low the mechanical jitter between different buttons and keys is probably higher.

The 'slower' way seems to be the way Ableton meant it to be, because elimination of MIDI jitter is prioritized over latency. As Ableton states in their MIDI fact sheet.

But I think this philosophy is getting a bit outdated, or is 'too safe', as MIDI jitter on computers/DAW has reduced greatly. And it would be nice to have 'as soon as possible' MIDI traffic through Live as an option.

[LightBeforeDawn]

I agree, MIDI Jitter is a huge bummer. I was hoping the problem lies in my computer hardware (and partly its true)
But new powerful standalone computer was not a game changer. I still have to manually edit audio files for perfect sync with grid. I don't believe the issue can't be fixed