Host compensation for Plugin with audio delay

[ronanconlan]

I'm using Intelligent Device's Slip-n-slide VST effect in Ableton on live audio. The live part is important.
I understand the plugin delays audio by 500ms to do it's processing on the signal - this is fine, I don't need to use it on anything too tight or staccato or anything.

But, when I turn the plugin off, all audio is still delayed by the ~500ms or so. It only goes back to normal, low latency when I actually delete the VST plugin from the track.

My audio is routed through the track containing the plugin.

Is there some way I can ensure that Ableton's audio goes back to being NOT delayed once I hit the OFF button on this VST effect?

Thanks a lot