Audio breakup when starting/stopping/scrubbing through audio

[Khaos & Effect]

I'm currently mixing a set which is fairly heavy, but in which I've frozen most of the tracks. Whenever I start playback or scrub through the audio, the CPU usage meter jumps and the audio breaks up for about a second getting very stuttery. It only seems to happen when using my audio interface (M-audio Fasttrack). First, is this normal? I can't recall this occurring in any of my other projects. Second, will this kind of audio input (heavily broken up) damage my monitors?