Live does not free memory when you delete/unload an internal plugin from a track/rack until you close/restart Live! Nor does it free up RAM when you delete/unload a sample from Simpler/Drum Racks or any track clip.
It does free memory at once when you delete/unload a VST. I don't know if this is a feature or a bug.
At least it wont eat more memory as long as you keep using the same internal plugins that you had used before and don't load samples bigger than the already reserved memory, so it seems like a sort of cache or just reserved memory. Still I don't like that behavior, but maybe it serves some purpose I fail to see?