Could be either the fact that the audio format is different, so it has other work to do for DD audio decoding. Could be that the signal size is bigger (DD takes up more space and therefore data), hence a slower reaction time.
I've never experienced it myself since I don't subscribe to any...