You think far faster than they can speak
People speak at roughly 125 to 175 words a minute, but your brain can process several times that fast. That leaves a pool of spare mental capacity, and unless you deliberately fill it with their words, it drifts to your phone, your to-do list, or your reply. Good listeners spend that surplus on the speaker.