I hear you, but as you just said it can be adapted to be used as an exaggeration, generally to indicate the person is out of touch in opinion.
A similar pattern with our speech is how terms for mental disability (eg idiot or some of the modern terms recently deemed a bit more hateful) were adapted in our language to become insults.