I would describe the difference by how the person responds to the labeling.

A shrug or even bettter a smile, that's a dumbass and they know it.

An angry or defensive response, that's an idiot and they cannot accept it.