Simple, because the compliments aren't for something you personally value. Imagine if you worked hard on something and want people to notice but instead they compliment something else.
For example my handwriting looks very good, not because I want it to be, but because when I was a child and asked mydasd to help me with my homework he made me rewrite it again and again until the handwriting was immaculate instead of actually helping me with the subject. People saying "Wow your handwriting looks so neat." Makes me just want to say "Thanks, for complementing my trauma, but have you actually read what I wrote?"