It's because they can understand a difference of philosophy or beliefs but cannot comprehend a total rejection of the concept of belief. It's too alien to a central aspect of their identity and challenges their perceived connection to existence.

Source: I've been making people uncomfortable by being openly hostile to religion for a long time.