Because it makes people think about their mother having sex with a stranger, and that thought is somehow worse than imagining her with your father. It's just one of those stupid things the human brain gets hung up on even if it's natural and not weird if in different contexts. Isn't the human brain fun?