If you tolerate intolerant people, you will lose your right to tolerate them. Full stop. You cannot tolerate and debate everyone. Some people dont play by the same rules or have the same goals as you, and the human brain is flawed enough that 'the marketplace of ideas' cant exist to serve all ideas. Some are objectively bad. They are objectively, morally, wrong.
And if you let them in, they destroy. They serve no other purpose. It's fine in fiction. Most people understand the empire in star wars are objectively the bad guys. Most people understand humans are the bad guys in Warhammer40k. Most people understand humanity were the bad guys in starship troopers. Most people understand the Klingons were the bad guys in the star trek universe despite some members of the species being reasonably morally Grey.
But in reality when we have fascists, we cant let them play out their fantasy.