What you, and everyone else, are saying is that “all men are potentially dangerous”.
"All men are dangerous" is the exact same sentence as "all men are potentially dangerous." These are the exact same sentence.
Someone is not "dangerous" if they're actively in the act of raping someone. They're well past the point of being dangerous to being an active violent threat. "Dangerous" in this context simply means "risky." And yes, all men are dangerous if you don't know about them. They present a potential danger. Ie, they're dangerous. A woman meeting stumbling across a random man on a street at night is in a dangerous situation. There's real risk there. Even if the guy turns out to be a saint, that danger still exists until proven otherwise.