Not a reasonable claim, but within a couple years it was just blanket public opinion that associating with atheism or men's rights meant you hate women.
My observation is that there is a minority in the men's rights movement that does hate women, and the movement is not great at making it clear that they do not represent the movement and fail to exclude them if necessary. Mind you, the same problem exists vice-versa within the feminism movement. Those movements then point at those extremist minorities in the "opposite" movement (if you remove the extremist elements they don't even really oppose each other, but unfortunately the extremist elements are a part of the movements) and say "hey, they're fighting against us!" as a kind of rallying cry. And it's not exactly wrong - it's almost similar to the Nazi bar analogy that if you don't draw clear lines to keep those extremists out of your movement, well, then yeah, they're part of your movement. And so things just get more and more extreme, with the social bubbles the internet creates just making things worse.
And honestly, it seems like every progressive social movement has this issue. Whether it's anti-racism or awareness for sexual violence, there's always that one extremist section the movement does not seem to be able to or want to exclude.