it just signifies you are in favor of equal rights among genders

It doesn't "just signify that" though, as much as feminists act like it does. The term "feminist" does signify a person who, at least ostensibly, is in favor of equal rights among genders, but using that term also, necessarily, implies belief in the harmful dogma that is inseparable from the term itself (patriarchy theory, etc.). This creates a false dichotomy that makes people feel that in order to support equal rights they must also buy into feminist dogma, and that's not at all the case.

Luckily, though, feminism doesn't have a monopoly on gender equality, and it's important to let people know that fact, both because of how incredibly misleading "feminism just means gender equality" is and because there are plenty of other more useful, more egalitarian frameworks through which to view the push for equality.