I think most people are ok with you choosing to not vaccinate. The problem is when you choose to inflict that decision on others.
Not vaccinating and not isolating yourself is violating everyone else's body sovereignty.
I don't care if you host diseases. I absolutely do care about you spreading them.