I think it's just something YOU care about and you're the reason women can't wear practical clothes or have real pockets on pants and have to be worried about panty lines. Stop worrying about panty lines and go worry about something that matters.

I'm not a woman, but by talking from time to time with the women I know (my GF, some friends and my sister), sometimes it's unavoidable. They tend to buy underwear made with certain fabrics that doesn't show as much as cotton panties with very thick borders. But also, from time to time, they pick certain underwear they know will give VPL as long as it looks sexy

Or is this just something men care about?

I don't think most others would care. I don't think it's something to care about. Calling it unflattering is a bit of a judgement. I would say this is something that women don't really need to spend any time thinking about and neither do men.