but other areas it would be illegal to ignore invasive plants and not remove them.

I've never been in a park or area where you, as an individual, were legally liable for not removing invasive species.

That being said, I did get in trouble for having thistle in my yard, but there was a specific city ordinance against thistle, and not against invasives as a whole (otherwise most of the fucking neighborhood would be torn up, which TBH would be great.)