It's a tough one, on one hand yes we killed their predators so we have to do it instead; on the other if those predators somehow come back then most often we just kill them again so we can continue hunting. I suppose my biggest issue with hunting is the seemingly inevitable wolf cull.
If there was no farmland, if the majority of this planets surface wasn't being used exclusively for human activities then I would have no problem at all with hunting; however that's not the case. Wild populations likely seem abundant but they are a fraction of what they were before industrialization/colonization (Overton Window) and further pressure on those populations, that are essentially holding on to slivers of the only land left for them, can only do harm (with the caveat of overpopulation).
As well, most hunters use lead bullets because they're cheap, so now all areas where hunting happens regularly are contaminated with lead, especially the predators (including humans) of that area (bioaccumulation).
I guess the least impactful way to hunt is to use copper/non-lead bullets(?) in rehabilitated farmland.