It's about squick.
Think about how they call sex with animals "bestiality". It's not "oh it's because it hurts the animals!", it's because "ew, gross, you're stooping to the level of an ANIMAL!".
I don't think it's about consent, either. If you were somehow able to communicate well enough to actually get consent (which, to be clear, is pretty iffy what with the whole language barrier and such), people would still be squicked.
(also I'm pretty sure "zoophilia" is more about being into animals rather than actual sex with them, which is the whole "bestiality" thing. You can be into someone/a group of people without wanting to rape them.)
It's probably because most of society fundamentally doesn't see other animals as people, and therefore killing them is totally fine ("what? they're not people!"), but having sex with them is Evil and Bad ("why would you want to have sex with them? they're not even a person!").
Yeah it makes no sense.