It may seem natural to us, but conceptualizing zero as a number in its own right (as opposed to "absence of [concrete thing]") is not trivial. It's an innovation on par with negative, irrational, and imaginary numbers.

Your hypothetical goat non-owner would likely say "I don't own any goats", and would be confused if you insisted that there is some number of goats they own.