A pinhole camera doesn't focus light. Only lenses can focus light. The reason why looking through a small hole makes the image sharper is exactly because that basically prevents the lens in your eye to "misdirect" light that's coming in from the side.

But there is a reason why our eyes are not pin hole cameras. The amount of light that hits your retina is pretty low when looking through a pin-hole so that actually only works on bright sunny days.