They mine it everywhere. They mined it in caves as well.

Before potassium mines and nitrogen extraction methods were discovered, bird droppings, typically from islands that were covered in mountains of the stuff, were the main source of fertilizer in the world and the main limiter of human population. Without these sources of bird poop, literally millions, potentially billions, would have died. You remember malthusianism? A fair bit of that was based on not having enough bird poop.