I grew up the same, mostly because allowing so much as not literal means and easier time for a pastor that also does car upholstery part time to sell any concept. Literalism is pretty demanding a a position.
The literalist interpretation was seen as extreme until maybe 20 years ago. I was shocked to learn about how many denominations are going in for it now. But maybe that's just the internet showing me parts of the world I hadn't seen before.