the whole bag is gone

Well, that's how it's supposed to be done. Since mold is a fungus, what shows up on the surface is the reproductive parts that spawn spores, meaning the rest of the bread probably has mold too.