[H]ow does one look at atoms and say "My God! There must be many worlds than just our one?"
Electrons. You've seen the model of the atom, right? Cluster of balls in the middle (protons and neutrons) and the electrons are little balls that whizz around like little planets around a Sun?
That model is a simplification of the truth. It turns out that it is impossible to pin down where an electron is and also know what it is doing. And if you know what it's doing (you can see its effects), you'll have no idea where it is.
Where they are has to be measured by probability. "It's bound to this nucleus / taking part in a chemical bond so it's likely to be in this vicinity", is about as close as you can get.
There is literally nothing excluding that electron from temporarily being a billion miles away. That's astronomically unlikely, but it's not impossible.
And by some measurement methods, when you do try to pinpoint where the electron is, it can appear to be in multiple places at once.
This can be interpreted as bleed-through from nearby quantum realms, maybe even other universes, where the electron is in one place per nearby universe. One of those places is ours, but we cannot tell which. And by the time we've made any kind of determination, the electron has moved. They never stop.
Photons - particles of light - also do this. All subatomic particles do this.
The more subatomic particles you have in some combined state (as an atomic nucleus, or even a molecule), the lower the probability is that that bound state can be in multiple places at once, but again, it is not ruled out.
But it does mean that the more bound particles an object is made from, the more definite its position appears to be, which is what we're used to at our human-sized scale.