There's 8 billion humans on earth, and if you assume that's the average planet population in the empire, you're only maybe a trillion of there's over 100 planetary civilizations under imperial rule.

I'm not a big enough nerd to know how many there really are, but thinking back to how many reps were in the republic senate scenes, it's probably around a trillion (they generally rep a planet, right?). But that's being generous, plenty of those planets are noticably smaller than Earth and I don't think they all end up imperial.