You are. You understand.
The real issue is where the communities are, rather than the users. Smaller instances can't easily attract large communities of people from outside their own, so the larger instances get to run the main communities for everyone. It's working as intended but it's all in the numbers.
A lot of the federation links were set up years ago, so I wonder if anyone is even activily trying to federate, like we used to when it all began.
Like, the NSFW instance went down some years ago, and I don't think anyone from the instance I'm using has bothered federating with a new one with that kind of content, or at least it doesn't pop up anymore. Which is fine, so I also don't bother, but for a newcomer it might seem odd that some instances have more connections than others and such.