This is just social darwinism.

The reason this and other concepts of meritocracy are bullshit is that the children of the elite will always have better access to education, and much more free time, than their less well-off peers - therefore those considered to have the most 'merit' are inevitably identical to those who already comprise the upper classes.