To be fair, the Europeans conquered the land from the native Americans before they could get conquered or enslaved by the Aztecs or Incas

The fact that they couldn't drive them away is just pure skill issue.