Which I guess you can get away with it if you’re white taking the land from someone darker than you

Nah. It’s whoever has the biggest weapons gets away with it. If you’ve got a larger country with a bigger stick standing behind you, you can attack anyone you want and no one says shit.