Colonialism isn't motivated by morals, colonialism is stealing other people's land for its resources (most of the time; and sometimes the people were the resources being stolen) or strategic importance, or because it's there, or because everyone else was doing it and not having colonies looked weak, or because it was a nigh uninhabitable far away place were you could dump any undesirables you wanted to forget about.

Morals were merely an excuse or justification, never the real reason.

More often than not colonialism didn't even consider the native cultures (or the natives themselves, except possibly as a nuisance or, unless they were the resource being stolen, and then they weren't considered as a culture or people, they were considered as product), they were irrelevant. Maybe some missionaries tried to teach them religion, so they could go to hell for their sins, and very occasionally some scholar categorised and described the cultural artifacts being stolen or destroyed, but that was never the driving force.

When I say fuck the prime directive I'm not talking about colonialism. I'm not talking about destroying cultures, or stealing their lands. The Faroese wouldn't be any less Faroese if they couldn't torture whales. They'd still have the rest of their culture, their language, their history, and, unlike with colonialism, their land and their lives. The “harm” to them would be minimal, and the world a less cruel and monstrous place.