But not everyone agrees on which colors go together and which clash

But perception is for a large part embedded in memory, which differs individually. For me steel foundries smell amazing because I used to play on the beach near a steel foundry, to the point I need to put effort into understanding that it's actually kind of acrid. So am I still "having the same perception" as someone who doesn't have the lived experience?

This can happen at a society-wide level too. Liminal beige and seafoam green were not intended to create a feeling of disquiet, but of calm neutrality. Modern audiences perceive them as disquieting because they have been systematically used in our society to impose a sense of calm on un-calm situations, such as operating rooms or hallways in sketchy buildings.

I honestly don't know how much of the commonality of associations across cultures comes from instinct and how much comes from the fact that all children learn to live on the same planet with the same physical laws. I would bet that for 99.9% of children, their first experience with a strong sulphur smell is going to be from rotten eggs (or similar rotten goods) that others act disgusted by. So the fact that sulphur smells disgusting to the vast majority of adults is not evidence for instinct over memory. The same goes for green plants, red blood, blue skies, etc.

i think the best example of this is music. most people have very good relative pitch perception, but mediocre absolute pitch perception. that shared relative pitch perception though will often be built based on ones own experiences and life. the shared perceptual relativity is both developed socially/culturally and innate based on how we process and abstract information, but that says nothing about our absolute perception of that information.

Here's the standard color wheel set to Red, Blue, Yellow as primary colors.

You'll notice that magenta is represented as almost a whole different color. It's light red in the CMYK, light purple in RGB.

And

Cyan, baby blue, sky blue, etc. isn't represented. Instead you get a blue-purple they call violet.

Light Red - Magenta

Light Blue - Cyan

Blue-Purple - Indigo

Light Purple - Red-Purple - Fushsia

We as a whole can't decide what constitutes purple/violet in RGB model

Even if someone doesn't know what a true "Indigo" looks like they are still experiencing that color for what it is. They will just call it Bluish-Purple or Purplish-Blue. And unless it really was the exact mix of 50/50 blue and purple it wouldn't be indigo. It would be a equivalent to a Redish-Orange. A Bluish-Indigo or Purplish-Indigo.

Sorry for the walls of text I was learning and thought I might as well share.

Yeah, I'm in agreement with you, my point is that we have proof that people perceive reality slightly differently, in general it's pretty standardized, but there are slight variations. That's all my point was.

Wow. You are just proving my point. It looks white and gold to me.