Any set of colors can make a primary color palette. It's just that we most often mean for human eyesight.

Even then, when talking about perception, because the response of red and green comes in our eye overlap we need imaginary colors to uniquely express them. https://upload.wikimedia.org/wikipedia/commons/thumb/e/eb/CIExy1931_ProPhoto.svg/1280px-CIExy1931_ProPhoto.svg.png