Sorry, to me that just sounds like reaching for explanations. The far simpler explanation is likely more accurate, which is that it's easier to mobilize resentment / drive wedges between people, if you label all conservatives, and any conservative position, as racist.
It's the same tactic used by right wingers, when they label any socially progressive approach/statement, as communism. People generally understand communism is bad. So if you can associate ideas with communism, you can get people thinking those ideas are bad, without weighing the merits of the actual argument/idea. Similarly, if you can associate an idea with racism, you can get people thinking those ideas are bad, without weight the merits of the actual argument/ideas.
It's prejudice, in either case.