I'm skeptical of this - where do you get those figures from?

Objective counter-evidence: Why are they not selling out big stadiums if spectating is that popular?

Anecdotal counter-evidence: I hear people talk about hockey matches, I rarely hear people talk about curling matches.

They're both good sports and I like both, I'd like to try curling sometime.
But surface level evidence suggests you're incorrect, so you need to provide more convincing evidence