Gamers nexus talks about your exact point at this Timestamp. And he agrees with you.
https://m.youtube.com/watch?v=6IFVTcM28KA&t=635 He says something like, “A common misconception is taking your TV off your network is enough to stop them from spying.”
You should watch the videos. The tvs also scan for open WiFi networks, like the ones att and xfinity offer and potentially can send your data even if you’ve personally never connected the tv to any network.
10 minutes 35 second in, he addresses this common misconception