If you tear out the parts that talk to Google, then the phone hardware isn't spying on you. It's just hardware.

The critical piece tying your phone to Google every 3 minutes is called "play services".

Calyxos was an OS for the pixel hardware that replaced play services with a FOSS library (called microG) which tricked regular apps into thinking they were talking to and getting responses from Google, when it was actually all happening on your phone.