When streaming content to a chromecast it usually makes a direct connection to the content source, not going through the device you're casting from. In limited cases like when sharing your screen to the chromecast it does only go from your device to the chromecast without internet, but that's rarely used because it's power and data intensive to your device. I guess you're right on the internet not being 'strictly' needed but it's used for a reason.

Should still buy your own router though, and move away from google services 👍