Pretty massive downside.

But also, your first two sentences are a huge exaggeration. The vast majority of ps5 physical games ship entirely on disk. You can buy them, insert them into an offline ps5, install, and play. No downloads, no internet required.