Most of them aren't necessary to most people, but the main concern is features that should reasonably be part of the core Android experience being removed, or features that have no reason to be reliant on Google at all.

For example, GrapheneOS can't support the detection of your phone being quickly ripped away from you to auto-lock the device, even though that should only require onboard sensors and processing, and it can't support the additional custom clocks for lock screen customization, because Google decided those would be built into the Google app then extended to Android after, rather than being built into AOSP.

You can reasonably see a future where other functionality gets put into these proprietary blobs too. Maybe the launcher becomes proprietary and GrapheneOS has to use or develop a separate FOSS one that might not support all the same features. Maybe charging optimization gets locked behind proprietary code because Google claims it uses "special algorithms" to adjust how your phone charges. Maybe Private Space gets turned proprietary because Google claims it needs special security features.

That's why it's particularly concerning, because in the future, Google could just decide that any number of features aren't part of AOSP anymore, and now GrapheneOS either has to give them up entirely, or make/find an alternative.