I think the systemic issue is just a lot of people being inconsiderate. Yes you have to choose. That should not be a problem for most people, as one of the choices is the obviously correct one, and the other is obviously illegal (even if it's practically unenforced, everyone knows you're not allowed to ride a bike on the sidewalk).

You don’t see car drivers having to share their infrastructure with pedestrians or vice versa.

Well yea, because cars don't fit on the sidewalk. And where they would, you often see barriers explicitly blocking cars, which are often installed because car drivers were ignoring laws and driving there anyway.

The issue is that's not really possible for bikes, because when you make something that blocks bikes from entering, it's going to block pedestrians as well. They are practically the same width.