It would depend on the features lost (I've heard rumours that some cars lock the trays or reclining the seats behind the modem access) and on whether such a modification would run afoul of the certification processes needed to keep the car on the street, legally speaking.

For much less mental health hassle, I could simply search for an older car.

We've got a 2015 Toyota Highlander with only Bluetooth for the headunit.

Well now that's an interesting one! It has been a long while since I last saw a CD player in a vehicle.