If I schedule a meeting and someone doesn't accept that invite I just assume they're not showing up and act accordingly. Might send them a ping a few minutes before hand but if they're a stakeholder and needed for the meeting and choose to ignore the invite(s)/email ... That meeting gonna get cancelled or rescheduled.

Not that that very often happens but I'd just start bothering said guy about it with pings or email follow ups until they get the picture. Or just cancel the meeting(s) if no response 24 hours before scheduled. Something like that.

Not even to be petty or annoying just because they should be accepting or declining for the simple fact that everyone else has schedules to keep too. Obviously this advice may be bad or not applicable but yeah that sucks.