I did some extra checking, and it sounds like for Linux, there isn't really a true exclusive fullscreen, but some games will behave differently depending on which mode you choose (because of how they've been coded). If you're getting bad tearing in borderless but not fullscreen, it may be one of those finicky cases.
It's strange that you can't turn on frame gen in fullscreen, though. Maybe it's a bug that you should report.
I haven't had an occasion to try it, but CachyOS's Proton supposedly has FSR 4 support. Maybe try that version...? Details here: