Even if they manage to keep the phone unlocked, and even if they manage to get to a safe spot before I am able to lock the device remotely, they still can't access any banking applications because they require the PIN/biometrics to get in, even on an unlocked phone.

If their bank is anything like my bank, you NEED to have the mobile app installed to login with for online banking.

The standard for banks like capital one is, in order to login in a browser you must use their app 2fa, they do not support third party 2fa methods, and their app doesn't work under compatibility layers. That last part may come with a workaround, but my earlier point is that the normal end user wouldn't feel comfortable using such a thing if it is not officially allowed.

They also claim you can use 2fa by SMS, but that is first and foremost wrong since it isn't an option for payment portals, and secondly insecure.