It turns out you can kind of get the same functionality on Windows if you rename the open file and place the new one with the original name, but it's a bit of a hack.

Only if you don't have OneDrive working. In that case, you have to wait for it to sync or it won't go through.

Anytime I have an issue at work where I can't change or delete a file, it's a 50/50 split between Excel and OneDrive being the cause