Real answer: it depends.

• Deleting a file in use: no problemo. File is removed from the directory immediately, but exists on disk until last program who had the file open closes. Everyone wins! (Unless you're trying to free up space by deleting a huge file that's being held open by a program and not understanding why the filesystem usage didn't go down)
• Unmounting a hard drive in use: Will error out similarly to Windows. lsof can tell you which process has which files open. There's nuance with lazy unmounts and whatnot but that should not be used in most cases.

Now in practice you should be wary of one very important thing that changes compared to Windows: Writes are asynchronous on Linux. First the kernel writes to RAM, then it flushes to disk at a later time for performance reasons (this is one of the reasons why writing a bunch of small files is many times faster on Linux than Windows). The upshot is that just because your file copy is "done" doesn't mean you can just yank the USB cable. Always safely unmount before unplugging a storage device on Linux.