If someone is coming physically for your stuff, They wont just take a drive, they will take the whole thing so they will have access to the whole thing.

This is where drive encryption comes in so when they power it back on it asks for the drive decryption key, of some kind of intermediate key to get the actual key from the TPM.

There is a defcon talk about anti-forensic and anti anti forensic mechanisms you can use to watch to make breaking your system take longer.