You can have an encryption key that can't decrypt the data. That's what public and private keys are. The device can have a public key to encrypt the data, and then it can only be decrypted with the attacker's private key. No need to even get it out of the device, you can also perform the decryption on the same device. Just don't store the private key in there.