I'm about to make you happy. The below script puts SSH into initramfs, so you can SSH in to a prompt and type your LUKS password at boot. No part of the system is accessible over this SSH connection, just the prompt. You also still get the prompt locally on screen.

PORT=22
PUBKEY=...
sudo apt install -y dropbear-initramfs
echo "DROPBEAR_OPTIONS=\"-I 180 -j -k -p ${PORT} -s\"" |sudo tee -a /etc/dropbear/initramfs/dropbear.conf
echo "no-port-forwarding,no-agent-forwarding,no-x11-forwarding,command=\"/bin/cryptroot-unlock\" ${PUBKEY}" |sudo tee /etc/dropbear/initramfs/authorized_keys
sudo dropbearconvert openssh dropbear /etc/ssh/ssh_host_ecdsa_key /etc/dropbear/initramfs/dropbear_ecdsa_host_key
sudo dropbearconvert openssh dropbear /etc/ssh/ssh_host_ed25519_key /etc/dropbear/initramfs/dropbear_ed25519_host_key
sudo dropbearconvert openssh dropbear /etc/ssh/ssh_host_rsa_key /etc/dropbear/initramfs/dropbear_rsa_host_key
sudo update-initramfs -u -k all

@tofu@lemmy.nocturnal.garden for if this is easier than what you are doing.