Did you try installing without ventoy?

Sometimes ventoy causes issues. Although every time I've had an issue I thought was a Ventoy issue it had turned out to lie somewhere else, but it's important to isolate variables.

Try posting the kernel panic messages, they should give an idea about the failure.

Proxmox has its own custom kernel, it may be something in the new 7 branch as opposed to the conservative 6 branch that Debian uses.

99% of the time it will be some driver that you don’t really need and can disable.