Surely it uses KVM for memory isolation as well.

I guess what you are saying is that VirtualBox is still doing device emulation. If so, we agree.