The lack of controllers is the biggest impediment to these "Homemade VR" projects.
You might as well slap your phone in a Cardboard VR and use ALVR, and you'll be surprised at how convincing it is, but you'll quickly find there aren't many VR games worth playing with the standard gamepad.