I understand your concern over many people lacking a voice under such a system. However, selecting people at random necessarily leads to average people deciding directly. You likely will never be selected, but someone like you will invariably get on a board.
The real problem is how hard it is for an average person to run for office. Do you have extra money laying around for campaign funds? Most people cannot self fund, thus they need donors. Holding office is for poor and middle class folks as well. Many people could never run as they cannot afford to miss work. That to me is undemocratic. This bypasses all of that.
I concur; referendums are a good tool, and some items should come with a required vote.