To build on this, this usage is called a non-count noun. Less beer, less water, less air, less sand, etc. all refer to non-countable quantities of some substance. Beer could be counted, if referred to by some metric (“one glass of beer,” “24 ounces of beer”), same as “a bottle of water,” “one tank of air,” “a truckload of sand.”

Which is all to say that you’re right. “Less beer” makes far more sense than “fewer beer.”