You could create a separate light group for the ones you typically do have on at those times and just use that when you want to dim the room lights