Personally I used to love it, if the DM did that it inspired players to play; usually whoever had theage would say something like I can't destroy what I can't see and the the fun starts... Someone throws flour from their pack at it (or dirt, oil, something to make the invisible object visable in another way).
I haven't played in over 20 years so I'm sure it's changed a lot but that kind of stuff was fun to me.