I get the impression that's supposed to be some kind of gotcha, kinda seems like projection though.

If you don't want to (or feel like you can't) engage on a point, no one is forcing you, it says enough that this is your response.