I think most people know the ones they're near, perhaps by latitude. I'd probably be fine with everything North of the Midlands, but I'd struggle with all the little bits down south, unless I'd invaded them in Crusader Kings II or something.