If you live here you don't really go through Penn station. That's a major commuter hub.
I've lived here for years and only go there if I need to go to NJ transit for some reason (which isn't often).
That's like thinking all of NJ is Secaucus train station, or all of someplace else is just the airport. It's not representative.