I think you explain it pretty well, but one thing to add. Due to the General Relativity and thus spacetime it is actually not directions that all point toward the singularity, but as soon as you cross the event horizon all of your future becomes the Singularity, not as a point in space, but a point in time

https://plato.stanford.edu/entries/spacetime-singularities/lightcone.html

This points at that, you would also need to be able to travel faster than light and that would make you time travel backwards in time