Factoring is not in any category because we don't have a proof for it. As it stands, the algorithms we have require exponential time in the worst case, but solutions can be verified in polynomial time, which is NP-complete. I said "at most" for a reason.
Also, every algorithm is verifiable. If there's no better way to do it then re-running the algorithm and that algorithm is exponential, then it's NP-Hard.