Did you discuss wether you'd stay? Did he just leave without saying anything? I mean if I invited someone in and left the presumption is that they're invited until I kick them out. As for speed, you do you. Don't make any decisions you can't take back, don't have a child, get married, unprotected sex, don't share your banking info yet, etc. Other than that, if there's a problem than you guys can just talk it out. He will let you know if it's moving too fast for him.