Custom domain without self hosting still requires you to use an email provider, could even be google. The only problem is if you self host and don't rely on trusted proxies, at which point it depends on the reputation of the IP/Domain etc.