Default passwords, old insecure versions of apps and system packages, etc. "Just getting it working" usually leaves things insecure, and you usually need to take things a step further to secure your publicly accessible services.