Yeah, somewhere along the line you end up with a question of trust. "Do I trust the developer of this AppImage?", "Do I trust the result of this automated tool that checks the code for malware?" or "Do I trust my IDE and myself when I downloaded the source and tried to verify it in my sandboxed VM?".
My main point was that the hash doesn't really tell you anything about the source, except whether you got an exact copy of it or not.