The only time I ever exposed my SSH port to the WAN, I got about 750 connection attempts from China and 250 attemps from Russia per hour.
post
tldr: disable password auth in ssh.
Also name misses the point I think, it's a musl-hole.
Given this is a vuln when entering the password, just turn off password auth and use ssh keys only. It's what you should be doing anyway.
Easier said than done.
If you have to support the sort of user that barely managed to download a copy of putty a few years ago and kept using this, then walking them through key generation and use is a nightmare.
One could fairly argue that maybe such people shouldn't have access to a system, but it is a reality.
Of course the likelihood that a system using musl is something that class of user would be accessing is lower.
Maybe now that ssh is generally "just there" for Windows users, you can more easily work with them to generate a key, or give them a zip file to make them have a key in a standard way. But the interaction of ssh and windows file security can be... Interesting, though mostly an issue on the ash server side.
One could fairly argue that maybe such people shouldn't have access to a system, but it is a reality.
More that, such people shouldn't be allowed to be sysadmins, yet self-host and/or expose any ports to the internet.
Yeah, issue being if they work for your company and management insists they have a job to do and you must provide what they ask for to do their job.
Though the most recent guy to inflict that on me is at risk of pissing off a customer as he pulled that on them and they are complaining rightly that someone that can't sort out key based ssh auth has zero business touching their sysems. So some people can give reasonable push back.
The vuln is specifically for key-based accounts:
Second, the target system account must be configured to be passwordless. Passwordless accounts are commonly used for accounts that use certificate-based authentication.
"passwordless" here means that the account doesn't have a password. What the person you're replying to means is that password authentication should be completely disabled in the SSH config, as that's the only way to exploit this issue.
But passwords need to be enabled for SSH. You must submit a faulty password of a specific kind, and the SSH server misinterprets it for passwordless accounts, which means the server must be configured to allow you to attempt a password.
not true. regardless if all your user accounts are using key based auth or not, some system accounts will always be passwordless, because they are not supposed to be logged into.
Fuck. OpenWRT might fit that bill.
Edit: it does, kind of:
https://forum.gl-inet.com/t/sshole-exploit-does-not-impact-gl-oem-firmware/72657
Can they please take a break from having vulnerabilities
nah, find and fix as many as possible.
It seems like the pattern emerging is that AI is great at finding these obscure vulnerabilities that are only exploitable on very particular systems with very particular, non-typical configurations, and only on nights where unicorns fart candycorn.
Sure, fix them, but the hype is too much. Also, disable password auth on SSH already. Jesus. Keys arnt that hard.
I agree about the AI, but desktop Linux has never been under haxxor pressure the same way windows or apple has. I also think the newer parts of the userbase are less technical than even ten years ago. Previously undocumented bugs found and squashed is great for everyone.
More like AI is much faster at finding obscure vulnerabilities, because it doesn't get tired of digging into the mind-numbing details of human-generated spaghetti code. Give a human the same kind of task, and they would die of boredom 12 hours into it.
We're entering an age where AI is quickly cleaning up the stupid security holes that have been created by decades of human mistakes, and will continue to be this way until increased AI security reviews will become more standard. The tribals wielding spears will continue to get shot by superior firepower until they adapt to the same weapons and protection themselves.
"Cleaning up" seems a little generous. It's finding them, but someone still has to patch them.
"Finding" is also rather generous, going by numbers from Anthropic and from the Linux kernel, things like Mythos are wrong 95% of the time.
They're much easier to run though, compared to more reliable tools. So even being wrong 95% of the time wins out if you've got 100x more people running them.
Most vulnerability patches are a few lines of code. Finding it is the hard part.
Even then, the AI session that found it probably already has a solution to fix it.
In my experience, the LLM usually proposes a solution, but it's a crapshoot whether it works well. And plenty of bugfixes go well beyond a few lines, especially in older code.
I use this shit on a daily basis at work. They work well, as long as you're choosing the right models, making the right engineering decisions, and have a good set of baseline rules.
Do you have a guide on which baseline rules are good? I use Claude Opus 5.5 at work and while it's definitely more helpful than not having it, it also does some incredibly baffling things that a human would realize immediately are unproductive.
"Baseline" is subjective, so I'd recommend just start a global rules file and steer it away from the behaviors you don't like. Don't be afraid of quizzing the agent itself on why it did that thing, and what's a good bit of language to add to the rules file to prevent it from happening.
Also, for specific tasks, skill files are really useful, and LLMs can write the whole thing, based on what you did the hard way in a session. Same thing for triggering a skill file and then ask it if there are any useful improvements to make to the skills file in the future, at the end of the session.
Yeah and this isn’t a new paradigm, in the sense that tools to find exploits have gotten better repeatedly and allowed those with enough knowledge to use the tools to find them, regardless of if they could manually craft the exploit themselves. There’s a multitude of bug bounties that have been claimed by those who knew how to let a reverse engineering tool like ghidra hammer away at something with variation for long enough. And the models are capable of novel exploits on their own, but generally they are just doing the same, hammering ghidra with an MCP at accelerated scale. So “only ai would have found this bug” is technically untrue and moot.
This example demonstrates my point though. It requires an extremely unusual system configuration that may or may not even exist in the real world it is so unusual. AI doesnt get tired, like you said, but its not about spaghetti code (both musl and openssh are gold-standard opposite of spaghetti code), its about trying every possible software combination in ways a human wouldnt bother.
It requires an extremely unusual system configuration that may or may not even exist in the real world it is so unusual.
You have to remember that the same AI agents finding the code are also able to exploit the code. So, if it's some extremely rare race condition requiring some convoluted pattern, it's still easy to exploit.
When I stack dominoes in a line on the floor and I push one of them on the outside as it is in the shape of a snake... they fall down in a prdictable pattern. I never knew that would happen until I was shown the first time. This knowlege was past down via print made by humans scraped by bots. For what and for who? Louder faster and dumber.... on a foundation of jUsT gEt iT d0Ne... now you are fired. Loops, poops and dupes.
I think you're buying snake oil, my friend. What is "AI"? Statistical tools have been used in security auditing for decades.
The people riding the AI bubble want you to think things are magically different right now, but we won't know that until we can get some solid data over the next year or five to do a proper comparison.
This particular vulnerability is precisely about logins with key. Using password login renders this vulnerability useless.
Using password login renders this vulnerability useless.
This isn't the case, though I can see how the article could lead you to believe that.
It's directly based on password rather than key input. The entire vulnerability is that typing a password longer than 256 characters automatically defaults to crypt() outputting the * character instead of a normal password hash, which is coincidentally the same default character used as the "hash" (not actually a value you could get from a normal hash, intended to be impossible to get from any normal password for that reason) for the built-in daemon account.
When the Ars Technica article says "Second, the target system account must be configured to be passwordless", what they mean is the default daemon account, which has a hash set to * by default, and thus wouldn't normally be accessible via any hashed password... unless you happen to find out that the system chooses to not make a hash, and just send * if you type more than 256 characters.
Look at https://sshhole.com/ if you want the details.
Technically they're correct though, in a way. If every account has a password, the system isn't vulnerable.
You're also not vulnerable if you use two factor auth, because all 2FA modules require PAM, whereas this vulnerability only exists when PAM is disabled or not installed.
Other way around. This requires a passwordless account configured with sha256 or sha512 password hash, musl system with openssh installed, no PAM, and password authentication allowed:
When logging into a passwordless account, the attacker can gain access by inputting more than 256 characters in the password field.
Is the name of the vulnerability literally *sshole?
Yes, it’s actually a very creative name. It’s caused by the character * allowing anyone to log into an open SSH port. So it’s an amalgam of {*}{SSH}{port (hole)}.
The long story short is that if you don’t have a password set for an account, Alpine saves the user’s password hash as “*” because no password hash will ever match it. Should be safe, right? Any password will always automatically fail, because the hash algorithm cannot return “*” as a result. So the only way to log in as that user is via something besides a password.
And when musl (the underlying system that Alpine uses for SSH) gets a Null response from its password hash, it uses “*” to represent that null value. Because lots of programs throw a fit if they get a true null value. So instead of returning a null value, it just returns a *. Which shouldn’t match anything, right?
So an attacker forces the password hash to return a null value. Simple to do, because the password hash is capped at 256 characters. So just enter something with more than 256 characters. Musl takes that null value, turns it into a *, and passes that onto Alpine. Alpine takes this *, and compares it with the list of saved usernames and password hashes. And it finds that the user (who has no password set, and shouldn’t be allowed to log in with a password) has a password hash of “*”. This matches the result musl just provided, so the attacker is able to log into the system.
* ssh olé!

practically need to rotate logs every hour if you foolishly leave ssh open on a open network. usually a bastion with ssh listening to port 34000 or such is a good first step.
that said the conditions for this exploit are fascinating. no pam, shadow password set to *, using a alternative to glibc. The sacrifices to be more efficient feel on par with replacing a tank's armor with paper mache and hope.
The best armor is to not get hit!
I got into it with another sys admin on shifting port numbers. "Security through obscurity is not security!" to which I responded "Obscurity cuts out most of the bots" (therefore easier to spot real threats).
Eventually we just went the secret squirrel bastion route.
Well that sucks. IIRC a lot of my docker images are alpine based, but they'd be safe due to never having their SSH ports exposed I assume.
Why would you even have an SSH server running in a Docker container (unless the purpose of the container is being a jumpbox or similar)?
I can't emphasise it enough how much more secure you get when you use rootless, read-only containers that are limited to truly just the software you want to run and its dependencies.
Alpine is great for dev containers, but the final container should be:
- immutable on the app layer (any executable path needs to be read only)
- have only the absolutely necessary GNU tooling, service management, etc. included (for most software you don't even need these tools, and why would you give an attacker who gain access to your container, an easy way to e.g. scan your network?)
- only the data mounted, rw only if absolutely necessary (e.g. if you host a Jellyfin server, just an example, you should have the mounted media be RO, and only the config and caches be writable, mounted noexec of course)
While I have not done it, I could see it being appealing as it's quicker than SSHing into the parent box first for a commonly used tool. I had a direct SSH into a systemd container for a while before deciding the extra config wasn't worth it.
Not just rootless and read-only, but distroless/chiseled too. There's no reason to have unneeded parts of an OS in your Docker containers. It should just be one app along with any libraries it needs (or just statically compile the app)
unless the purpose of the container is being a jumpbox or similar
What's wrong with a rootless jumpbox?
Further, OpenSSH using musl (pronounced muscle) must not use PAM (pluggable authentication modules),
which Alpine doesn't ship by default. Maybe one or the other remote administration suite depends on it, but those are often suspectible to exploits anyway. But you don't expose either to WAN, right?
RIP. I am glad I just walled off my own garden from the internet. Though I dont think I ever had a need for my own stuff to have exposed ssh ports. That sounds scary before I knew about this.

top 98 comments