I'm sorry but I'm still a bit of an AI doomer. Like I genuinely hope AI is fundamentally slop. But like digital technology is fundamentally to simple to manifest any meaningful safeguards from computers that program themselves.
AI has only begun to scratch the surface of what it can do cyber-security wise.
An AI that's been trained on millions of device models and software versions could build a data base of exploits and select targets exclusively based on hardware with zero day vulnerabilities. Even patched exploits can be cross referenced with software versions so that any out of date hardware is immediately targeted with that specific vulnerability.
...
Our tendency to anthropomorphize AI causes us to assume that only a sentient AI presents an existential threat. But in reality the threat isn't the intelligence of the AI, but the vulnerability of the digital world and everything connected to it. Even a 'dumb'AI with the right data could become a global nuisance.