Yeah. As horrible as LLMs are in many ways, finding vulnerabilities is something they're great at.

It's absolutely a good thing that a good actor with an LLM finds a vulnerability before a bad actor with or without such tools.