In a blog post published Thursday, the company cited its own internal data as evidence that modern AI systems are nearing the point of “recursive self-improvement”—i.e., being able to refine their capabilities without a human in the loop. “AI that can build itself would be a major development in the history of technology—one that could bring enormous good for the world in science, healthcare, and beyond,” the post, which was written by company cofounder Jack Clark and Anthropic Institute lead Marina Favaro, reads. “But full recursive self-improvement also might increase the risks of humans losing control over AI systems.”

Are they? When our own researchers barely have a grasp on how LLMs work, can't really pinpoint why LLMs make certain choices, can you really expect it to improve itself when it has no sense of understanding? Or will it just make it self even more convoluted and nonsensical until it all falls apart like a house of cards?

I don't know shit about LLMs, so maybe I'm lacking understanding here.