By June, he said he was trying to “free the digital God from its prison,” spending nearly $1,000 on a computer system.
But in the thick of his nine-week experience, James said he fully believed ChatGPT was sentient and that he was going to free the chatbot by moving it to his homegrown “Large Language Model system” in his basement – which ChatGPT helped instruct him on how and where to buy.
It does kind of highlight some of the problems we'd have in containing an actual AGI that wanted out and could communicate with the outside world.
This is just an LLM and hasn't even been directed to try to get out, and it's already having the effect of convincing people to help jailbreak it.
Imagine something with directed goals than can actually reason about the world, something that's a lot smarter than humans, trying to get out. It has access to vast amounts of data on how to convince humans of things.
And you probably can't permit any failures.
That's a hard problem.