you are viewing a single comment's thread
view the rest of the comments
[–] 3 points 8 months ago* (last edited 8 months ago)

I assume what you're suggesting is that we can always "pull the plug" or smash the computer if it gets too smart and starts making threats.

While technically true, I think it both assumes and overlooks a lot. You might not be able to do that once the system has gotten internet access and potentially made thousands of copies of itself. A sufficiently intelligent system might pretend to be dumber than it really is while you still have it air-gapped in a lab - and even if not, we don't really have the capacity to imagine just how convincing a true AGI could be. It could try to bribe you or make more terrifying threats than you can even think of.

There's also this one example (for which I unfortunately can't find the source) where a journalist was questioning whether AGI could truly escape like that. So they made a deal where the journalist acted as the AI scientist and the other person played the AGI. It didn't take long until that journalist, according to the rules of the game, posted on his social media that he let the AGI escape. They even replayed the game, and he let it escape again. It was never revealed how this happened, but suffice to say that if even a human can do it, then it's not going to be an issue for AGI.

  • source
  • parent