I’m not worried about AI going rogue and destroying society. I’m worried about what human assholes out in the world can do with AI.

It’s not hard to imagine how AI can have destructive powers. In Europe there was recently news about someone hacking into freezer shelves of grocery stores and turn their temperature to max. One of my local grocery stores got affected had to throw out lots of their food due to this. Not saying it’s due to AI, but I wouldn’t be surprised if some AI was involved.

This kind of attack on our food supply chain and other parts of our infrastructure could be even more large scale. Not because of some rogue superintelligence, but because some human asshole with AI is able to brute force billions of combinations of preexisting ideas faster than previously possible.

Putting the models out in the open won’t help a tiny bit. Even if it’s possible to perfectly align open models so they refuse to do these sorts of attacks, anyone could remove these safeguards by modifying its weights or train them on new data.

The answer is: ban development of frontier models until we better understand how to regulate this technology. But given the incompetency of politicians, I doubt this will happen before it’s too late.