Do you want to explain to a thirteen year old how it’s not a big deal if someone is passing around realistic nudes of them that were AI-generated?
No of course not. But how's that different than when one of her classmates takes Photoshop 101? What do you tell the 13yo whose face is photochopped onto a porn star? Same thing.
I'm 100% with you that it sucks, and I'm not convinced generative AI has done much of anything useful to outweigh its societal and resource costs.
You could require all cloud AI to embed tracking data in each image, so when little Johnny nudify's his classmate we can trace who made the image. But lots of cloud AI companies are outside the USA. And that won't affect local models.