Just off the dome an ethically acquired data set would be one where the data providers have knowledge that their data will be used for training a model and consent. When I did a deepfake project for a class back in 2019 that looked like me asking my buddies “hey can you sit and read this article on deepfakes in front of a camera so I can try swapping your faces?”

In a commercial setting there are a couple ideas:

1. Train on your own past work. This has the added benefit of ensuring the model will be better at approximating your own style.
2. Stock image sources but for data. Gettysburg has a ton of images, and while they weren’t originally gathered for image generating model training they could start a new collection where they expressly get that consent. If it comes out that there are stolen images in that dataset, Gettysburg is the one liable and should likely need to provide compensation for any training already performed then remove that data from the set they sell.

Frankly niche models is probably where the technology should have gone in the first place. Why invest trillions of dollars into making general purpose models that are kinda mediocre at everything when you could build a smaller, more focused model that only does what you actually care about maybe pretty well.

Seems like you get it just fine.