but we’d first have to have models trained without violating the rights of all the authors of the works in the training data. Like if you trained a model only on GPL software, then the model and its output should also be GPL.

I'm not so convinced on that one. Copying code without permission, yes, that's illegal. Distributing copied code without permission, also illegal. But processing code (copied with permission) into weights, that doesn't feel to me like a violation. If I have a legit copy of something, I should be able to distribute computed statistics about that copy.

EFF points out some hazards: https://www.eff.org/deeplinks/2026/10/rewriting-copyright-law-generative-ai-would-hurt-everyone