There is a lot of deliberation over where they should go.
If you say that, then I’ll have to believe you. My standpoint on this is more akin to your next statement,
Music is a language. People in a jam session are speaking words and phrases to each other. There are grammar rules
That means that once the piece of music turns out to be in, say G major, then there is little (I even dare say no) deliberation on whether you’re playing a G natural or a G sharp. There may be deliberation in the phrases you play or in the general direction the piece you’re playing will go. There may even be deliberation in not playing a G natural from time to time. But by default you don’t think about these things. You just play.
This is what I’m comparing to using an AI tool. One doesn’t think about every single note every single time, just like one doesn’t think about ever single filler word that’s only there for grammatical purposes in writing prose, and just like one doesn’t think about every single color channel of every single pixel when creating a digital painting. So why does using an AI tool invalidate the art? Because there is no deliberation over every single tiny aspect of the output?
If you’re using an LLM, then your jam partners aren’t speaking to you, they’re speaking to a robot.
This is just plain wrong. It’s like saying you cannot be a musician if your instrument is MIDI. Or you can’t be a singer if you’re using an artificial voicebox. The LLM still produces what its user intends it to produce. The LLM doesn’t produce output on its own. It is operated by the user. Just like a guitar is operated by its player.