No - they don’t.

I really don't understand what you mean. Off course they are compiled, otherwise they cannot be executed. Script language does not mean "run without compile" (whatever that would mean). It means, compile the moment it is needed. Python compiles the entire code in one step, while JavaScript compiles only parts of the script the moment it needs.

The source code is read and executed on the fly.

By compiling on the fly.

Python is traditionally only interpreted AFAIK

And what do you think "interpreted" does mean? It means it compiles the code when it needs. Shaders work like Python, you distribute the source code and the code is compiled on demand.

If there was a GPU shader-interpreter that could skip the compilation step at the cost of execution speed

GPU Shader compilers are interpreters, like with Python. Python interpreter produces a Bytecode cache that only needs half of the compilation process on your specific machine.

For short, the term Interpreter I think is misunderstood here. An Interpreter just means it compiles the code right now on demand, and executes it right now. While a Compiler compiles the code ahead and produces code that you can execute later. JIT is an Interpreter that only compiles parts of the code that is needed right now.