They definitely are not deterministic
Yes, they are.
The two main inputs to an LLM are the prompt and a seed. Pass the same prompt and the same seed, and you get the exact same output every time.
Most commercial models automatically generate the seed using a PRNG and hide it from you by default, but with local models (and also OpenAI, via their API) you can set the seed yourself and keep it fixed.
IMO, this PRNG slight-of-hand is the largest underlying factor that helps convince confused people that LLMs are conscious or intelligent, and it’s very easy to show that’s it’s a complete illusion.