Yeah, but it does take a lot of work to coerce them to do it.

GCC is honestly obnoxious, and you have to do a bunch of unintuitive things to get it. The compile stage needs the built-in RNG seeded, parent file paths stripped, you need to ensure that the date/time macros are not used anywhere, and you need to ensure that all command line flags are passed in the exact same order every time.

I just went through this with GCC16 on a new project.

Not by default usually but yes, you need to do a lot of work to set all necessary configurations and sometimes provide your own RNG seed for things which insist on random looking values.

Not only can they but certified toolchains exist in the safety critical space. Medical, aerospace, nuclear projects, and the like are often required to use and procure such toolchains as part of their validation.

You don't need parallelism to have a race condition, just not handling an event with expected timing can cause one - like when two keys are pressed within one polling cycle and you depend on one being pressed before the other for some logic like up and right arrow for a diagonal but they register as right and up so the diagonal movement doesn't trigger

Compiler optimisation strategies sometimes use statistical machines and link time optimisation does use random number generators for producing output

You can have single threaded race conditions appear simply from inputs appearing in an unintended ordering. Or on a single core CPU, you can have one task meant to be done first take unusually long time, so the CPU gives time to another thread which finishes first but expected to finish last.

From my understanding, which is very limited, race conditions are more an issue with concurrent programming. Parallel computing uses separate processor cores for each task so there's less reliance on stack machines. But I guess each core still shares the memory, so maybe it still happens. Like I said, my understanding is limited. Just use rust.

In short: yes. You can tune these values when self hosting - it basically changes the b chance which tokens will be used under which circumstances.

Like Scipitie said, sort of.

In the ChatGPT app, there's tons of shuffling in the background, like context being injected, maybe sampling changed, agenic action, quantization... its "far" from the actual LLM, opaque by design, and as a result certainly not deterministic.

You can sometimes get deterministic output with the OpenAI API, but it's also dependent on nothing changing on their end. And their end changes a lot.

But self hosting or using a more consistent provider will give you deterministic output.

No. Floating point arithmetic and ordering of operations won’t make 0.0 deterministic.