you are viewing a single comment's thread
view the rest of the comments
[–] 15 points 3 weeks ago (6 children)

Predicting what a compiler outputs is quiet hard, especially if you enable optimization flags or get near any undefined behavior. But I think you're right to say that a compiler is more predictable than any LLM. I don't get the point why you should be unable to audit the output of an LLM? If its used as a tool like described the user should have the knowlage to audit any output

  • source
  • parent
  • hideshow 6 child comments
  • [–] 15 points 3 weeks ago* (3 children)

    I can run a compiler.

    Then I can run it again, and the output will be the same.

    I can run a LLM prompt.

    Then I can run it again, and the output should not be the same. The unpredictability is what makes the tech great - Nobody in decision making actually understands this, but the goal is not for identical outputs. Just like me and you, don't respond to the same question in the same way every time

  • source
  • parent
  • hideshow 3 child comments
  • [–] 1 point 2 weeks ago

    Compilers aren’t all that deterministic without effort. Example, both gc and clang if ran with parallelism can produce different results purely off file system lookup differences alone. This means intermediate archive files can be equivalent in content from the perspective of functions built but differ in order. This can easily filter up to the final binary being different. Making compilers bitwise deterministic is a lot more effort than running the same compiler on the same input. This is ignoring things like timestamps and other common sources of non determinism. But I can say the Debian reproducible build project has most of this figured out but compilers are not as deterministic as your comment implies. They can be, but you should really check that your final binary is bit for but the same. If you’ve used any parallelism it likely isn’t.

    If we then bring in lto et al it gets worse.

    Let’s take just gcc, are you setting -frandom-seed, SOURCE_DATE_EPOCH, -file-prefix-map, are you ensuring the input file list is deterministic? If you’ve used any can’t affirm yes to at least these 4, note these are just the things I can remember, you do not have a deterministic compile chain.

  • source
  • parent