LLMs are notoriously non-deterministic, and even at temperature zero you still can't predict exactly where the weights will take you next.
An LLM can be made to be completely deterministic. I use them in this mode so I can reproduce test cases. Of course it requires complete control over the model, etc. but this myth that a computer program is non-deterministic needs to end.
You can 100% predict where the weights “will take you” given a set of inputs.
By "can't predict exactly where the weights will take you next" I meant with your brain. The blind chess analogy suggests you can predict, using your own thought process, the exact output of a prompt.
>Floating point matrix calculations are non-deterministic.
This is not inherent to floating-point math. That actual (true) claim in the article is that different hardware and different hardware configurations produce different results. But deterministic inference is possible, e.g. llama.cpp on CPU is deterministic by default.
You can 100% predict where the weights “will take you” given a set of inputs.
Do you mean reproduce?
Sorry it's just if you are saying what your statement implying then either the model is very simple, or you've figured out something incredible
[0] https://arxiv.org/html/2506.09501
This is not inherent to floating-point math. That actual (true) claim in the article is that different hardware and different hardware configurations produce different results. But deterministic inference is possible, e.g. llama.cpp on CPU is deterministic by default.