>Floating point matrix calculations are non-deterministic.
This is not inherent to floating-point math. That actual (true) claim in the article is that different hardware and different hardware configurations produce different results. But deterministic inference is possible, e.g. llama.cpp on CPU is deterministic by default.
This is not inherent to floating-point math. That actual (true) claim in the article is that different hardware and different hardware configurations produce different results. But deterministic inference is possible, e.g. llama.cpp on CPU is deterministic by default.