> Early LLM models would spit out my code almost verbatim. Modern systems borrow heavily on the creative design of those systems.
Do you have some specific, actual examples you can share?
I’ve been hearing this exact claim for years, but every time I ask for a specific example, any example at all, I can’t seem to find anything more than a highly generic snippet of code or a coerced continuation.
Emailed the author of the Rust book Zero to Production a year or so ago about how his examples were being used verbatim in the output of AI systems. I own the book, yet I could inadvertently get any of its examples just by asking an AI about one of the book's concepts.
He basically said, "I know, there's nothing I can do."
This is a sparse matrix library, IIRC. Numerical linear algebra and applications using it are black art at this point. I have written a Boundary Element Method evaluator for my Ph.D., and know what it entails to work with these things.
Doing fast operations on sparse matrices is a cutthroat race and when you can shave your hot path by seconds, your solution can save hours in some cases. So, it's an interesting niche. i.e.: It's no ordinary boilerplate, and worthy of publications in academia.
While I worked with dense matrices; the effect, difficulty, blood sweat and tears are still there.
When a model can give back your hard work back to you with a "completion" without attribution, that's bad.
Edit: Somebody just posted this below [1]. Added here so it doesn't get lost as the discussion grows bigger.
When you start to work with low level constructs like that, there really is no room for "artistic expression". Meaning if you want to write a piece of code that does something _optimally_ and correctly, you really, really only have one way to write it, especially in a language like C. Yes, formatting might change, and you could try to force some code acrobatics to make the code look different, but at the end of the day the code will look exactly the same. If you look at Linux socket opening code for various libraries, nearly all of them will look identical because you just need to do the same exact things to make the operation work properly. Same applies to fast math code. Any artistic liberties will almost surely cause performance regressions compared to the optimal solution, unless the LLM comes up with something truly unique. So if you task models to output high-performance low-level math code, most of them will likely spit out something from their training data near-verbatim rather than pull off code gymnastics and invent something new.
Sorry, I'll disagree. When you write high performance code like this, there's no artistic expression, but different approaches.
These algorithms' design starts on paper, as pure maths. You get the analytic form, and convert this to numerical/discrete form and start to design from there.
You need to add normalization, subnormal number prevention, maybe some architecture specific optimization if you are targeting a specific CPU generation, etc. Then it gets optimized further as you run tests, profile the code and run your test cases.
So, no BLAS package looks the same, because they make different trade-offs, have different programming models, and handle things differently, at different points. Even more so, a newer generation processor with much higher FPU performance can have lower performance than previous generation, so you implement another copy of that function or add to detect that processor and have different paths just for that one.
IOW, when doing numerical maths on processors, esp. if you're doing this on many machines with MPI or multiple cores with hand rolled code or OpenMP, there's no single way. Maybe a best way for the specific CPU you have, but it still depends on the trade-offs and design choices you made in the very beginning.
[−]londons_explore · 2026-11-15 Sun 18:57 UTC ·
link
I disagree.... Plenty of fast code in C is jam packed full of macros to try and help the optimizer produce the exact sequence of operations the writer had in mind, whilst still keeping the code readable.
The design of those macros is more art than science.
Thats pretty much everywhere or was in raw form in the beginning. I used to do a fun exercise of searching for the explicit human comments for verification. Comments can be considered novel and unique, so you pick up some comment from popular open source project and then describe LLM for a similar functionality and add part of the comment. The response would have the complete comment for almost 70% of the time.
By “forced continuation” I mean the type of example that was popular for a while where someone would feed a lot of text into LLMs and then ask it to continue the text, then be surprised when it some times predicted the next tokens identical to the original example.
Do this enough times and you can find “examples” that prove your point, but it’s highly automatable so people were using it like a million monkeys at a million typewriters.
Do you have some specific, actual examples you can share?
I’ve been hearing this exact claim for years, but every time I ask for a specific example, any example at all, I can’t seem to find anything more than a highly generic snippet of code or a coerced continuation.
He basically said, "I know, there's nothing I can do."
This is a sparse matrix library, IIRC. Numerical linear algebra and applications using it are black art at this point. I have written a Boundary Element Method evaluator for my Ph.D., and know what it entails to work with these things.
Doing fast operations on sparse matrices is a cutthroat race and when you can shave your hot path by seconds, your solution can save hours in some cases. So, it's an interesting niche. i.e.: It's no ordinary boilerplate, and worthy of publications in academia.
While I worked with dense matrices; the effect, difficulty, blood sweat and tears are still there.
When a model can give back your hard work back to you with a "completion" without attribution, that's bad.
Edit: Somebody just posted this below [1]. Added here so it doesn't get lost as the discussion grows bigger.
[0]: https://x.com/docsparse/status/1581461734665367554
[1]: https://news.ycombinator.com/item?id=50509252
These algorithms' design starts on paper, as pure maths. You get the analytic form, and convert this to numerical/discrete form and start to design from there.
You need to add normalization, subnormal number prevention, maybe some architecture specific optimization if you are targeting a specific CPU generation, etc. Then it gets optimized further as you run tests, profile the code and run your test cases.
So, no BLAS package looks the same, because they make different trade-offs, have different programming models, and handle things differently, at different points. Even more so, a newer generation processor with much higher FPU performance can have lower performance than previous generation, so you implement another copy of that function or add to detect that processor and have different paths just for that one.
IOW, when doing numerical maths on processors, esp. if you're doing this on many machines with MPI or multiple cores with hand rolled code or OpenMP, there's no single way. Maybe a best way for the specific CPU you have, but it still depends on the trade-offs and design choices you made in the very beginning.
The design of those macros is more art than science.
You’re disqualifying any example that people may provide preemptively. No wonder you’ve missed them.
By “forced continuation” I mean the type of example that was popular for a while where someone would feed a lot of text into LLMs and then ask it to continue the text, then be surprised when it some times predicted the next tokens identical to the original example.
Do this enough times and you can find “examples” that prove your point, but it’s highly automatable so people were using it like a million monkeys at a million typewriters.