Hacker News

Favorites Setup
Comment by Scene_Cast2 | original | What thousands of conversations look like inside a language model
[−]Scene_Cast2 · 2026-11-15 Sun 13:46 UTC · link
The methodology feels either heavily undercooked to the point of being misleading, or heavily under explained. Why are we in 3D? At which point did dimensionality reduction hit? And if we're doing dimensionality reduction, does the "plane" really mean anything?
[−]vhold · 2026-11-15 Sun 13:59 UTC · link
It's under explained. It's in 3D because of PCA. It mentions PCA all over the place, but never even says what it is, less alone briefly describe what it does.

https://en.wikipedia.org/wiki/Principal_component_analysis

There are a lot of more fancy ways to reduce dimensions and create visualizations like these that tend to preserve more of the higher dimensional structure.

Here's a semi recent paper discussing and comparing some of them, I didn't dig too deep to try to find a more recent or better one : https://pmc.ncbi.nlm.nih.gov/articles/PMC12453773/

[−]Scene_Cast2 · 2026-11-15 Sun 14:11 UTC · link
I'm very familiar with PCA. There are so, so many ways you can apply it to this problem.

Assuming the simplest interpretation of what the analysis did, then it's saying "the variance of the biggest variance direction stays constant". Which in itself is an interesting thread to tug on, but the charts aren't how you'd present such a result.

[−]thece0 · 2026-11-15 Sun 14:12 UTC · link
Not familiar enough it seems. But PCA is outdated anyways
[−]oldjohnnylane · 2026-11-15 Sun 15:18 UTC · link
There’s no meat here. It’s just someone throwing data into a plotting tool with no meaningful results or conclusions. I was expecting a meaningful analysis. Nope.
[−]pohl · 2026-11-15 Sun 18:34 UTC · link
> Why are we in 3D?

Isn’t there an obvious answer to this? It’s about visualization. There’s a natural upper limit to dimensions we can see, and 3 is where we hit it.