I’ve been playing around with mapping steam games + their reviews based on Qwen embeddings. I revisit this project every year around this time to see what improvements I can make to make a “better” map than last year (According to my own preferences). [0/1]
One thing I learned this year is that naive text mapping (embed a corpus of reviews with a Qwen embedding model, use umap to plot, then compare where things fall) primarily compares embedded text length more than anything else. What a review says affects the position less than how long the review is.
This was a little surprising to me. For whatever reason, I didn’t think the embedding would so strongly describe the length of the embedded passage. So most of this year’s time was spent fixing that artifact.
This is an interesting idea, do you find it has more interesting properties than doing cosine similarly by reviewers? Have you found things adjacent to games you enjoyed that were also fun to play?
Yes, I’ve actually found a few interesting games I had never heard of before! So the project has been a success for me already.
But there’s still plenty of places where games are placed strangely on the map, and I like to investigate why that’s happening (it’s usually because Steam reviews are pretty noisy signal).
The first version of this idea was actually based on cosine similarity. The user would enter arbitrary text or an existing steam app id, and would get a list of games back.
The arbitrary text results were never very good because to get related results you needed to write a fake review of the kind of game you were looking for, which is a difficult concept to explain to users. Most users just entered text like "beautiful game", and that lead to bad results. "Not a beautiful game" winds up very close to that.
The results based on steam app id were much better, because it was comparing like-for-like. That lead to me considering different ways of presenting the games based on their review similarity. And so this map was made.
The umap projection does introduce a fair amount of distortion, so it's not a strictly better result than cosine similarity between specific Steam games. I'm still playing with the umap parameters to try and find something which causes the least problems.
was one of very few useful results of the first decade of TREC research... And the main thing it addresses is balancing various factors which could cause a full text search engine to privilege either long or short documents.
In general, ranking relevant keywords in the context of the Negative reviews will likely have more hits the longer people complain.
The random fluctuations of individual experience only cancel out bias if and only if the results are truly unbiased normal distributions.
In the case of positive:negative ratio feedback scoring, it only makes sense if you ignore the entire message body of what people enjoyed.
Negative-weighted ratio ranking more accurately describes the end users experience. Any game with paying unique customer greifer total scores under 15% without game-play reliability related complaint keywords is going to be a fairly good game if the genre matches your profile preferences.
Otherwise, people will just see shit scores from the response bias of pissed off users on dodgy OS/Networks. Would recommend checking your conclusions with the average current active players in the SteamDB plugin. If users aren't engaged with the paid gameplay experience 6 months after launch, than the release likely isn't fun or broken garbage. =3
A good game, but takes some skill to play (4270 daily players 10 years later):
I have played with similar problem in different domain, and if you could have “which users play which games” (or in general “which articles are read by which people”), you can do really good similarity search and self organizing map based on no content at all, just the metadata.
Which is of course why Facebook can say they don’t read your conversation content, while still be able to target ads precisely based on conversion metadata.
Not sure how related the cause is, but the fix seems to apply too.
In single cell genomics you often have a large correlation between the data and the length of the sequencing reads.
You can calculate the correlation between your metadata and every PC dimension and exclude the highly correlating dimensions before doing the umap. Similar can be done for other metrics. I've seen it done for patient gender for example. I could imagine looking at the correlation for publish year or genre.
One thing I learned this year is that naive text mapping (embed a corpus of reviews with a Qwen embedding model, use umap to plot, then compare where things fall) primarily compares embedded text length more than anything else. What a review says affects the position less than how long the review is.
This was a little surprising to me. For whatever reason, I didn’t think the embedding would so strongly describe the length of the embedded passage. So most of this year’s time was spent fixing that artifact.
[0]: 2025’s map: https://storage.danieltperry.me/share/steam-map/
[1]: 2026’s map (still a work in progress, both the ui and the review processing are incomplete): https://storage.danieltperry.me/share/steam-map-refresh/
(Edit: They're live now, you'll need to hard refresh/clear your cache to get the updated HTML and JS)
But there’s still plenty of places where games are placed strangely on the map, and I like to investigate why that’s happening (it’s usually because Steam reviews are pretty noisy signal).
The first version of this idea was actually based on cosine similarity. The user would enter arbitrary text or an existing steam app id, and would get a list of games back.
The arbitrary text results were never very good because to get related results you needed to write a fake review of the kind of game you were looking for, which is a difficult concept to explain to users. Most users just entered text like "beautiful game", and that lead to bad results. "Not a beautiful game" winds up very close to that.
The results based on steam app id were much better, because it was comparing like-for-like. That lead to me considering different ways of presenting the games based on their review similarity. And so this map was made.
The umap projection does introduce a fair amount of distortion, so it's not a strictly better result than cosine similarity between specific Steam games. I'm still playing with the umap parameters to try and find something which causes the least problems.
https://en.wikipedia.org/wiki/Okapi_BM25
was one of very few useful results of the first decade of TREC research... And the main thing it addresses is balancing various factors which could cause a full text search engine to privilege either long or short documents.
The random fluctuations of individual experience only cancel out bias if and only if the results are truly unbiased normal distributions.
In the case of positive:negative ratio feedback scoring, it only makes sense if you ignore the entire message body of what people enjoyed.
Negative-weighted ratio ranking more accurately describes the end users experience. Any game with paying unique customer greifer total scores under 15% without game-play reliability related complaint keywords is going to be a fairly good game if the genre matches your profile preferences.
Otherwise, people will just see shit scores from the response bias of pissed off users on dodgy OS/Networks. Would recommend checking your conclusions with the average current active players in the SteamDB plugin. If users aren't engaged with the paid gameplay experience 6 months after launch, than the release likely isn't fun or broken garbage. =3
A good game, but takes some skill to play (4270 daily players 10 years later):
https://store.steampowered.com/app/219990/Grim_Dawn/
A similar genre game, but people score it higher (3650 daily players 1 year later):
https://store.steampowered.com/app/1145350/Hades_II/
A top ranked genre game, with a nostalgic series scoring bias (9651 daily players released this year):
https://store.steampowered.com/app/2536520/Diablo_II_Resurre...
Which is of course why Facebook can say they don’t read your conversation content, while still be able to target ads precisely based on conversion metadata.
In single cell genomics you often have a large correlation between the data and the length of the sequencing reads.
You can calculate the correlation between your metadata and every PC dimension and exclude the highly correlating dimensions before doing the umap. Similar can be done for other metrics. I've seen it done for patient gender for example. I could imagine looking at the correlation for publish year or genre.
Edit: fixed a misconception